One-time file-sharing

Say you rent a box somewhere on the Internet. You installed Debian stable on it because you want it to be nice and stable and run a few daemons that are useful to have online. Could be to hold your vast music collection, family pictures, or use it as remote storage for backup. Imagine you wanted to share some of the files hosted on this box with your relatives, who may or may not be computer-literate. Most of them would know how to use a webmail but asking them to install an ftp client is just beyond reach.  Obviously, you do not want to give these guys too many rights over your box (like an ssh access for scp). What are the solutions?

Setting up a dedicated HTTP server

Simple enough: set up an HTTP server to distribute static files. lighttpd is simple enough to setup in a couple of minutes and is very efficient for static stuff. But you do not want to distribute your files to the whole Internet. Sooner or later a web spider will crawl in and index your family pictures and all sorts of things you never meant to be public.  Next step: configure password-protection on the server

Fair enough. Now you have limited file downloads to people who know the password -- provided they know how to enter a password. Do you create multiple accounts, one for each of your peers? It would be preferrable, otherwise you will never know who downloaded what. But then you have to communicate their passwords to your peers and make sure they have a procedure in case they forget it. You know you are headed straight to massive butt pains.

Second issue: passwords can be shared. You shared that 2GB movie with a couple of friends and a couple of weeks later you find out that there are currently 1,549 active downloads for this file. Sharing is in human nature and that is completely Ok, but you probably did not sign up to become a content distributor over the whole Internet, only with a couple of friends and relatives.

Next step: use one-time authentication

There are better solutions out there: since you only mean to share one single file (or set of files) each time, you do not need to create accounts for your friends. You give them a one-time download token and forget about it.

A one-time download token is a URL. It looks like the kind of URLs you get from URL shorteners with the funny string at the end. Something like http://shortener/12398741

One-time tokens can be shared but since they can only be used once, the person who shared it has lost it. The token is randomly generated and invalidated immediately after it is used to avoid having robots automatically scan all possible URLs in a row until they find a valid one.

There are many ways to achieve this on regular HTTP servers. Apache probably has a million configuration options for user authentication, including one-time passwords or something similar, but I have to admit I did not even try. I already wasted enough of my life in Apache config files. lighttpd can be configured to do that but the only solution I found required some Lua scripting and I did not feel up to the task.

Next-step: Do It Yourself

After reviewing countless pages of configuration options for various HTTP servers, I decided that it would be shorter for me to implement this in a tiny web app rather than try and understand complex configuration options.  My first iteration made use of a Python FCGI script in web.py attached to a lighttpd process. Pointing out static files from a Python web app to the embedding lighttpd process is reasonably simple.

This implementation suffered from a number of pitfalls though. For one thing, performance was bad. For some reason, the Python process would eat insane amounts of CPU and RAM when sending big files, slowing down the server to a crawl. Second showstopper was the complexity involved for such a simple setup. I had to write a Python script to generate the lighttpd configuration file with a number of deployment options: where to put config files, log files, static files, port number, etc. And then came the inevitable issues with dependencies: Python version versus web.py version versus lighttpd version.  Some combinations worked fine, some did not.  Nothing specific to Python or lighttpd, but the more you have gears, the more you have places for grains of sand to fit in.

I still survived with this setup for a year or so, when Go came in. I have already reviewed the language in the past and will not come back to that, but suffice it to say that developing HTTP servers in Go is the most natural thing. Adding the one-time token ingredient to the soup was implemented in just one evening.

Once rewritten in Go, I found out that the end-result was about just as big as the Python implementation, excluding the script that created the lighttpd config. The main difference was of course that I do not have to maintain cross-references between package versions for Python, lighttpd, and web.py, since there is only one dependency to cover: Go itself.

It was straightforward to enhance the program to support more options, respond to favicon, and handle a JSON-readable database for active tokens.  Performance is astounding. The serving Go process never takes more than a few megs of RAM (about the size of the executable itself) and only uses tiny amounts of CPU since the process is mostly I/O based anyway.

There is one thing I should have foreseen and had to re-implement. I am sending the one-time links by email and more and more people are reading their emails from their smartphone or tablet. Many just clicked the link without thinking twice, triggering a 2-4GB download and killing both their mobile and data plan at the same time. Wrong move.

The next version features a two-time download page: the first link sends users to a page summarizing the download, offering a second link to actually start the real thing with a big warning about the size of what will actually be sent.

There are many other features I would like to add to the current version, and I am hoping other people have better ideas for new features, which is the reason why I shared it on github. Find it here:

https://github.com/nicolas314/onetime

Since we are talking about sharing private date between friends and relatives, protecting the download may be a good idea. A recently added feature was support for HTTPS. You only need to point your config to server certificate and key files and off you go. The HTTP/HTTPS thing is completely handled by Go.

The resulting program is far from top-quality but it fulfils the needs. Go give it a try if you want to. Careful though: it will only work on Linux boxes for now.