http://benderydt.com

Nigel Cooper wrote an article called The Art of White Balancing wherein he argues that it’s best not to use manual White Balance but use the factory presets instead. His case is founded on three points:

1. White balance was invented as a way to counter color drift in old analogue tube based cameras. Modern cameras are perfectly stable, they do not need constant calibration anymore.
2. Perfect white doesn’t exist in real life. You’ll always see some kind of tint in whites, be it amber, eggshell, or blue. Sunsets are perfect example of a situation where manual white balance will wreck the atmosphere.
3. Manufacturers spend a lot of effort on getting the skin tones right in the factory presets. You wreak havoc on this delicate colour balance by using manual white balance.

I really don’t know what to think about this.

If you’re working with video and the title of this post means nothing to you, you should check out Graeme Nattress’s page on Y’CbCr chroma sampling.

Yesterday I wrote a post about the problems with BitTorrent and trackers and today I find out that BitTorrent has gone trackerless.

The basic idea is that the list of peers downloading the same torrent doesn’t need to be hold by one or more central trackers, but it can be contained inside the network of peer downloaders itself. The algorithm used for this is a Distributed Hash Table. The key is a checksum of some of the info in the .torrent file, and the value correlating to this key is the list of peers. These key-value pairs form the hash table.The table is distributed according to the following principle: the node(s) whose node address is the closest towards the key holds the correlating value, in this case the list of peers.

So, originally the BitTorrent client would look at the .torrent file to find the tracker URL inside it. It would then contact this tracker, recieve a list of peers, and start downloading the file. Now, the client downloads the .torrent file, computes the key from the .torrent file, uses this key to contact the node who has the peer list (the Distributed Hash Table lookup), recieves the peer list and starts downloading. Look Ma, no hands, no tracker necessary!

It may look perfect but there is one problem: how does the BitTorrent client finds the node where the peer list resides? The good thing about a Distributed Hash Table lookup is that you do not need to know every single node on the network, but you need to know at least one of them. So how does BitTorrent find at least one of the peer downloaders?

I generated a .torrent file with the 4.1.0 Beta Bittorrent client to find out. The program hung, but it got far enough to create a torrent file. Turns out the answer is router.bittorrent.com. There is a new piece of info attached to the torrent file, called node, and it contains the address router.bittorrent.com, port 6881. Your client will contact this server to find out about the Distributed Hash Table. router.bittorrent.com is not a tracker and it won’t participate in any downloading and uploading at all. It’s just a well known node in the Distributed Hash Table.

I’m not sure if this a good idea. The tracker is gone but instead of a multiple trackers, each of them serving different torrents, we now have a single point of failure for all “trackless automatic” torrents, not just the ones confined to a certain tracker. I guess that’s why there’s a “trackless node” option, where you can fill in a “first contact” node for yourself. This means that there are three ways to set up a torrent: the old fashioned way with a tracker, the new way with a “first contact” node of your own or you could rely on router.bittorrent.com.

In conclusion, trackerless BitTorrent makes BitTorrent easier for peer-to-peer file sharing, but it will not make a difference for reliable large content serving from a shared webhost. The seeding problem remains, you will have to keep seeding your torrent for as long as you want it to be alive. The tracker part is gone, but that was the easiest one to implement on a shared webhost. The tracker protocol was just HTTP. Seeding is more difficult from a webhost: it uses long living connections and a more difficult protocol. I can’t find any PHP seeders at this moment.

BitTorrent is often heralded as the greatest thing since sliced bread for large content distribution. According to this article, BitTorrent usage accounts for about 53% of all peer-to-peer traffic. But is it any good for small filmmakers?

BitTorrent is the brain child of Bram Cohen. He originally devised BitTorrent as a clever way to distribute large files, like 700MB .iso files of a Linux distribution. The principle idea of BitTorrent is that every downloader is an uploader too. The download is chopped into chunks, and every downloader keeps track of whatever chunks he has. Instead of downloading every chunk from one central server, the downloaders will download chunks from each other. As long as the whole file to download is somewhere in this ad-hoc peer-to-peer network, the download stays alive.

You need to do three things to make a movie available via BitTorrent:

1. You need to find a tracker. The tracker keeps track of all downloaders, and gives new downloaders a list of peers. This tracker will be contacted by every downloader periodically, so it needs to be on a public machine with enough processing power and bandwidth. You don’t need to run your own tracker, you can use public service trackers instead. Most public trackers require you to upload the .torrent file but it’s not strictly necessary.
2. You need to create a .torrent file from your original movie. This small file contains checksums for every chunk of the file, and the address of the tracker. The torrent file is on the one your visitors will initially download, so it’s the one that needs to be on your website.
3. You start seeding the torrent, presumably from your own computer. Create a directory with the original movie and the .torrent file you created in step two. You point your bittorrent client to the .torrent file and start “downloading” it. This may sound contra-intuïtive, but what actually happens is that your client starts uploading chunks from the original movie to every other downloader that asks for it. This is how your movie enters the network of downloaders.

Your visitor downloads the .torrent file, fires up his BitTorrent client and starts downloading your movie. The more people downloading your movie, the faster it goes, and aside from multiple downloads of the .small torrent file, your webserver won’t see any load at all.

Sounds like a great way to distribute movies, isn’t it? Well, it is, and it isn’t. People quickly found out that BitTorrent is ideal for sharing TV shows, Hollywood movies, anime, porn and other big files, but you won’t see many small independent movie makers using BitTorrent. Most have their own website and use traditional means of spreading their movies, mostly http downloads.

The problem is that while BitTorrent as a protocol is perfectly suited towards small movie distribution, the official (and other) implementations are not. Creating and distributing a torrent file is not a problem, but running a tracker is and seeding the torrent is.

The tracker poses a problem because you can not run one a shared webhost and because it is open to anyone. Anyone can use a tracker to spread whatever material they have, and because it’s not easy to set up a tracker, they will have to. This means that a) public trackers are easily overloaded and b) your movie is thrown into heap of crap ranging from pirated Hollywood movies to porn and other shady material. Hollywood doesn’t really like seeing its movies all over internet, so they let the MPAA shut down quite a few trackers (although technically they do not distribute a single byte of the infringing material). So, trackers are not exactly reliable, and you won’t be setting one up soon because you can not control what passes through it. (note).

Seeding is another problem. A torrent without a single seed is dead. You yourself must keep a connection to the tracker for as long as you want your movie to be available. Obviously this is not a problem when you just want to spread the latest hit movie. That torrent will stay alive long enough by itself, and when it’s dead another hit movie will be there. But if you want to keep your movies available forever, you will need to find a way to keep seeding the torrent forever.

What is needed for BitTorrent to be usefull for me (and presumably other content distributors) is a way to install the server software on a shared server, upload your movie and .torrent file, and be done with it. The software should seed the torrent automatically and the tracker should be restricted to the torrents I feed it with. And this should have been done from the beginning.

Open semi-anonymous trackers might have been a wonderfull technical idea, but it opened the floodgates for all kinds of piracy and gave BitTorrent an unnecessary bad name. Compare it to SMTP: open mail relays are still causing us problems.

I just hope somebody finds the incentive to create a neat BitTorrent package that does what I described above, or starts a similar simple BitTorrent hosting service. Right now the best way to use BitTorrent for distributing movies might be a dedicated server running a restricted tracker and a seeder.

In all fairness, the current official BitTorrent distribution allows you to restrict the tracker to local torrents only. Most PHP-based trackers require you to upload the torrent file first and can easily be restricted to certain torrents. But the damage is done: my webhost forbids anything remote looking like a tracker (even restricted ones) because the risk of copyright infringment is too big for them.