A Fair Proxy Balancer for Nginx and Mongrel

Nginx is my favorite webserver as you astute reader all probably know. It’s very fast, light and rock solid in production. My only complaint was that the proxy modue was round robin only. This means that if you had 3 mongrels and one of them performed and action that took say 20 seconds, for that 20 seconds every 3rd request would get stuck in line behind the mongrel doing the long blocking request. This is because nginx was merrily doing round robin and didn’t care if a mongrel was busy or not.
Now we all know that you should not have actions that take this long in our apps right? Of course we do, but that doesn’t stop it from happening quite a bit in a lot of apps I’ve seen.
So EngineYard.com put out a bounty on getting a fair proxy balancer that would keep track of when a mongrel is busy and not send requests to it until it has finished and is ready to take requests. This means that now we can effectively queue inside nginx very efficiently and only send requests to non busy mongrels. Thereby avoiding the waiting in the wrong line for service problem described above.
Grzegorz Nosek has stepped up to claim the bounty and has done a wonderful job of making this work very nicely. this is not a production release but I wanted to get this out there to get some testing from interested parties.
You can grab a tarball of the git repo here:
http://git.localdomain.pl/?p=nginx.git;a=tree;hb=upstream_fair-0.6
click the snapshot link to download a tarball. Then you can compile nginx with the new module like this:
$ cd nginx.git $ ./configure—with-http_ssl_module $ make $ sudo make install
The only difference in configuration will be to add the “fair;” directive to your upstream block. For example:
upstream mongrel {fair;server 127.0.0.1:5000;server 127.0.0.1:5001;server 127.0.0.1:5002;}
Here is an easy way to try this out and verify it’s working for you. Create a rails or merb app with one action that does a “sleep 15” and another action that returns some text immediately. Fire up a cluster of 3 mongrels behind your newly compiled nginx with the fair directive commented out.
Now open up 2 browser windows and hit the sleepy action in one window and repeatedly hit the other action in the second window. with the fair directive commented out every 3rd request for the fast action will hang until the sleeping action is done. This is because mongrel is queuing requests and nginx is just using round robin.
‘Now uncomment the “fair;” line and restart nginx and do the experiment again. This time you will notice that nginx avoids sending any requests to the sleeping mongrel and only balances between the free ones until the sleep finishes, then it is added back to the available pool.
Please give this a shot and beat it up some folks. Report any problems you have here in the comments. This is going to be huge for any rails apps running under nginx and I;’d like to ferret out any bugs before we have an official release.