Hi Jeff, Sorry, I hadn't checked the list in a while. Here are a few thoughts. Any thoughts from anyone about them are more than welcome, I've even numbered them to make that easier :)
1) Increasing the ring bffer is fine. Only downside is that more of the file gets buffered in RAM. 2) IIRC you can use the control socket to see how much data has been sent on a connection and use that to throttle the send. I've not done it myself though. 3) I have started on a form of x-sendfile type support, but life has been busy and I haven't had much time to work on it. 4) Be aware that with 1.7.5 you typically will end up buffering the entire file in RAM since you'll be sending to m2 much faster than m2 is sending to the client. 5) I'm trying to think of a good way to have m2 let the backend know when the ring buffer is getting full so that the backend can slow-down. If you have any suggestions for this, let me know. 6) Tordek suggested on IRC that we replace the ring-buffer with a dynamically sized queue. This would fix your problem though you would still end up with #4. 7) The ring buffers were introduced to fix a real DoS problem that was demonstrated. While they don't work well for sending files, they are great for fixed-rate streaming where you would want to kill connections that are falling too far behind. #6 still solves the DoS problem, so I am open to it, but I think it might be better to fail obviously when you are dumping lots of data into m2 than it is to silently eat up all your RAM. -Jason On 15:54 Mon 13 Aug , Jeff Van Voorst wrote: > > > > What would be the consequences of increasing the value to say 256 or > > even higher than 1024? > > > > One could argue that the handler is broken as it is splitting the > > message into lots of very small messages (length less than 210), but > > as a stopgap measure, I would like to, for the time being increase the > > number of outstanding messages and see how that impacts the > > performance of mongrel2. > > > > --Jeff > For the time being, I am reverting back to using 1.7.5 because I am not > sure how I should handling file downloads in my handler. > > My setup is one mongrel2 process and one handler process with 1 thread. > Both are running on the same VM that has one processor core. > > Unfortunately, when running mongrel2 1.8.0 and streaming one 100+ MB > file it is very easy (happens almost all the time) with one mongrel2 and > one handler chunking the files into 64k pieces to run over the > deliverRing with 128 slots (default is 16 slots). Note that the files > are created on demand, and generally no two are the same. Without > something like X-sendfile, this "ring" makes it difficult to stream > files from a webpage as someone can generally make some case where the > "ring" is too small. > > I don't have the time to investigate the issues at this time, but is > anyone using mongrel2 1.8.0 in production? How do you get around a > similar issue of having multiple outstanding requests with multiple > handlers that send a relatively large number of messages (relative to > DELIVER_OUTSTANDING_MSG)? In other words, mongrel2 is supposed to scale > very well, but with multiple users making lightning fast requests (or a > handler that sends lots of small messages per request), it appears quite > easy for a connection to have more than 16 outstanding messages. > > --Jeff >
