One other thing. With the recent addition of sendfile support, we are left with the situation being:
1) Fixed rate streaming is easy 2) Sending a big file is easy The only thing that is hard is streaming a large amount of data that isn't a file. What are the use cases for that? I know there are some, but there might be few enough (or the majority might be similar enough) that a targeted solution would make more sense then a general purpose "This will let you stream anything without using too much RAM or loading down the server too much" which is a non-trivial problem to solve. For example, the extended reply system is flexible enough that we could use it to establish a plain-old TCP (or unix domain) connection betweem m2 and the handler to send the data over on a 1 connection per client ID basis. The problem with zeromq is that it buffers arbitrarily large amounts of data under-the-hood. Streaming might make more sense over TCP. -Jason On 16:50 Fri 05 Oct , Jason Miller wrote: > A fixed number of messages has a couple of advantages: > > 1) Fixed-rate streaming applications can just size their messages such > that 16 messages is equal to however far behind they want to allow their > clients to get, and then they don't need to do any other management > > 2) The default is for the connection to get dropped, rather than to have > a single bad client DoS the entire system. I think this is a good > default to have > > -Jason > > On 13:45 Fri 05 Oct , Henry Finucane wrote: > > On Fri, Oct 5, 2012 at 12:16 PM, Jason Miller <[email protected]> wrote: > > > Would increasing the limit from 16 to something bigger make this more > > > easy? 16 was selected arbitrarily by me. > > > > IIRC, the original problem was too much memory usage, wasn't it? > > Perhaps instead of a fixed array, the solution is a list and a running > > sum of the length of the strings in it. > > > > This seems like something people could usefully tune, and means that > > in the future you could even talk about a feature like "don't worry > > about the memory used by connections until the total is over 4gigs." > > > > -- > > ----------------------- > > | Henry Finucane > > | (510) 473-7148 > > ----------------------- > > > >
