Hi Sergey, I was going to suggest this too, so "yes", I really like the idea. I think there are two goals:
* keep the process' memory footprint small * keep Mongoose responsive I think delivering reasonably sized chunks at intervals would make those goals possible. The fact that Mongoose normally delivers all the content when the callback is invoked is really nice. So please keep that, if you can, and make data upload different some how... :-) By-the-way, I must give you a big "thank you" for your support. You are amazingly responsive and you have an excellent product. We've been at 3.1 for a long time and just did a test upgrade to 5.4. The performance difference is huge. Great stuff. We're so happy we're going to be making a purchase. Cheers, Doug On Tuesday, August 19, 2014 4:41:39 PM UTC-6, Sergey Lyubka wrote: > > Mongoose can introduce a new event, e.g. MG_DATA_CHUNK trigger a callback > for every data chunk that arrives. > > It would be a little bit harder for a event handler to handle the stream. > I thought that spooling everything and triggering the handler with > ready-to-use content + content_len would be easier for the developer to > handler. > > The case with huge files is special as we can see. Any objections / > comments about introducing MG_DATA_CHUNK event? > > > > On Tue, Aug 19, 2014 at 11:37 PM, jeff shanab <[email protected] > <javascript:>> wrote: > >> PSS. why can't it call the callback when it has some data and then call >> it again when it gets more? kinda like curl does. There is only so much a >> socket can get at a time. >> >> So your loop holds onto the socket and if there is data calls the call >> back for that socket, then goes back to the top of the loop. >> A slow reader or a slow writer just doesn't have data on the socket and >> does not impact the other readers and writers. (like LIVE555 does for RTSP) >> >> >> On Tue, Aug 19, 2014 at 5:33 PM, Sergey Lyubka <[email protected] >> <javascript:>> wrote: >> >>> Mongoose does this: spools file to memory, and when the whole file is >>> buffered, it invokes a callback. >>> I think mongoose blocks during big file write IO operation, and the >>> callbacks blocks for a long time. >>> >>> Could you try using upload.c example please? >>> >>> I think I might refactor the example to save the file in the separate >>> thread. >>> >>> >>> On Tue, Aug 19, 2014 at 9:48 PM, <[email protected] <javascript:>> wrote: >>> >>>> Hi, >>>> >>>> When uploading a large file (100+ megabytes), Mongoose seems to block >>>> until the entire file is uploaded. New requests are not accepted and the >>>> user callback function is not invoked. It can take many minutes before >>>> unblocking. I'm patterning my code off of the "big_upload.c" example. >>>> >>>> Is it expected that Mongoose will block while reading the data and only >>>> once that is done, invoke the callback and accept new connections? Does >>>> Mongoose hold the entire data set in memory or could there be some sort of >>>> streaming going on? >>>> >>>> Thanks, >>>> Doug >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "mongoose-users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected] <javascript:>. >>>> To post to this group, send email to [email protected] >>>> <javascript:>. >>>> Visit this group at http://groups.google.com/group/mongoose-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "mongoose-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> To post to this group, send email to [email protected] >>> <javascript:>. >>> Visit this group at http://groups.google.com/group/mongoose-users. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "mongoose-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at http://groups.google.com/group/mongoose-users. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "mongoose-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/mongoose-users. For more options, visit https://groups.google.com/d/optout.
