On Tue, Mar 6, 2007, Paul J Stevens <[EMAIL PROTECTED]> said: > [EMAIL PROTECTED] wrote: >> ---------------------------------------------------------------------- >> More fundamental problem: linux fwrite doesn't fail apparently until the C >> library's internal buffers are full and flushed to disk. At that point, >> the next fwrite will return 0. >> >> So we need a broader strategy for allocating space before filling it, or >> -- what about the memory stream code that's commented out? > > I just remembered that one. I selected a file based buffer rather than a > memory buffer during my tests with inserting a very large message (1GB). > I did that on a heavily used production server that serves as my testbed > for live testing. I found that there was excessive copying going on in > dbmail-message.c during insertion, and I removed as much of that as I > could at that point. But I also found that delivery of large mails > behaved more reliable when using a file rather than memory. > > Doing it in memory would of course be much faster. But the case remains > how to deal with resource constraints, be it disk or ram. If we can > cover that I'm all for switching back to a memory stream.
So let's keep this on the list of performance issues that need to be improved upon at some point down the road. At least for now we don't have a reliability problem. We should add a note that tmpfs is very strongly recommended! I'm almost certain that we'll end up needing a dual approach, one for small messages and another for large ones. Going back to one of the early designs of immediately inserting headers and then processing body might help -- by parsing the header, we'll know more about what to expect from the body: maybe just 1k of plain text, maybe 100MB of complex MIME data. Anyways, more to think about another day. Aaron _______________________________________________ Dbmail-dev mailing list [email protected] http://twister.fastxs.net/mailman/listinfo/dbmail-dev
