From: "Ryan Bloom" <[EMAIL PROTECTED]> Sent: Tuesday, November 27, 2001 11:26 PM
> > What I'm suggesting is that as each bucket is consumed, it is un-memaped. > > The buckets are only mmaped as they are read. This way, we get the > > benefits of mmap, without the pain of page consumption. > > > > Idea? > > Not good. The problem is that you will take the real chance that multiple > filters will be converting to mmap, and then unmapping large files without > making any changes. Think of a large SSI file that does have any tags. > You'll MMAP each piece, unmapping as you go, and then you'll do the > same thing again in the next filter. That would hurt, yes. But we need two things to make this happen without such a nasty side effect. One, refcounting. Two, the simple fact that once consumed, the other filters have no business touching any old bucket or data. The only reasonable scenario, short of a really, really messed up filter, is that something like a big SSI tag would span _two_ mmapped buckets. That's fine, so we actually _drop_ the threshold for a single mmap (say 256kb or something, that should actually become tunable.) In any case, any filter hanging on to several meg at once is just asking for problems, if it might be expected to be invoked by many requests concurrently. Cliff has his hands around this concept better than I... I'll wait to grok his patch :) Time to get back to work by this weekend, and actually start authoring the three filters I had in mind ... so I can start stressing such things. Bill
