On 26 Mar 2020, at 13:41, Joe Orton <jor...@redhat.com> wrote: > On Thu, Mar 26, 2020 at 01:11:10AM +0200, Graham Leggett wrote: >> The question you’re asking is “why is is an async path being taken >> when AP_MPMQ_IS_ASYNC is false”. The setasides and reinstates should >> be noops in this situation. > > The "noop" path in ap_filter_setaside_brigade is when it is passed an > empty brigade, right?
It’s not right, no. To understand how this works, you need to understand how the core network filter works in Apache 2.4. For the event MPM to be able to be an event MPM, there needs to be a mechanism by which buckets are not swallowed whole. Concrete example: 1TB file bucket. We cannot block forever trying to write 1TB of data to one client, starving all other clients of the opportunity to be sent data, and call ourselves event driven. So, what the core network filter does is this: - Reinstate earlier setaside data (if there was any). - Try make non-blocking writes until the OS tells us it’s going to block. - Setaside any unconsumed buckets, back off and let someone else have a go. The core network filter also handles flow control. Pipelining is a thing, so we have to limit the number of requests in the entire request. File descriptors are a thing, so we need to track how many file buckets are in the brigade. Beyond a threshold, the core filter blocks to keep things sane. The type of buckets in this case are irrelevant and always has been. They could be heap buckets with a fixed size, virtual buckets of some kind like file buckets that are a fixed size but access their data by reference, or morphing buckets with no fixed size at all, or EOR buckets that clean up a request when consumed, or an EOC bucket that cleans up a connection being consumed. Now, lets look at trunk. This was generalised so that any filter can do what the core network filter can do: - Reinstate earlier setaside data (if there was any). - Ask if the OS is going to block - this is the should_yield. If the answer is “no”, the filter writes data. If the answer is always “no”, old filter behaviour applies, the entire brigade will get written as it did, and the core will block as it did. - If should_yield says “yes” there will data left over, set the data aside, and let someone else have a go. If should_yield is always “no”, setaside will always be empty in every filter. What happens when we enter write completion? Async MPMs will keep writing empty brigades down the filter stack, until every last setaside bucket is flushed out, then we’re done and we can handle the next request when it arrives. Now let’s look at non-async MPMs: All non-async MPMs must return “no” to should_yield. This means that all data in the filters gets written, and no data gets set aside. Because no data is set aside, there is no need to send any empty brigades down the stack to coax any unwritten setaside data to go down the chain, so existing non-async MPMs work unchanged. This is what "The setasides and reinstates should be noops in this situation” means. Now let’s look at non async filters: Non async filters don't stop until their entire brigade is written, all 1TB of it. If all the filter did was counting bucket sizes, filter passes the entire lot down and steps aside, no harm done. But if the filter transformed the data, eg by compressing it, all 1TB gets eaten in one go. Oh well. Still works though. So, the bug. It looks like async filters work fine. We are seeing the non-async case not working properly. We need to find the reason that in a non-async case, data is being setaside, and we need to fix it. Regards, Graham —
smime.p7s
Description: S/MIME cryptographic signature
