I have a wide assortment of queries on a site, some of which take several minutes to
execute, while others execute in less than one second. If understand this analogy
correctly, I'd be better off with the current incarnation of mod_perl because there
would be more cashiers around to serve the "quick cups of coffee" that many customers
request at my dinner.
Is this correct?
Sam Horrocks wrote:
>
> I think the major problem is that you're assuming that just because
> there are 10 constant concurrent requests, that there have to be 10
> perl processes serving those requests at all times in order to get
> maximum throughput. The problem with that assumption is that there
> is only one CPU - ten processes cannot all run simultaneously anyways,
> so you don't really need ten perl interpreters.
>
> I've been trying to think of better ways to explain this. I'll try to
> explain with an analogy - it's sort-of lame, but maybe it'll give you
> a mental picture of what's happening. To eliminate some confusion,
> this analogy doesn't address LRU/MRU, nor waiting on other events like
> network or disk i/o. It only tries to explain why you don't necessarily
> need 10 perl-interpreters to handle a stream of 10 concurrent requests
> on a single-CPU system.
>
> You own a fast-food restaurant. The players involved are:
>
> Your customers. These represent the http requests.
>
> Your cashiers. These represent the perl interpreters.
>
> Your cook. You only have one. THis represents your CPU.
>
> The normal flow of events is this:
>
> A cashier gets an order from a customer. The cashier goes and
> waits until the cook is free, and then gives the order to the cook.
> The cook then cooks the meal, taking 5-minutes for each meal.
> The cashier waits for the meal to be ready, then takes the meal and
> gives it to the customer. The cashier then serves another customer.
> The cashier/customer interaction takes a very small amount of time.
>
> The analogy is this:
>
> An http request (customer) arrives. It is given to a perl
> interpreter (cashier). A perl interpreter must wait for all other
> perl interpreters ahead of it to finish using the CPU (the cook).
> It can't serve any other requests until it finishes this one.
> When its turn arrives, the perl interpreter uses the CPU to process
> the perl code. It then finishes and gives the results over to the
> http client (the customer).
>
> Now, say in this analogy you begin the day with 10 customers in the store.
> At each 5-minute interval thereafter another customer arrives. So at time
> 0, there is a pool of 10 customers. At time +5, another customer arrives.
> At time +10, another customer arrives, ad infinitum.
>
> You could hire 10 cashiers in order to handle this load. What would
> happen is that the 10 cashiers would fairly quickly get all the orders
> from the first 10 customers simultaneously, and then start waiting for
> the cook. The 10 cashiers would queue up. Casher #1 would put in the
> first order. Cashiers 2-9 would wait their turn. After 5-minutes,
> cashier number 1 would receive the meal, deliver it to customer #1, and
> then serve the next customer (#11) that just arrived at the 5-minute mark.
> Cashier #1 would take customer #11's order, then queue up and wait in
> line for the cook - there will be 9 other cashiers already in line, so
> the wait will be long. At the 10-minute mark, cashier #2 would receive
> a meal from the cook, deliver it to customer #2, then go on and serve
> the next customer (#12) that just arrived. Cashier #2 would then go and
> wait in line for the cook. This continues on through all the cashiers
> in order 1-10, then repeating, 1-10, ad infinitum.
>
> Now even though you have 10 cashiers, most of their time is spent
> waiting to put in an order to the cook. Starting with customer #11,
> all customers will wait 50-minutes for their meal. When customer #11
> comes in he/she will immediately get to place an order, but it will take
> the cashier 45-minutes to wait for the cook to become free, and another
> 5-minutes for the meal to be cooked. Same is true for customer #12,
> and all customers from then on.
>
> Now, the question is, could you get the same throughput with fewer
> cashiers? Say you had 2 cashiers instead. The 10 customers are
> there waiting. The 2 cashiers take orders from customers #1 and #2.
> Cashier #1 then gives the order to the cook and waits. Cashier #2 waits
> in line for the cook behind cashier #1. At the 5-minute mark, the first
> meal is done. Cashier #1 delivers the meal to customer #1, then serves
> customer #3. Cashier #1 then goes and stands in line behind cashier #2.
> At the 10-minute mark, cashier #2's meal is ready - it's delivered to
> customer #2 and then customer #4 is served. This continues on with the
> cashiers trading off between serving customers.
>
> Does the scenario with two cashiers go any more slowly than the one with
> 10 cashiers? No. When the 11th customer arrives at the 5-minute mark,
> what he/she sees is that customer #3 is just now putting in an order.
> There are 7 other people there waiting to put in orders. Customer #11 will
> wait 40 minutes until he/she puts in an order, then wait another 10 minutes
> for the meal to arrive. Same is true for customer #12, and all others arriving
> thereafter.
>
> The only difference between the two scenarious is the number of cashiers,
> and where the waiting is taking place. In the first scenario, each customer
> puts in their order immediately, then waits 50 minutes for it to arrive.
> In the second scenario each customer waits 40 minutes in to put in
> their order, then waits another 10 minutes for it to arrive.
>
> What I'm trying to show with this analogy is that no matter how many
> "simultaneous" requests you have, they all have to be serialized at
> some point because you only have one CPU. Either you can serialize them
> before they get to the perl interpreter, or afterward. Either way you
> wait on the CPU, and you get the same throughput.
>
> Does that help?
