On Jul 3, 2:33 pm, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > If the problem does not require two way communication, which is > > typical of a producer-consumer, it is a lot faster to allow P to fully > > run before C is started. > > Why do you say it's *a lot* faster. I find that it is a little faster. > The only additional overhead from switching forth and back between > consumer and producer is the overhead for context switching, which > is typically negligible, compared to everything else that is going > on.
True it needn't be *a lot*. I did observe 25% gain or more when there were a chain of processes involved as in a shell pipeline. Again this could be very problem specific. What I had, included something like 4 or 5 processes connected as in p1 | p2 | p3 | p4 ... here I found the back-n-forth context switching was slowing down quite a bit (some thing like 2 mins task completed in under 40 seconds without the piping) If all you had is just two processes, P and C and the amount of data flowing is less (say on the order of 10's of buffer-size ... e.g. 20 times 4k), *a lot* may not be right quantifier. But if the data is large and several processes are involved, I am fairly sure the overhead of context-switching is very significant (not negligible) in the final throughput. Thanks, Karthik > > Regards, > Martin
-- http://mail.python.org/mailman/listinfo/python-list