On Wed, Mar 18, 2015 at 11:43 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I think I figured out the problem. That fix only helps in the case >> where the postmaster noticed the new registration previously but >> didn't start the worker, and then later notices the termination. >> What's much more likely to happen is that the worker is started and >> terminated so quickly that both happen before we create a >> RegisteredBgWorker for it. The attached patch fixes that case, too. > > Patch fixes the problem and now for Rescan, we don't need to Wait > for workers to finish.
I realized that there is a problem with this. If an error occurs in one of the workers just as we're deciding to kill them all, then the error won't be reported. Also, the new code to propagate XactLastRecEnd won't work right, either. I think we need to find a way to shut down the workers cleanly. The idea generally speaking should be: 1. Tell all of the workers that we want them to shut down gracefully without finishing the scan. 2. Wait for them to exit via WaitForParallelWorkersToFinish(). My first idea about how to implement this is to have the master detach all of the tuple queues via a new function TupleQueueFunnelShutdown(). Then, we should change tqueueReceiveSlot() so that it does not throw an error when shm_mq_send() returns SHM_MQ_DETACHED. We could modify the receiveSlot method of a DestReceiver to return bool rather than void; a "true" value can mean "continue processing" where as a "false" value can mean "stop early, just as if we'd reached the end of the scan". This design will cause each parallel worker to finish producing the tuple it's currently in the middle of generating, and then shut down. You can imagine cases where we'd want the worker to respond faster than that, though; for example, if it's applying a highly selective filter condition, we'd like it to stop the scan right away, not when it finds the next matching tuple. I can't immediately see a real clean way of accomplishing that, though. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
