On Sat, Jun 15, 2013 at 6:00 AM, Robert Haas <robertmh...@gmail.com> wrote:
> The second patch, dynamic-bgworkers-v1.patch, revises the background > worker API to allow background workers to be started dynamically. > This requires some communication channel from ordinary workers to the > postmaster, because it is the postmaster that must ultimately start > the newly-registered workers. However, that communication channel has > to be designed pretty carefully, lest a shared memory corruption take > out the postmaster and lead to inadvertent failure to restart after a > crash. Here's how I implemented that: there's an array in shared > memory of a size equal to max_worker_processes. This array is > separate from the backend-private list of workers maintained by the > postmaster, but the two are kept in sync. When a new background > worker registration is added to the shared data structure, the backend > adding it uses the existing pmsignal mechanism to kick the postmaster, > which then scans the array for new registrations. I have attempted to > make the code that transfers the shared_memory state into the > postmaster's private state as paranoid as humanly possible. The > precautions taken are documented in the comments. Conversely, when a > background worker flagged as BGW_NEVER_RESTART is considered for > restart (and we decide against it), the corresponding slot in the > shared memory array is marked as no longer in use, allowing it to be > reused for a new registration. > > Since the postmaster cannot take locks, synchronization between the > postmaster and other backends using the shared memory segment has to > be lockless. This mechanism is also documented in the comments. An > lwlock is used to prevent two backends that are both registering a new > worker at about the same time from stomping on each other, but the > postmaster need not care about that lwlock. > > This patch also extends worker_spi as a demonstration of the new > interface. With this patch, you can CREATE EXTENSION worker_spi and > then call worker_spi_launch(int4) to launch a new background worker, > or combine it with generate_series() to launch a bunch at once. Then > you can kill them off with pg_terminate_backend() and start some new > ones. That, in my humble opinion, is pretty cool. > This looks really interesting, +1. I'll test the patch if possible next week. -- Michael
