Re: haproxy-systemd-wrapper spawning multiple processes
On Sun, Feb 16, 2014 at 11:12:03AM +0900, Marc-Antoine Perennou wrote: > On Sat, 2014-02-15 at 20:04 -0600, Ryan O'Hara wrote: > > On Sun, Feb 16, 2014 at 10:08:31AM +0900, Marc-Antoine Perennou wrote: > > > > > This is why you get > > > > > > haproxy-systemd-wrapper -> main haproxy process -> haproxy worker. > > > > > > haproxy-systemd-wrapper waits for the main haproxy process to exit to > > > avoir zombies. The main haproxy process exits when all its workers are > > > done. > > > > It has been while since I dealt with this, but can't you double fork > > to avoid zombies? Is it a double fork that causes problems for systemd? > > > > Double forking would work fine, but the wrapper wouldn't be linked to > the worker processes and thus couldn't check whether there has been a > problem or not. Yes. That is precisely what I was thinking. Thanks again. Ryan
Re: haproxy-systemd-wrapper spawning multiple processes
On Sat, 2014-02-15 at 20:04 -0600, Ryan O'Hara wrote: > On Sun, Feb 16, 2014 at 10:08:31AM +0900, Marc-Antoine Perennou wrote: > > > This is why you get > > > > haproxy-systemd-wrapper -> main haproxy process -> haproxy worker. > > > > haproxy-systemd-wrapper waits for the main haproxy process to exit to > > avoir zombies. The main haproxy process exits when all its workers are > > done. > > It has been while since I dealt with this, but can't you double fork > to avoid zombies? Is it a double fork that causes problems for systemd? > Double forking would work fine, but the wrapper wouldn't be linked to the worker processes and thus couldn't check whether there has been a problem or not. Without the double fork, if there is a problem, the wrapper will exit and you'll be able to notice it. With the double fork, we would end up with some kind of endless loop instead of a waiting loop, and it would never exit. > > > Thanks. > > > Ryan > > > > > > > Hope that helps and sounds right. > > > > Marc-Antoine > > It does help. Thank you. > > Ryan > Regards Marc-Antoine
Re: haproxy-systemd-wrapper spawning multiple processes
On Sun, Feb 16, 2014 at 10:08:31AM +0900, Marc-Antoine Perennou wrote: > Hi, > > On 16 February 2014 01:51, Ryan O'Hara wrote: > > > > I started tinkering with haproxy-systemd-wrapper recently and noticed > > that I get two haproxy processes when I start: > > > > # systemctl start haproxy > > # systemctl status haproxy > > haproxy.service - HAProxy Load Balancer > >Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled) > >Active: active (running) since Sat 2014-02-15 10:39:20 CST; 1s ago > > Main PID: 10065 (haproxy-systemd) > >CGroup: /system.slice/haproxy.service > >├─10065 /usr/sbin/haproxy-systemd-wrapper -f > >/etc/haproxy/haproxy.cfg -p /run/haproxy > >├─10066 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p > >/run/haproxy.pid -Ds > >└─10067 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p > >/run/haproxy.pid -Ds > > > > That doesn't seem right. A quick look the haproxy processes shows an > > interesting parent/child relationship: > > > > # ps -C haproxy -o pid,ppid > > PID PPID > > 10066 10065 > > 10067 10066 > > > > Can someone explain what is going on here? I'm using 1.5-dev22 and the > > systemd service file from the source. > > > > Here is how haproxy works (correct me if I'm wrong, it's not all that > fresh in my mind): > - the main haproxy process is started > - it forks as many child processes as asked in its configuration file > - it goes away letting only the worker child processes Right. This seems pretty standard. > First thing I did was to make it wait for the worker child processes > instead of leaving, that's what -Ds is for. This is in order to avoid > the double fork which would happen because of what I'll describe just > below. > > Here is how haproxy "reloads" its configuration: > - A new haproxy is spawned with the pids of the old workers > - The new haproxy tells the old workers not to listen anymore, and to > exit when they have finished dealing with their current requests > - The new haproxy spawns its own workers and starts listening > - The old haproxy quits eventually when it has dealt with all pending requests > > systemd doesn't like this behavious at all as the main process > completely goes away, replaced by a brand new one, just for a > *reload*. > > The easiest way to get it working without having to rework the core > behaviour of haproxy was to put a wrapper around it, which spawns > haproxy, listens to a signal which systemd emits on reload, and which > spawns a new haproxy when this signal is received. This way, the main > process never changes ans systemd can reload gracefully. I understand. I went back and read the description you provided when you submitted the patch. I just wasn't expecting the main haproxy process to _not_ exit. The more I think about this, the more it makes sense. My initial assumption was that the MAINPID (in systemd) would be pid of haproxy-systemd-wrapper, the main haproxy would spawn the workers and exist, and a 'systemctl reload' could signal the workers. > This is why you get > > haproxy-systemd-wrapper -> main haproxy process -> haproxy worker. > > haproxy-systemd-wrapper waits for the main haproxy process to exit to > avoir zombies. The main haproxy process exits when all its workers are > done. It has been while since I dealt with this, but can't you double fork to avoid zombies? Is it a double fork that causes problems for systemd? > > Thanks. > > Ryan > > > > Hope that helps and sounds right. > > Marc-Antoine It does help. Thank you. Ryan
Re: haproxy-systemd-wrapper spawning multiple processes
Hi, On 16 February 2014 01:51, Ryan O'Hara wrote: > > I started tinkering with haproxy-systemd-wrapper recently and noticed > that I get two haproxy processes when I start: > > # systemctl start haproxy > # systemctl status haproxy > haproxy.service - HAProxy Load Balancer >Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled) >Active: active (running) since Sat 2014-02-15 10:39:20 CST; 1s ago > Main PID: 10065 (haproxy-systemd) >CGroup: /system.slice/haproxy.service >├─10065 /usr/sbin/haproxy-systemd-wrapper -f >/etc/haproxy/haproxy.cfg -p /run/haproxy >├─10066 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p >/run/haproxy.pid -Ds >└─10067 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p >/run/haproxy.pid -Ds > > That doesn't seem right. A quick look the haproxy processes shows an > interesting parent/child relationship: > > # ps -C haproxy -o pid,ppid > PID PPID > 10066 10065 > 10067 10066 > > Can someone explain what is going on here? I'm using 1.5-dev22 and the > systemd service file from the source. > Here is how haproxy works (correct me if I'm wrong, it's not all that fresh in my mind): - the main haproxy process is started - it forks as many child processes as asked in its configuration file - it goes away letting only the worker child processes First thing I did was to make it wait for the worker child processes instead of leaving, that's what -Ds is for. This is in order to avoid the double fork which would happen because of what I'll describe just below. Here is how haproxy "reloads" its configuration: - A new haproxy is spawned with the pids of the old workers - The new haproxy tells the old workers not to listen anymore, and to exit when they have finished dealing with their current requests - The new haproxy spawns its own workers and starts listening - The old haproxy quits eventually when it has dealt with all pending requests systemd doesn't like this behavious at all as the main process completely goes away, replaced by a brand new one, just for a *reload*. The easiest way to get it working without having to rework the core behaviour of haproxy was to put a wrapper around it, which spawns haproxy, listens to a signal which systemd emits on reload, and which spawns a new haproxy when this signal is received. This way, the main process never changes ans systemd can reload gracefully. This is why you get haproxy-systemd-wrapper -> main haproxy process -> haproxy worker. haproxy-systemd-wrapper waits for the main haproxy process to exit to avoir zombies. The main haproxy process exits when all its workers are done. > Thanks. > Ryan > Hope that helps and sounds right. Marc-Antoine
haproxy-systemd-wrapper spawning multiple processes
I started tinkering with haproxy-systemd-wrapper recently and noticed that I get two haproxy processes when I start: # systemctl start haproxy # systemctl status haproxy haproxy.service - HAProxy Load Balancer Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled) Active: active (running) since Sat 2014-02-15 10:39:20 CST; 1s ago Main PID: 10065 (haproxy-systemd) CGroup: /system.slice/haproxy.service ├─10065 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy ├─10066 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds └─10067 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds That doesn't seem right. A quick look the haproxy processes shows an interesting parent/child relationship: # ps -C haproxy -o pid,ppid PID PPID 10066 10065 10067 10066 Can someone explain what is going on here? I'm using 1.5-dev22 and the systemd service file from the source. Thanks. Ryan
