Re: [systemd-devel] socket activation systemd holding a reference to an accepted socket
On Wed, Feb 24, 2016 at 4:29 AM, Lennart Poettering wrote: > On Tue, 23.02.16 14:58, Ben Woodard (wood...@redhat.com) wrote: > > > On Thu, Feb 18, 2016 at 9:08 AM, Lennart Poettering < > lenn...@poettering.net> > > wrote: > > > > > On Wed, 17.02.16 17:44, Ben Woodard (wood...@redhat.com) wrote: > > > > > > > Is it intentional that systemd holds a reference to a socket it has > > > > just accepted even though it just handed the open socket over to a > > > > socket.activated service that it has just started. > > > > > > Yes, we do that currently. While there's currently no strict reason to > > > do this, I think we should continue to do so, as there was always the > > > plan to provide a bus interface so that services can rerequest their > > > activation and fdstore fds at any time. Also, for the listening socket > > > case (i.e. Accept=no case) we do the same and have to, hence it kinda > > > is makes sure we expose the same behaviour here on all kinds of > > > sockets. > > > > > > > I'm having trouble believing that consistency of behavior is an ideal to > > strive for in this case. Consistency with xinetd's behavior would seem to > > be a better benchmark in this case. > > Well, we are implementing our own socket passing protocol anyway, and > support the inetd style one only for completeness. > Which I believe supports my assertion that for the legacy inetd style socket activation, we reproduce its semantics so that people porting systemd have a measure of backwards compatibility. Then for those services which are updated to the new socket passing protocol they can have whatever semantics are needed to support that new kind of interaction. It should be pretty obvious from the unit file when the service is expecting simple inetd semantics vs implementing the socket passing interface. > > > And I'm not sure that I understand the value being able to rerequest a > > fdstore of fds. To me this sounds like it would be a very rarely used > > feature. Could this be made an option that could be enabled when you add > > the bus service that allows a service to rerequest their activation and > > fdstore fds? > > Apple's launchd does activation that way. They have a "check-in" > protocol, where instead of passing fds through exec() the activated > process ask for them via an IPC call. This has quite some benefits as > it allows the service manager to update the set of sockets > dynamically, and the service can query them at any time to acquire the > newest set. > > I think it is good that you are patterning this off of apple's prior art. However, I'm still moderately skeptical about the utility of this. It is interesting; it sounds cool; I have no idea when I would ever want to use it and can't think of any cases where a service that I'm familiar with needs that feature. I'm not in anyway trying to lobby against it. I just feel like these future plans shouldn't prevent a slightly more complete backwards compatibility mode for inetd socket activation services. > Another reason btw to keep the fd open in systemd is that we allow a > series of programs to be invoked via multiple ExecStartPre=, > ExecStart=, ExecStartPost=, ExecStop=, ExecStopPost= lines, and we > have to keep things open at least for all but the last one hence, so > that we have something to pass to the last invocation. > I think that is the most convincing argument thus far. I'd still argue for special case logic that with simple socket activation accept=yes when the last invocation is made, systemd closes its reference to the socket. > > > > Did you run into problems with this behaviour? > > > > > > > Oh yes. It breaks a large number of management tools that we have on to > do > > various things on clusters. It is a kind of pseudoconcurrency. Think of > > like this: > > > > foreach compute-node; > >rsh node daemonize quick-but-not-instant-task > > > > With xinetd the demonization would close the accepted socket. Then the > > foreach loop would nearly instantly move onto the next node. We could zip > > through 8000 nodes in a couple of seconds. > > > > With systemd holding onto the socket the "rsh" hangs until the > > quick-but-not-instant-task completes. This causes the foreach loop to > take > > anywhere from between 45min and several hours. Because it isn't really > rsh > > and demonize and I just used that to make it easy to understand what is > > going on, rewriting several of our tools is non-trivial and would end up > > violatin
Re: [systemd-devel] socket activation systemd holding a reference to an accepted socket
On Thu, Feb 18, 2016 at 9:08 AM, Lennart Poettering wrote: > On Wed, 17.02.16 17:44, Ben Woodard (wood...@redhat.com) wrote: > > > Is it intentional that systemd holds a reference to a socket it has > > just accepted even though it just handed the open socket over to a > > socket.activated service that it has just started. > > Yes, we do that currently. While there's currently no strict reason to > do this, I think we should continue to do so, as there was always the > plan to provide a bus interface so that services can rerequest their > activation and fdstore fds at any time. Also, for the listening socket > case (i.e. Accept=no case) we do the same and have to, hence it kinda > is makes sure we expose the same behaviour here on all kinds of > sockets. > I'm having trouble believing that consistency of behavior is an ideal to strive for in this case. Consistency with xinetd's behavior would seem to be a better benchmark in this case. And I'm not sure that I understand the value being able to rerequest a fdstore of fds. To me this sounds like it would be a very rarely used feature. Could this be made an option that could be enabled when you add the bus service that allows a service to rerequest their activation and fdstore fds? > Did you run into problems with this behaviour? > Oh yes. It breaks a large number of management tools that we have on to do various things on clusters. It is a kind of pseudoconcurrency. Think of like this: foreach compute-node; rsh node daemonize quick-but-not-instant-task With xinetd the demonization would close the accepted socket. Then the foreach loop would nearly instantly move onto the next node. We could zip through 8000 nodes in a couple of seconds. With systemd holding onto the socket the "rsh" hangs until the quick-but-not-instant-task completes. This causes the foreach loop to take anywhere from between 45min and several hours. Because it isn't really rsh and demonize and I just used that to make it easy to understand what is going on, rewriting several of our tools is non-trivial and would end up violating all sorts of implicit logical layering within the tools and libraries that we use to build them. Where is this in the source code? I've been planning to send you a patch to change the behavior but I have't quite connected the dots from where a job within a transaction is inserted on the run queue to where the fork for the "ExecStart" actually happens. -ben Red Hat Inc. > Lennart > > -- > Lennart Poettering, Red Hat > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] socket activation systemd holding a reference to an accepted socket
> On Feb 18, 2016, at 4:23 AM, Zbigniew Jędrzejewski-Szmek > wrote: > > On Thu, Feb 18, 2016 at 09:59:32AM +0300, Andrei Borzenkov wrote: >> On Thu, Feb 18, 2016 at 4:44 AM, Ben Woodard wrote: >>> Is it intentional that systemd holds a reference to a socket it has just >>> accepted even though it just handed the open socket over to a >>> socket.activated service that it has just started. >> >> yes. >> >> >> systemd continues to listen for new requests on this socket. Each >> request will spawn new service instance. > > OP's question is about the connection socket, not about the listening > socket. > Zbyszek is correct, I expect systemd to continue to hold onto the listening socket to be able to handle subsequent incoming connections. I’m talking about the socket that was accepted. I don’t see a good reason to hold onto that. Before I prepare a patch to change the behavior, I wanted to find out from the developers if there is some reason it holds onto the reference to the accepted socket that I’m not recognizing. -ben I > Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] socket activation systemd holding a reference to an accepted socket
Is it intentional that systemd holds a reference to a socket it has just accepted even though it just handed the open socket over to a socket.activated service that it has just started. For example given the following unit files: /etc/systemd/system/test.socket: [Unit] Description=test service [Socket] ListenStream=804 Accept=yes [Install] WantedBy=sockets.target -- /etc/systemd/system/test@.service: [Unit] Description=test per-service unit [Service] ExecStart=-/home/ben/Work/sysdtest/sysdtest -- The sysdtest is just a program which essentially daemonizes itself and then sleeps for a few seconds. So it doesn’t have an open reference to the file descriptor that was accepted. Yet the socket stays open and connected to the client until the daemon like process exits or the timeout for the service instance is reached because systemd holds onto a reference to the file descriptor. Is this intentional? The behavior differs from xinetd which didn’t hold onto any references to the newly accepted socket. This allowed the socket to close as soon as the socket activated process daemonized. Holding onto a reference to the socket FD surprised me enough that I verified that it was in fact systemd holding the reference by using lsof: [ben@localhost sysdtest]$ sudo lsof -i :804 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME systemd1 root 19u IPv6 384026 0t0 TCP localhost.localdomain:804->localhost.localdomain:46220 (ESTABLISHED) systemd1 root 30u IPv6 376208 0t0 TCP *:804 (LISTEN) telnet 1021 ben3u IPv4 384025 0t0 TCP localhost.localdomain:46220->localhost.localdomain:804 (ESTABLISHED) -ben ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
