Re: [systemd-devel] Why systemd keeps references on passed sockets ?
On Tue, Sep 8, 2020 at 5:41 PM Lennart Poettering wrote: > > On Di, 08.09.20 17:35, Francis Moreau (francis.m...@gmail.com) wrote: > > > On Mon, Sep 7, 2020 at 4:38 PM Lennart Poettering > > wrote: > > > > > > "React on the socket close?" — What do you mean by that? > > > > > > > I mean if my service explicitly calls close() then systemd could stop > > the socket on its side so its are freed until the service is > > restarted. I think it is what you described below. > > close() just drops a ref to the socket. Only when close() drops the > last ref something actually happens on the socket and it is > destroyed. This means: systemd doesn't get notified about your code > invoking close(), because all you did is drop one ref of many. > > > > Note that on Linux you can invoke shutdown() on a listening socket > > > (i.e. not just on the connection socket, but on a listening > > > socket). iirc in that case systemd actually notices and will put the > > > .socket unit in failure mode... > > > > > > > I looked at the code and there is: > > > > if (state != SOCKET_LISTENING) > > socket_unwatch_fds(s); > > > > So I'm not sure how systemd can react on shutdown(). And I tried to > > call shutdown() in my service but it has no effects. > > Hmm, that suggests we'd have to slightly update our logic then for > this to work: keep the listening fds in the poll, but turn off all > the EPOLLIN bit we listen on. > > > > Would that work for you? (Maybe we could even tweak this a bit in > > > systemd, so that when you invoke shutdown() on the socket systemd > > > holds for you we do not consider that a failure anymore, but a clean > > > way to tell systemd to stop the socket). > > > > That is a good idea especially if systemd doesn't consider an error > > when the service closes or shutdown the socket. > > > > But again in the code I can see: > > > > sd_event_add_io(UNIT(s)->manager->event, >event_source, p->fd, > > EPOLLIN, socket_dispatch_io, p); > > > > It seems that it only listens to "EPOLLIN" events. So it doesn't > > listen to "EPOLLUP". > > EPOLLHUP is always implied, you don't have to specify it. if you > specifiy zero as mask you will still get notified about EPOLLHUP + > EPOLLERR. > Thank you for the info > But you are right we currently remove the fd from polling while the > socket's service is running. We'd have to change that (as mentioned > above). happy to review a PR for that. I'm afraid I'm not skilled enough in systemd internals to work on this, sorry. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Why systemd keeps references on passed sockets ?
On Di, 08.09.20 17:35, Francis Moreau (francis.m...@gmail.com) wrote: > On Mon, Sep 7, 2020 at 4:38 PM Lennart Poettering > wrote: > > > > "React on the socket close?" — What do you mean by that? > > > > I mean if my service explicitly calls close() then systemd could stop > the socket on its side so its are freed until the service is > restarted. I think it is what you described below. close() just drops a ref to the socket. Only when close() drops the last ref something actually happens on the socket and it is destroyed. This means: systemd doesn't get notified about your code invoking close(), because all you did is drop one ref of many. > > Note that on Linux you can invoke shutdown() on a listening socket > > (i.e. not just on the connection socket, but on a listening > > socket). iirc in that case systemd actually notices and will put the > > .socket unit in failure mode... > > > > I looked at the code and there is: > > if (state != SOCKET_LISTENING) > socket_unwatch_fds(s); > > So I'm not sure how systemd can react on shutdown(). And I tried to > call shutdown() in my service but it has no effects. Hmm, that suggests we'd have to slightly update our logic then for this to work: keep the listening fds in the poll, but turn off all the EPOLLIN bit we listen on. > > Would that work for you? (Maybe we could even tweak this a bit in > > systemd, so that when you invoke shutdown() on the socket systemd > > holds for you we do not consider that a failure anymore, but a clean > > way to tell systemd to stop the socket). > > That is a good idea especially if systemd doesn't consider an error > when the service closes or shutdown the socket. > > But again in the code I can see: > > sd_event_add_io(UNIT(s)->manager->event, >event_source, p->fd, > EPOLLIN, socket_dispatch_io, p); > > It seems that it only listens to "EPOLLIN" events. So it doesn't > listen to "EPOLLUP". EPOLLHUP is always implied, you don't have to specify it. if you specifiy zero as mask you will still get notified about EPOLLHUP + EPOLLERR. But you are right we currently remove the fd from polling while the socket's service is running. We'd have to change that (as mentioned above). happy to review a PR for that. Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Why systemd keeps references on passed sockets ?
On Mon, Sep 7, 2020 at 4:38 PM Lennart Poettering wrote: > > "React on the socket close?" — What do you mean by that? > I mean if my service explicitly calls close() then systemd could stop the socket on its side so its are freed until the service is restarted. I think it is what you described below. > Note that on Linux you can invoke shutdown() on a listening socket > (i.e. not just on the connection socket, but on a listening > socket). iirc in that case systemd actually notices and will put the > .socket unit in failure mode... > I looked at the code and there is: if (state != SOCKET_LISTENING) socket_unwatch_fds(s); So I'm not sure how systemd can react on shutdown(). And I tried to call shutdown() in my service but it has no effects. > Would that work for you? (Maybe we could even tweak this a bit in > systemd, so that when you invoke shutdown() on the socket systemd > holds for you we do not consider that a failure anymore, but a clean > way to tell systemd to stop the socket). That is a good idea especially if systemd doesn't consider an error when the service closes or shutdown the socket. But again in the code I can see: sd_event_add_io(UNIT(s)->manager->event, >event_source, p->fd, EPOLLIN, socket_dispatch_io, p); It seems that it only listens to "EPOLLIN" events. So it doesn't listen to "EPOLLUP". Thank you. -- Francis ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Why systemd keeps references on passed sockets ?
On So, 06.09.20 22:23, Francis Moreau (francis.m...@gmail.com) wrote: > > It's how socket activation works really (at least with Accept=no): pid > > 1 allocate the socket and activates your service every time something > > happens on it, except if your service is already running. Thus a > > service can auto-start, process one or more events on it, can exit > > when idle, or even crash, it doesn't matter, no traffic is lost > > (except the very datagram it was processing or the connection that it > > was handling when it crashed), and as soon as POLLIN is seen on the > > socket again the service is started again can continue where it left > > off. > > > > You are apparently looking for a different model? i.e. where automatic > > restart and exit-on-idle do not exist? A model like that is currently > > not implemented. Can you elaborate on the usecase? > > > > My service is listening several sockets (which are all pased by > systemd) but depending on user configuration my service may not need > to use all sockets and in this case it would be great if the sockets > not needed stop receiving messages and theirs resources are > released. Hmm, I see. > I tried to close the socket but nothing seems to happen. > > Couldn't systemd react on the socket close and if the service still > runs then it closes the socket until it detects the service is stopped > ? "React on the socket close?" — What do you mean by that? Note that on Linux you can invoke shutdown() on a listening socket (i.e. not just on the connection socket, but on a listening socket). iirc in that case systemd actually notices and will put the .socket unit in failure mode... Would that work for you? (Maybe we could even tweak this a bit in systemd, so that when you invoke shutdown() on the socket systemd holds for you we do not consider that a failure anymore, but a clean way to tell systemd to stop the socket). Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Why systemd keeps references on passed sockets ?
On Sat, Sep 5, 2020 at 12:18 AM Lennart Poettering wrote: > > On Fr, 04.09.20 21:53, Francis Moreau (francis.m...@gmail.com) wrote: > > > Hi, > > > > I have a service which is activated by a socket unit. When systemd > > passes the (netlink) socket to my service it seems that it still keeps > > a reference on the socket, at least ss(8) showed this. > > > > Is this expected ? If yes can this be prevented ? > > > > I'm asking because my service may not need the socket in some cases > > hence it closes it but systemd keeps it open and the socket keeps > > receiving messages and consuming memory for nothing. > > It's how socket activation works really (at least with Accept=no): pid > 1 allocate the socket and activates your service every time something > happens on it, except if your service is already running. Thus a > service can auto-start, process one or more events on it, can exit > when idle, or even crash, it doesn't matter, no traffic is lost > (except the very datagram it was processing or the connection that it > was handling when it crashed), and as soon as POLLIN is seen on the > socket again the service is started again can continue where it left > off. > > You are apparently looking for a different model? i.e. where automatic > restart and exit-on-idle do not exist? A model like that is currently > not implemented. Can you elaborate on the usecase? > My service is listening several sockets (which are all pased by systemd) but depending on user configuration my service may not need to use all sockets and in this case it would be great if the sockets not needed stop receiving messages and theirs resources are released. I tried to close the socket but nothing seems to happen. Couldn't systemd react on the socket close and if the service still runs then it closes the socket until it detects the service is stopped ? -- Francis ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Why systemd keeps references on passed sockets ?
On Fr, 04.09.20 21:53, Francis Moreau (francis.m...@gmail.com) wrote: > Hi, > > I have a service which is activated by a socket unit. When systemd > passes the (netlink) socket to my service it seems that it still keeps > a reference on the socket, at least ss(8) showed this. > > Is this expected ? If yes can this be prevented ? > > I'm asking because my service may not need the socket in some cases > hence it closes it but systemd keeps it open and the socket keeps > receiving messages and consuming memory for nothing. It's how socket activation works really (at least with Accept=no): pid 1 allocate the socket and activates your service every time something happens on it, except if your service is already running. Thus a service can auto-start, process one or more events on it, can exit when idle, or even crash, it doesn't matter, no traffic is lost (except the very datagram it was processing or the connection that it was handling when it crashed), and as soon as POLLIN is seen on the socket again the service is started again can continue where it left off. You are apparently looking for a different model? i.e. where automatic restart and exit-on-idle do not exist? A model like that is currently not implemented. Can you elaborate on the usecase? Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Why systemd keeps references on passed sockets ?
Hi, I have a service which is activated by a socket unit. When systemd passes the (netlink) socket to my service it seems that it still keeps a reference on the socket, at least ss(8) showed this. Is this expected ? If yes can this be prevented ? I'm asking because my service may not need the socket in some cases hence it closes it but systemd keeps it open and the socket keeps receiving messages and consuming memory for nothing. Thank you. -- Francis ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
