Re: [systemd-devel] Deferring start of service until file exists

[Lennart Poettering]

On Mi, 21.08.19 09:32, Colin Hogben (syst...@pythontech.co.uk) wrote:

> On 20/08/19 16:54, Silvio Knizek wrote:
>
> > The NFS root needs to be mounted in the initrd step. Else everything
> > will break. With systemd in the initrd you gain the possibilty to
> > depend on those units in the actual system.
>
> Booting with an NFS root is working fine.  I'm not sure how to tell whether
> or not I have "systemd in the initrd" as you describe; I built the initrd
> with dracut, and I get the
>   random: systemd-udevd: uninitialized urandom read (16 bytes read)
> messages before
>   dracut: Mounted root filesystem ...
>
> > Why not systemd-networkd,
>
> systemd-networkd is inadequate for the task.  Even though I could
> potentially parse the ntp servers out of the lease file (is that even a
> public interface with stable format?), I didn't find a way, documented or
> undocumented, for getting the log-server.  Indeed, from scanning the source
> code it seems that any DHCP options other than some core ones and those in
> the private range, e.g. 7 (log-server), are just ignored. (Bizarrely,
> systemd-networkd does include a DHCP server - WTF?! DOTADIW?  But I
> digress...)

Please file an RFE issue that asks for the syslog field to be exposed
in our interfaces. It's easy to do just, noone did this so far.

Lennart

--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Deferring start of service until file exists

[Silvio Knizek]

Am Mittwoch, den 21.08.2019, 09:32 +0100 schrieb Colin Hogben:
> On 20/08/19 16:54, Silvio Knizek wrote:
>
> > The NFS root needs to be mounted in the initrd step. Else
> > everything will break. With systemd in the initrd you gain the
> > possibilty to depend on those units in the actual system.
>
> Booting with an NFS root is working fine.  I'm not sure how to tell
> whether or not I have "systemd in the initrd" as you describe; I
> built the initrd with dracut, and I get the
>random: systemd-udevd: uninitialized urandom read (16 bytes read)
> messages before
>dracut: Mounted root filesystem ...
Ah, than it should be fine. dracut actually does it right®
>
> > Why not systemd-networkd,
>
> systemd-networkd is inadequate for the task.  Even though I could
> potentially parse the ntp servers out of the lease file (is that even
> a public interface with stable format?), I didn't find a way,
> documented or undocumented, for getting the log-server.  Indeed, from
> scanning the source code it seems that any DHCP options other than
> some core ones and those in the private range, e.g. 7 (log-server),
> are just ignored. (Bizarrely, systemd-networkd does include a DHCP
> server - WTF?! DOTADIW?  But I digress...)
sd-networkd would tell sd-timesyncd by themself. But yeah, currently
you can't add random options to the dhcp client.
>
> > systemd-journald with remote-forwarding
> > enabled
>
> I didn't know it could do that.  Googling e.g.
> https://unix.stackexchange.com/questions/394822 and the linked
> Redhat recommendation suggest otherwise.  I found references to
> systemd-journal-netlogd (WIP? PR #1890 rejected), and some other
> systemd stuff which didn't support traditional UDP output.
sd-journald would use it's own transport via HTTP(S). See man:systemd-
journal-gatewayd, man:systemd-journal-upload and man:systemd-journal-
remote
There are also other tools to directly read from the journal and into
the appropriate backend for elasticsearch, graylog and other.
>
> > and systemd-timesyncd?
>
> Only because it isn't (wasn't) packaged for CentOS 7.  In fact I used
> it  in another project and I'd forgotten I even found a way to
> integrate it with dhclient:
> https://www.raspberrypi.org/forums/viewtopic.php?t=217832
If you're stuck with dhclient and old versions of systemd, than use
ntpd or ntpdate, no problem.
>
> > > > moreover you want daemon.service to be part of the startup
> > > > transaction (I'm not certain why)
> > >
> > > Well, I'm not sure why I wouldn't.  The whole purpose of the
> > > system is to run that service.
> > But the trigger for the .service is not start up, but accessibilty
> > and if certain information are available.
>
> Other network-facing services e.g. sshd have WantedBy=multi-
> user.target - I don't see how my service is in principle
> different.  I just has some extra dependencies.
sshd.service is totaly fine by starting without network. It sounds like
your service needs a network and information from dhclient to be up
probably.
>
> > > Having to write looping shell-scripts or reinvent inotifywait
> > > seems contrary to the spirit of systemd, but maybe I have to do
> > > that.
> > You don't have to reinvent the wheel. You just need to know about
> > the capabilities of systemd.
>
> I'm trying to learn.  How can I get a systemd unit to defer its
> start until certain information is available from dhclient?
Well, in dhclient you can define a script which is executed as soon as
there is a lease. Use this to trigger further units as this is your
trigger. See man:dhclient-script
The normal start up logic is not appliable here. You want dhclient as
trigger after writing config files.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Deferring start of service until file exists

[Colin Hogben]

On 20/08/19 16:54, Silvio Knizek wrote:


The NFS root needs to be mounted in the initrd step. Else everything
will break. With systemd in the initrd you gain the possibilty to
depend on those units in the actual system.


Booting with an NFS root is working fine.  I'm not sure how to tell 
whether or not I have "systemd in the initrd" as you describe; I built 
the initrd with dracut, and I get the

  random: systemd-udevd: uninitialized urandom read (16 bytes read)
messages before
  dracut: Mounted root filesystem ...


Why not systemd-networkd,


systemd-networkd is inadequate for the task.  Even though I could 
potentially parse the ntp servers out of the lease file (is that even a 
public interface with stable format?), I didn't find a way, documented 
or undocumented, for getting the log-server.  Indeed, from scanning the 
source code it seems that any DHCP options other than some core ones and 
those in the private range, e.g. 7 (log-server), are just ignored. 
(Bizarrely, systemd-networkd does include a DHCP server - WTF?! 
DOTADIW?  But I digress...)



systemd-journald with remote-forwarding
enabled


I didn't know it could do that.  Googling e.g. 
https://unix.stackexchange.com/questions/394822 and the linked Redhat 
recommendation suggest otherwise.  I found references to 
systemd-journal-netlogd (WIP? PR #1890 rejected), and some other systemd 
stuff which didn't support traditional UDP output.



and systemd-timesyncd?


Only because it isn't (wasn't) packaged for CentOS 7.  In fact I used it 
in another project and I'd forgotten I even found a way to integrate it 
with dhclient: https://www.raspberrypi.org/forums/viewtopic.php?t=217832



moreover you want daemon.service to be part of the startup
transaction
(I'm not certain why)


Well, I'm not sure why I wouldn't.  The whole purpose of the system
is to run that service.

But the trigger for the .service is not start up, but accessibilty and
if certain information are available.


Other network-facing services e.g. sshd have WantedBy=multi-user.target 
- I don't see how my service is in principle different.  I just has some 
extra dependencies.



Having to write looping shell-scripts or reinvent inotifywait seems
contrary to the spirit of systemd, but maybe I have to do that.

You don't have to reinvent the wheel. You just need to know about the
capabilities of systemd.


I'm trying to learn.  How can I get a systemd unit to defer its start 
until certain information is available from dhclient?


--
Colin Hogben
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Deferring start of service until file exists

[Silvio Knizek]

Am Dienstag, den 20.08.2019, 16:32 +0100 schrieb Colin Hogben:
> Hi Jérémy, thanks for responding.
>
> > I'm not very clear on what you are trying to do, so if my
> > understanding
> > is correct:
>
> OK, I'll try to clarify.  In fact I'm lumping together several
> similar circumstances.  It's possible that for at least some of
> these, I'm not thinking about things in the right way.
Try to solve one problem at a time. Even if problems seems to be
similiar the can need different solutions.
>
> The broader context is a diskless server (NFS root) with potentially
> several instances whose configuration is managed through DHCP and/or
> kernel command-line options.
The NFS root needs to be mounted in the initrd step. Else everything
will break. With systemd in the initrd you gain the possibilty to
depend on those units in the actual system.
>
> One case is that I want to run rsyslogd with messages forwarded to an
> external server.  The server address is configured via a standard
> DHCP option.  My idea was to defer starting rsyslogd until the
> configuration had been set via the dhclient hook script.  On
> reflection, perhaps the dhclient script could do a "systemctl
> reload-or-try-restart rsyslog".  Another very similar case is time
> synchronisation, using chronyd.
Why not systemd-networkd, systemd-journald with remote-forwarding
enabled and systemd-timesyncd?
>
> But now we come to the nitty-gritty.  My application needs to
> interact with a hardware device and also needs some info from DHCP
> (essentially which hosts are allowed to talk to it).  In addition,
> the application executable and libraries are on a filesystem mounted
> from a location derived from DHCP.
Than your dhclient should start the network-online.target and you
should order everything accordingly. Or you use systemd-networkd which
already have a systemd-networkd-wait-online.service
>
> > moreover you want daemon.service to be part of the startup
> > transaction
> > (I'm not certain why)
>
> Well, I'm not sure why I wouldn't.  The whole purpose of the system
> is to run that service.
But the trigger for the .service is not start up, but accessibilty and
if certain information are available.
>
> > and not triggered on the file creation as a .path
> > unit would do.
>
> One key point is that there are other dependencies.
>
> My hope was that I would be able to implement most of this in the
> systemd declarative style - the application depends on a device, on
> the existence of a file and on a mounted filesystem.  I don't
> necessarily know in which order these things will appear, but systemd
> should sort it all out for me.
There are the proper dependencies. If you need network, there is the
network-online.target.
>
> > I would create an intermediate service wait_for_file.service that
> > would be Type=oneshot and would simply trigger some sort of shell
> > script waiting for trigger_file to appear and the terminate.
> >
> > daemon.service would have Wants=wait_for_file and
> > After=wait_for_file and you should be good.
>
> Having to write looping shell-scripts or reinvent inotifywait seems
> contrary to the spirit of systemd, but maybe I have to do that.
You don't have to reinvent the wheel. You just need to know about the
capabilities of systemd.
> Thanks for the suggestions.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Deferring start of service until file exists

[Colin Hogben]

Hi Jérémy, thanks for responding.

I'm not very clear on what you are trying to do, so if my understanding 
is correct:


OK, I'll try to clarify.  In fact I'm lumping together several similar 
circumstances.  It's possible that for at least some of these, I'm not

thinking about things in the right way.

The broader context is a diskless server (NFS root) with potentially
several instances whose configuration is managed through DHCP and/or
kernel command-line options.

One case is that I want to run rsyslogd with messages forwarded to an
external server.  The server address is configured via a standard DHCP
option.  My idea was to defer starting rsyslogd until the
configuration had been set via the dhclient hook script.  On
reflection, perhaps the dhclient script could do a "systemctl
reload-or-try-restart rsyslog".  Another very similar case is time
synchronisation, using chronyd.

But now we come to the nitty-gritty.  My application needs to interact
with a hardware device and also needs some info from DHCP (essentially
which hosts are allowed to talk to it).  In addition, the application
executable and libraries are on a filesystem mounted from a location
derived from DHCP.

moreover you want daemon.service to be part of the startup transaction 
(I'm not certain why)


Well, I'm not sure why I wouldn't.  The whole purpose of the system is 
to run that service.


and not triggered on the file creation as a .path 
unit would do.


One key point is that there are other dependencies.

My hope was that I would be able to implement most of this in the
systemd declarative style - the application depends on a device, on
the existence of a file and on a mounted filesystem.  I don't
necessarily know in which order these things will appear, but systemd
should sort it all out for me.

I would create an intermediate service wait_for_file.service that would 
be Type=oneshot and would simply trigger some sort of shell script 
waiting for trigger_file to appear and the terminate.


daemon.service would have Wants=wait_for_file and After=wait_for_file 
and you should be good.


Having to write looping shell-scripts or reinvent inotifywait seems
contrary to the spirit of systemd, but maybe I have to do that.  Thanks
for the suggestions.

--
Colin Hogben
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Deferring start of service until file exists

[Jérémy ROSEN]

I'm not very clear on what you are trying to do, so if my understanding is
correct:

daemon.service : the service you are trying to delay
trigger_file : the file created by dhclient

moreover you want daemon.service to be part of the startup transaction (I'm
not certain why) and not triggered on the file creation as a .path unit
would do.

I would create an intermediate service wait_for_file.service that would be
Type=oneshot and would simply trigger some sort of shell script waiting for
trigger_file to appear and the terminate.

daemon.service would have Wants=wait_for_file and After=wait_for_file and
you should be good.


a .path would be a slightly different way of doing it that would not be
limited to the startup phase, but again i'm not completely sure what you
are trying to do

HTH
Jeremy

Le lun. 19 août 2019 à 14:49, Colin Hogben  a
écrit :

> Hi,
> (Hoping this is an appropriate place to ask this question...)
>
> During system start-up, I want to defer starting a unit (a service)
> until a particular file is written (amongst other dependencies).  In
> my case I am writing the service's configuration file within a
> dhclient hook script.
>
> Any guidance on achieving this with systemd would be much appreciated.
>
> I don't think a .path unit is the appropriate tool for the job, since
> AIUI that is intended to start a new systemd transaction rather than
> for scheduling within an existing transaction.  I tried having my
> service depend on (via Requires/After) the foo.unit related to
> foo.path, but then systemd starts foo.unit straight away regardless of
> foo.path.  If I have After=foo.unit and Requires=foo.path, then it seems
> like foo.unit is not "pulled in", and the After has no effect.  But
> maybe I missed something.
>
> The .path causes its related unit to start, whereas I think I want to
> provoke a transition in a unit from startING to startED - am I right?
>
> I thought of using the 'inotify' executable within a unit upon which
> my service could depend.  Unfortunately, CentOS (& Redhat) in their
> wisdom decided not to package inotify-tools.
>
> Thanks for any help,
> --
> Colin Hogben
> ___
> systemd-devel mailing list
> systemd-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel



-- 
[image: SMILE]  

20 rue des Jardins
92600 Asnières-sur-Seine
*Jérémy ROSEN*
Architecte technique

[image: email] jeremy.ro...@smile.fr
[image: phone]  +33 6 88 25 87 42
[image: url] http://www.smile.eu

[image: Twitter]  [image: Facebook]
 [image: LinkedIn]
 [image: Github]


[image: Découvrez l’univers Smile, rendez-vous sur smile.eu]

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Deferring start of service until file exists

[Colin Hogben]

Hi,
(Hoping this is an appropriate place to ask this question...)

During system start-up, I want to defer starting a unit (a service)
until a particular file is written (amongst other dependencies).  In
my case I am writing the service's configuration file within a
dhclient hook script.

Any guidance on achieving this with systemd would be much appreciated.

I don't think a .path unit is the appropriate tool for the job, since
AIUI that is intended to start a new systemd transaction rather than
for scheduling within an existing transaction.  I tried having my
service depend on (via Requires/After) the foo.unit related to
foo.path, but then systemd starts foo.unit straight away regardless of
foo.path.  If I have After=foo.unit and Requires=foo.path, then it seems 
like foo.unit is not "pulled in", and the After has no effect.  But 
maybe I missed something.


The .path causes its related unit to start, whereas I think I want to
provoke a transition in a unit from startING to startED - am I right?

I thought of using the 'inotify' executable within a unit upon which
my service could depend.  Unfortunately, CentOS (& Redhat) in their
wisdom decided not to package inotify-tools.

Thanks for any help,
--
Colin Hogben
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel