Re: [systemd-devel] [systemd.service] TCP listen port conflict resolution / explicit erring?

[Alec Taylor]

Hmm, maybe I'll chuck:
X-Listen-Port
X-Listen-Name

On all the generated SystemD units.

Okay, will rewrite to use .socket files. Is there a trick to overcoming the
PATH issue which avoids launching bash?

On Tue, May 2, 2017 at 10:11 PM, Lennart Poettering 
wrote:

> On Mon, 01.05.17 13:57, Alec Taylor (alec.tayl...@gmail.com) wrote:
>
> > Wrote some scripts to generate systemd .service files and start-up those
> > services.
> >
> > Currently just for some REST APIs, but soon will include distributed
> > systems also.
> >
> > I set the TCP listen port variable that is used by my .service like so:
> > `Environment=PORT=4200`
> >
> > To get PATH to work nicely, my `ExecStart` begins with:
> > /bin/bash -c 'PATH=bash -c PATH=$PREPEND_PATH:$PATH
> >
> > (would appreciate an alternative to launching a new shell there also!)
> >
> >
> >
> > *Is there a systemd trick of explicitly erring on TCP listen-port
> > conflicts?*
> > And/or should I just write a parser that loops through all .service's and
> > checks the `PORT=` and raises on conflict?
>
> No, systemd can't do that for you. And even if it did, this would be
> awfully racy as in the time between such a check and the service
> trying to ultimately bind() to it something else might already have
> taken it.
>
> I'd generally recommend using socket activation for this, i.e. the
> ".socket" unit type systemd provides. This howver requires explicit
> support in the relevant daemons. Many daemons do support that scheme
> out-of-the-box now, but the majority probably does not.
>
> Lennart
>
> --
> Lennart Poettering, Red Hat
>
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [ANNOUNCE] systemd 219

[Lennart Poettering]

On Wed, 03.05.17 22:57, Florian Kerle (flo.ke...@gmx.at) wrote:

> Hi Lennart,
> 
> it's correct, that checksums are not suitable to recover a file;
> BUT when using btrfs RAID, checksums are used to determine which copy of the 
> file is malformed.
> (and restore it, if any redundant OK copy exists)
> 
> Using FS_NOCOW on journal files does prevent btrfs from restoring the 
> journal, even if a sane copy would exist.
> (i.e. hardware / drive failure.)
> That probably means losing important data.
> 
> While this IMHO seems like a temporary workaround until btrfs autodefrag (on 
> a per file basis) exist,
> I'd rather make this configurable and surely not the default!
> 
> Do you have any further info or opinion on this?

As discussed already, there's a very easy way to disable this:

ln -s /dev/null /etc/tmpfiles.d/journal-nocow.conf

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Systemd, cgroups and short-lived processes

[Lennart Poettering]

On Wed, 03.05.17 18:02, Boris Dolgov (bo...@dolgov.name) wrote:

> Hello!
> 
> I am using systemd and repeatedly need to do the following:
> - Start a process with memory limit and cpu affinity;
> - Be notified when it and all its children are finished (typically in
> 0.01-1 second);
> - Get cpu and memory usage statistics.
> 
> Cgroups look perfect for me, but what is the best way of managing them?
> 
> As far as I understand, ControlGroupInterface article advises to
> create a transient scope unit for each process I run, but I need to
> preserve cpu accounting information, so scope units will not work.

They will. Just make sure to use a recent systemd version that
provides you with the Ref/Unref() bus calls, which permits you to ref
specific units, so that you can read statistics off them.

> Alternatively, I could create service units, but I am concerned with
> performance issues and would like to create processes by myself.
> 
> Are any lower-level APIs available and advised to use?
> For example, I'd be happy to create a slice using systemd, create its
> subtrees manually and manage them myself, but the aforementioned
> article does not recommend it.

It should be sufficient to use scopes with Ref/Unref and the CPUShares
setting.

However, the cpuset controller is still not exposed in systemd, for
the reason that its semantics are really really broken, still. We'll
add support as soon as that is cleaned up in the kernel.

However, given that the per-process CPU affinitty stuff exposed by
libc is usually enough this should not be a big limitation.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [ANNOUNCE] systemd 219

[Florian Kerle]

On Wed, 18.02.15 06:22, Andrei Borzenkov (arvidjaar at gmail.com 
) wrote:


>/В Wed, 18 Feb 2015 01:14:44 +0100 />/Zbigniew Jędrzejewski-Szmek > пишет: />//>/> On Tue, Feb 17, 2015 at 08:05:29PM +0100, Goffredo Baroncelli wrote: />/> > Hi Lennart, />/> > />/> > On 2015-02-16 23:59, Lennart Poettering wrote: />/> > > * journald now sets the special FS_NOCOW file flag for its />/> > > journal files. This should improve performance on btrfs, by />/> > > avoiding heavy fragmentation when journald's write-pattern />/> > > is used on COW file systems. It degrades btrfs' data />/> > > integrity guarantees for the files to the same levels as for />/> > > ext3/ext4 however. This should be OK though as journald does />/> > > its own data integrity checks and all its objects are />/> > > checksummed on disk. Also, journald should handle btrfs disk />/> > > full events a lot more gracefully now, by processing SIGBUS />/> > > errors, and not relying on fallocate() anymore. />/> > />/> > If I read correctly the code, the FS_NOCOW is a temporary 
workaround, i.e. />/> > when the file is closed (or rotated ?) the FS_NOCOW flags is unset 
again. />/> > It is true ? />/> Yes, but you miss the point in general. FS_NOCOW is set during the />/> entire time when the file is being written to, which could be months, />/> and then it is unset when the file will not be written to anymore. So />/> indeed, the file is not protected by btrfs checksums for the majority />/> of time, but journald does its own checksumming, so the contents are />/> protected in a different way. />/> />//>/btrfs checksumming theoretically allows you to transparently recover />/after media corruption if filesystem has redundancy (more than one copy />/of data). Journald checksum will probably detect corruption, but can it />/repair it? /

No it cannot.

But btrfs checksumming cannot fix things for you either if you lose
non-trivial amounts of data. It might be able to fix a few bits of
errors, but not non-trivial amounts. I mean, that's a simple property
of error correction codes: the more you want to be able to correct the
longer must your checksum be. Neither btrfs' nor journald's are
substantial enough to correct even a sector...

Lennart

--
Lennart Poettering, Red Hat


Hi Lennart,

it's correct, that checksums are not suitable to recover a file;
BUT when using btrfs RAID, checksums are used to determine which copy of the 
file is malformed.
(and restore it, if any redundant OK copy exists)

Using FS_NOCOW on journal files does prevent btrfs from restoring the journal, 
even if a sane copy would exist.
(i.e. hardware / drive failure.)
That probably means losing important data.

While this IMHO seems like a temporary workaround until btrfs autodefrag (on a 
per file basis) exist,
I'd rather make this configurable and surely not the default!

Do you have any further info or opinion on this?


Best regards,
Florian

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] EnableUnitFiles / DisableUnitFiles

[Chris Trobridge]

> From: Lennart Poettering 
> Sent: 03 May 2017 14:45
> To: Chris Trobridge
> Cc: systemd-devel@lists.freedesktop.org
> Subject: Re: [systemd-devel] EnableUnitFiles / DisableUnitFiles
>     
> On Tue, 02.05.17 08:38, Chris Trobridge (christrobri...@hotmail.com) wrote:
> 1;4601;0c
> > Using 230/231:
> > 
> > 
> > I just switched from using the systemd Unit interface to control services 
> > via dbus to using the systemd Manager interface so that I could control 
> > access via polkit-1.
> > 
> > 
> > Using EnableUnitFiles and DisableUnitFiles, I noticed the expected changes 
> > to symlinks were being made but the unit file status (systemctl status, 
> > dbus) was not changing.
> > 
> > 
> > Reloading the units files with systemctl reload-daemon or  dbus Manager 
> > Reload method cause the unit file state to update.
> > 
> > 
> > I don't know whether this is expected behaviour or not but I didn't find it 
> > obvious from the documentation so I am putting this out in case it helps 
> > anyone.
> 
> This is expected behaviour, and permits you to make multiple changes
> that are then put into effect in one go (though it still won't be full
> transaction scheme). "systemctl enable" will always implicitly follow
> all changes by a reload, exept if you specify --no-reload.
> 
> Lennart

Thanks Lennart, that's what I thought was most likely, and I've already taken 
advantage of this to enable several units before reloading.

Cheers,
Chris
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Systemd, cgroups and short-lived processes

[Boris Dolgov]

Hello!

I am using systemd and repeatedly need to do the following:
- Start a process with memory limit and cpu affinity;
- Be notified when it and all its children are finished (typically in
0.01-1 second);
- Get cpu and memory usage statistics.

Cgroups look perfect for me, but what is the best way of managing them?

As far as I understand, ControlGroupInterface article advises to
create a transient scope unit for each process I run, but I need to
preserve cpu accounting information, so scope units will not work.
Alternatively, I could create service units, but I am concerned with
performance issues and would like to create processes by myself.

Are any lower-level APIs available and advised to use?
For example, I'd be happy to create a slice using systemd, create its
subtrees manually and manage them myself, but the aforementioned
article does not recommend it.

Thanks,
Boris.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] template service unit include.d directory

[Anoop Alias]

Thanks.

Scheme  - /etc/systemd/system/PHP70@somet
hing.service.d/ works!





On Wed, May 3, 2017 at 7:58 PM, Tomasz Torcz  wrote:

> On Wed, May 03, 2017 at 07:51:45PM +0530, Anoop Alias wrote:
> > Hi,
> >
> > I am trying to include extra config for a template service file
> >
> > PHP70@.service
> >
> > ##
> >
> > I created /etc/systemd/system/PHP70@.service.d/something.d/test.conf
> >
> > According to the doc:
> > https://www.freedesktop.org/software/systemd/man/systemd.unit.html
> >
> > "Along with a unit file foo.service, a "drop-in" directory foo.service.d/
> > may exist. All files with the suffix ".conf" from this directory will be
> > parsed after the file itself is parsed. This is useful to alter or add
> > configuration settings for a unit, without having to modify unit files.
> > Each drop-in file must have appropriate section headers. Note that for
> > instantiated units, this logic will first look for the instance ".d/"
> > subdirectory and read its ".conf" files, followed by the template ".d/"
> > subdirectory and the ".conf" files there"
> >
> > What am I doing wrong here?
>
>   I think "instance .d/ subdirectory" would be:
>
>  /etc/systemd/system/PHP70@something.service.d/
>
>  and the "teplate .d/ subdirectory":
>
>  /etc/systemd/system/PHP70@.service.d/
>
>  rather than your scheme.
>
> > # systemctl --version
> > systemd 219
> > +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
> > +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN
> >
> > OS: CentOS Linux release 7.3.1611 (Core)
>
>   Always check local man pages. The online pages are appropriate for latest
> systed version.  The features described may not exist in such old versions
> as 219.
>
> --
> Tomasz TorczOnly gods can safely risk perfection,
> xmpp: zdzich...@chrome.pl it's a dangerous thing for a man.  -- Alia
>
> ___
> systemd-devel mailing list
> systemd-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel
>



-- 
*Anoop P Alias*
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] template service unit include.d directory

[Tomasz Torcz]

On Wed, May 03, 2017 at 07:51:45PM +0530, Anoop Alias wrote:
> Hi,
> 
> I am trying to include extra config for a template service file
> 
> PHP70@.service
> 
> ##
> 
> I created /etc/systemd/system/PHP70@.service.d/something.d/test.conf
> 
> According to the doc:
> https://www.freedesktop.org/software/systemd/man/systemd.unit.html
> 
> "Along with a unit file foo.service, a "drop-in" directory foo.service.d/
> may exist. All files with the suffix ".conf" from this directory will be
> parsed after the file itself is parsed. This is useful to alter or add
> configuration settings for a unit, without having to modify unit files.
> Each drop-in file must have appropriate section headers. Note that for
> instantiated units, this logic will first look for the instance ".d/"
> subdirectory and read its ".conf" files, followed by the template ".d/"
> subdirectory and the ".conf" files there"
> 
> What am I doing wrong here?

  I think "instance .d/ subdirectory" would be:

 /etc/systemd/system/PHP70@something.service.d/

 and the "teplate .d/ subdirectory":

 /etc/systemd/system/PHP70@.service.d/

 rather than your scheme. 

> # systemctl --version
> systemd 219
> +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
> +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN
> 
> OS: CentOS Linux release 7.3.1611 (Core)

  Always check local man pages. The online pages are appropriate for latest
systed version.  The features described may not exist in such old versions
as 219.

-- 
Tomasz TorczOnly gods can safely risk perfection,
xmpp: zdzich...@chrome.pl it's a dangerous thing for a man.  -- Alia

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] template service unit include.d directory

[Anoop Alias]

Hi,

I am trying to include extra config for a template service file

PHP70@.service

##

I created /etc/systemd/system/PHP70@.service.d/something.d/test.conf

But

systemctl start PHP70@something.service

does not include the file test.conf

According to the doc:
https://www.freedesktop.org/software/systemd/man/systemd.unit.html

"Along with a unit file foo.service, a "drop-in" directory foo.service.d/
may exist. All files with the suffix ".conf" from this directory will be
parsed after the file itself is parsed. This is useful to alter or add
configuration settings for a unit, without having to modify unit files.
Each drop-in file must have appropriate section headers. Note that for
instantiated units, this logic will first look for the instance ".d/"
subdirectory and read its ".conf" files, followed by the template ".d/"
subdirectory and the ".conf" files there"

What am I doing wrong here?

# systemctl --version
systemd 219
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
+GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN

OS: CentOS Linux release 7.3.1611 (Core)


Thanks in advance
-- 
*Anoop P Alias*
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] template service unit include.d directory

[Silvio Knizek]

Am Mittwoch, den 03.05.2017, 19:51 +0530 schrieb Anoop Alias:
> Hi,
> 
> I am trying to include extra config for a template service file
> 
> PHP70@.service
> 
> ##
> 
> I created /etc/systemd/system/PHP70@.service.d/something.d/t
> est.conf
> 
> But
> 
> systemctl start PHP70@something.service
> 
> does not include the file test.conf
> 
> According to the doc:
> https://www.freedesktop.org/software/systemd/man/systemd.unit.html
> 
> "Along with a unit file foo.service, a "drop-in" directory
> foo.service.d/
> may exist. All files with the suffix ".conf" from this directory will
> be
> parsed after the file itself is parsed. This is useful to alter or
> add
> configuration settings for a unit, without having to modify unit
> files.
> Each drop-in file must have appropriate section headers. Note that
> for
> instantiated units, this logic will first look for the instance ".d/"
> subdirectory and read its ".conf" files, followed by the template
> ".d/"
> subdirectory and the ".conf" files there"
> 
> What am I doing wrong here?
> 
> # systemctl --version
> systemd 219
> +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP
> +LIBCRYPTSETUP
> +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN
> 
> OS: CentOS Linux release 7.3.1611 (Core)
> 
> 
> Thanks in advance

Hi Anoop,

the right directory for your drop-in is /etc/systemd/system/PHP70@somet
hing.service.d/
Use »systemctl cat PHP70@something« to see what is parsed.

BR
Silvio
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] EnableUnitFiles / DisableUnitFiles

[Lennart Poettering]

On Tue, 02.05.17 08:38, Chris Trobridge (christrobri...@hotmail.com) wrote:
1;4601;0c
> Using 230/231:
> 
> 
> I just switched from using the systemd Unit interface to control services via 
> dbus to using the systemd Manager interface so that I could control access 
> via polkit-1.
> 
> 
> Using EnableUnitFiles and DisableUnitFiles, I noticed the expected changes to 
> symlinks were being made but the unit file status (systemctl status, dbus) 
> was not changing.
> 
> 
> Reloading the units files with systemctl reload-daemon or  dbus Manager 
> Reload method cause the unit file state to update.
> 
> 
> I don't know whether this is expected behaviour or not but I didn't find it 
> obvious from the documentation so I am putting this out in case it helps 
> anyone.

This is expected behaviour, and permits you to make multiple changes
that are then put into effect in one go (though it still won't be full
transaction scheme). "systemctl enable" will always implicitly follow
all changes by a reload, exept if you specify --no-reload.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel