Re: [gentoo-user] systemd mount - what on earth is it doing ...

[antlists]

On 25/09/2021 21:30, Canek Peláez Valdés wrote:
On Sat, Sep 25, 2021 at 1:48 PM antlists > wrote:

[...]

Bear in mind, integritysetup is not in my initrd, so ...


I think you would need to put it in your initrd.

If I've got my root on it (which is the plan, but that means learning 
how to cut my own stuff into the initrd ...)



It looks to me like grub switches to the real root partition read-only
before firing stuff off, so if I tell my integritysetup.service to run
before local-fs.target, does that mean everything should be working
before systemd runs fstab?

Bear in mind dm-integrity is on my raw partitions, and I'm running lvm
and md-raid, I'm guessing they're part of local-fs-pre.target? So I
really need to run dm-integrity before that? Or preferably add
dm-integrity TO that - how do I do that?

The problem is this isn't all that discoverable - even digging around
/etc/systemd, it's hard to find clues ...


As Rich already mentioned it, noauto != remove mount point from fstab. 
That's the first thing.


Well, it's working, which is the important thing. I just want to 
understand WHY it's working - especially since when it was working the 
other way it broke Wayland ... which was why it was weird ...


Then you need to specify that home.mount should run *after* md/LVM, 
which in turn runs after your integrity setup service. The problem is: 
you have root (/) on md/LVM too; I know they are different 
RAIDs/volumes, but there is one *service* for RAID and LVM, at least 
out-of-the-box, in systemd. You could try to split different instances 
using templates (the units with @ in their names), but the simplest 
solution is to put your integrity logic in the initrd (with the added 
benefit that you can mount it in emergencies if something goes wrong 
with root).


At the moment, with root on lvm but not anything else, gentoo takes care 
of all that for me. I'm not ready for the next learning curve of doing 
all that myself in the initrd ...


If you have the integrity logic in early boot, then your integrity setup 
service should have:


Before=mdmonitor.service lvm2-lvmetad.service

And I think that should be enough? The home.mount unit depends on the 
underlying device being available, and that will happen only after md 
and LVM have done it's thing.


I'll try that. lvm should act on the root (sda3) device directly, but 
integrity needs to run before md will join sda4 and sdc4, and then lvm 
will run on the raid to make home available for fstab.


I'll play with that and get rid of the home.mount, and hope that all 
that doesn't break Wayland. What I don't understand was that when 
home.mount was enabled, it worked but broke Wayland, now I've disabled 
it it still appears to be working, but Wayland is working too! That's 
what I'd like to understand.


Although if fixing it how you suggest fixes everything, then I'll have 
an understandable, working system, and I'll be happy.


Cheers,
Wol

Re: [gentoo-user] systemd mount - what on earth is it doing ...

[Rich Freeman]

On Sat, Sep 25, 2021 at 4:30 PM Canek Peláez Valdés  wrote:
>
>  the simplest solution is to put your integrity logic in the initrd (with the 
> added benefit that you can mount it in emergencies if something goes wrong 
> with root).
>

If you want to try this, consider this article I wrote ages ago.  I no
longer use this but it is a quick recipe for how to create your own
dracut module:
https://rich0gentoo.wordpress.com/2012/01/21/a-quick-dracut-module/

It is really easy to tweak dracut, and it has a lot of automagic
functionality which means you mostly only need to worry about whatever
is particular with your setup.  You can also have your module grab any
config files and binaries/libraries needed off of your filesystem so
that they are available when it runs.

-- 
Rich

Re: [gentoo-user] systemd mount - what on earth is it doing ...

[Canek Peláez Valdés]

On Sat, Sep 25, 2021 at 1:48 PM antlists  wrote:
[...]

> Bear in mind, integritysetup is not in my initrd, so ...
>

I think you would need to put it in your initrd.


> It looks to me like grub switches to the real root partition read-only
> before firing stuff off, so if I tell my integritysetup.service to run
> before local-fs.target, does that mean everything should be working
> before systemd runs fstab?
>
> Bear in mind dm-integrity is on my raw partitions, and I'm running lvm
> and md-raid, I'm guessing they're part of local-fs-pre.target? So I
> really need to run dm-integrity before that? Or preferably add
> dm-integrity TO that - how do I do that?
>
> The problem is this isn't all that discoverable - even digging around
> /etc/systemd, it's hard to find clues ...
>

As Rich already mentioned it, noauto != remove mount point from fstab.
That's the first thing.

Then you need to specify that home.mount should run *after* md/LVM, which
in turn runs after your integrity setup service. The problem is: you have
root (/) on md/LVM too; I know they are different RAIDs/volumes, but there
is one *service* for RAID and LVM, at least out-of-the-box, in systemd. You
could try to split different instances using templates (the units with @ in
their names), but the simplest solution is to put your integrity logic in
the initrd (with the added benefit that you can mount it in emergencies if
something goes wrong with root).

If you have the integrity logic in early boot, then your integrity setup
service should have:

Before=mdmonitor.service lvm2-lvmetad.service

And I think that should be enough? The home.mount unit depends on the
underlying device being available, and that will happen only after md and
LVM have done it's thing.

Regards.
--
Dr. Canek Peláez Valdés
Profesor de Carrera Asociado C
Departamento de Matemáticas
Facultad de Ciencias
Universidad Nacional Autónoma de México

Re: [gentoo-user] systemd mount - what on earth is it doing ...

[antlists]

On 24/09/2021 21:30, Canek Peláez Valdés wrote:
On Fri, Sep 24, 2021 at 2:25 PM antlists > wrote:


This is weird ...

My /home directory is set up somewhat strangely, as in I've got two raw
partitions, I've put dm-integrity on them, raided them together, then
put lvm on top of that.


Which got me into a bind with fstab. I've created a systemd service,
which fires up dm-integrity on those two partitions. But I get the
impression it doesn't run until fstab completes. Catch-22 - fstab tries
to mount /home, but it can't until dm-integrity has made the volume
appear!


Have you tried using Before=local-fs.target in the service? Or even 
Before=local-fs-pre.target?


Bear in mind, integritysetup is not in my initrd, so ...

It looks to me like grub switches to the real root partition read-only 
before firing stuff off, so if I tell my integritysetup.service to run 
before local-fs.target, does that mean everything should be working 
before systemd runs fstab?


Bear in mind dm-integrity is on my raw partitions, and I'm running lvm 
and md-raid, I'm guessing they're part of local-fs-pre.target? So I 
really need to run dm-integrity before that? Or preferably add 
dm-integrity TO that - how do I do that?


The problem is this isn't all that discoverable - even digging around 
/etc/systemd, it's hard to find clues ...


Cheers,
Wol

Re: [gentoo-user] systemd mount - what on earth is it doing ...

[Rich Freeman]

On Sat, Sep 25, 2021 at 6:49 AM antlists  wrote:
>
> On 24/09/2021 21:30, Canek Peláez Valdés wrote:
>
> There's a lot of info about how to write a service file, but if you need
> to run before or after system service files, it's nowhere near so easy
> to find out Before=WHAT or After=WHAT :-)

So, I have a similar challenge because I use lizardfs and it can take
a while to initialize, and it isn't something systemd understands out
of the box.  (Plus it depends on a container running on the same host,
and those tend to run late, and other containers depend on the mount.)

I solved my issues with drop-ins and dependencies.

I did not create a mount unit for the mount.  I'm using fstab and its
generator to manage that.  However, I did use a drop-in so that the
automatically-created unit obtains the additional dependencies and
loads in the right order.  This requires using the right unit name as
created by the generator.

I set up dependencies for the mount unit so that it would run after
things that it needed.  I also set dependencies on services that use
the mount, so that they load later.  This wasn't something like home,
so I didn't apply those dependencies to a target.  However, if you
want to tie this into a standard target I suggest checking man
systemd.special or ask around.

> >
> > So I created a systemd mount unit for /home, which only runs after
> > dm-integrity. Great - I enabled it and it appeared to work!
> >
> >
> > You removed the /home entry from fstab, right?
> >
> mad it noauto, yes ...

That won't prevent the system from creating a unit for it - it just
won't be a dependency of local-fs.target.

I would suggest using only ONE of the following two approaches:1.
Entry in fstab plus a drop-in to modify its behavior with the same
name.
2. Custom mount unit with no entry in fstab at all (commented out is
fine, noauto is not).

It is possible you have more than one unit for the same mountpoint.
I'm not sure how exactly that behaves.  I'd just run "systemctl | grep
mount" and see what comes up - see if you have two apparent entries
for home.  If so one is probably from fstab and the other is probably
from your custom unit.  Unless they have the exact same name your unit
won't override the system one, and I'm actually not quite sure how
generators interact with unit overrides.  They seem to work fine with
drop-ins which is why I used this approach.

> > That should be home.mount. What does systemctl status home.mount says?
> >
> anthony@thewolery ~ $ systemctl status home.mount
> ● home.mount - Mount Home
>   Loaded: loaded (/etc/systemd/system/home.mount; disabled; vendor
> preset: disabled)
>
> > So what on earth is going on, and more importantly, what am I
> > misunderstanding or doing wrong. I would very much like to know why
> > it's
> > working, when I think it shouldn't be!
> >
> So fstab says noauto, home.mount is disabled, and /home is mounted on
> boot...
>

Using noauto only prevents it from being a dep for local-fs.target.
It doesn't prevent it from being mounted for some other reason.

I'd look at:
systemctl list-dependencies --reverse home.mount

Another command you might find useful is systemd-analyze dot.  The
direct output is actually readable, though intended to be converted
into a chart with graphviz using the hinted command it outputs.

If anything explicitly wants home.mount then it is going to get
mounted even if set to noauto.

Systemd is one of those things that is really capable, but you have to
grok a lot of concepts to make it dance for you, otherwise you'll feel
like you're the one being made to dance.  The internet is full of
one-liners now that it is so popular, but these often obfuscate how it
actually works.  It operates using a series of basic rules and once
you get how they're being strung together to make things work then it
is a lot easier to make it do what you expect.  Lennart's original
blog posts about systemd are very useful for this - maybe there are
some other good resources out there these days.

-- 
Rich

Re: [gentoo-user] systemd mount - what on earth is it doing ...

[antlists]

On 24/09/2021 21:30, Canek Peláez Valdés wrote:
On Fri, Sep 24, 2021 at 2:25 PM antlists > wrote:


This is weird ...

My /home directory is set up somewhat strangely, as in I've got two raw
partitions, I've put dm-integrity on them, raided them together, then
put lvm on top of that.


Which got me into a bind with fstab. I've created a systemd service,
which fires up dm-integrity on those two partitions. But I get the
impression it doesn't run until fstab completes. Catch-22 - fstab tries
to mount /home, but it can't until dm-integrity has made the volume
appear!


Have you tried using Before=local-fs.target in the service? Or even 
Before=local-fs-pre.target?


You mean in my integritysetup.service file? No, because in scrabbling 
around to try and find out what magic incantation was required, I didn't 
find it.


There's a lot of info about how to write a service file, but if you need 
to run before or after system service files, it's nowhere near so easy 
to find out Before=WHAT or After=WHAT :-)


So I created a systemd mount unit for /home, which only runs after
dm-integrity. Great - I enabled it and it appeared to work!


You removed the /home entry from fstab, right?


mad it noauto, yes ...


Only problem, startplasma-wayland now dumped me at a blank screen.

Now for the weirdo. I disabled it, thinking I would have to log in as
root, mount /home, and go from there. Except that, when I logged in,
/home was mounted and startplasma-wayland worked!


That sounds like /home was still on fstab...


#LABEL=boot /boot   ext4noauto,noatime  1 2
/dev/disk/by-uuid/F436-354A /boot   vfatnoauto,noatime  1 2
#UUID=58e72203-57d1-4497-81ad-97655bd56494  /   ext4
noatime 0 1
#LABEL=swap noneswapsw  0 0
#/dev/cdrom /mnt/cdrom  autonoauto,ro   0 0
/dev/vg-root/lv-gentoo  /   ext4rw  0   0
/dev/vg-home/lv-data/home   ext4noauto,rw   0   0
/dev/disk/by-uuid/e2b75f09-8cb4-4e20-9cfd-21c0aadfa540  noneswap 
sw,pri=10   0
/dev/disk/by-uuid/e03459f5-1f57-485b-92e4-b09cf50534c1	none	swap 
sw,pri=1	0	0


systemctl tells me mount.home is disabled, but also tells me that it
ran
and mounted the drive.


That should be home.mount. What does systemctl status home.mount says?


anthony@thewolery ~ $ systemctl status home.mount
● home.mount - Mount Home
 Loaded: loaded (/etc/systemd/system/home.mount; disabled; vendor 
preset: disabled)
 Active: active (mounted) since Fri 2021-09-24 20:13:49 BST; 2min 
51s ago

  Where: /home
   What: /dev/mapper/vg--home-lv--data
CPU: 13ms
 CGroup: /system.slice/home.mount

Sep 24 20:13:48 thewolery systemd[1]: Mounting Mount Home...
Sep 24 20:13:49 thewolery systemd[1]: Mounted Mount Home.
anthony@thewolery ~ $ cd Scans/HP-M477/


So what on earth is going on, and more importantly, what am I
misunderstanding or doing wrong. I would very much like to know why
it's
working, when I think it shouldn't be!

So fstab says noauto, home.mount is disabled, and /home is mounted on 
boot...


Cheers,
Wol

Re: [gentoo-user] systemd mount - what on earth is it doing ...

[Canek Peláez Valdés]

On Fri, Sep 24, 2021 at 2:25 PM antlists  wrote:

> This is weird ...
>
> My /home directory is set up somewhat strangely, as in I've got two raw
> partitions, I've put dm-integrity on them, raided them together, then
> put lvm on top of that.
>

> Which got me into a bind with fstab. I've created a systemd service,
> which fires up dm-integrity on those two partitions. But I get the
> impression it doesn't run until fstab completes. Catch-22 - fstab tries
> to mount /home, but it can't until dm-integrity has made the volume appear!
>

Have you tried using Before=local-fs.target in the service? Or even
Before=local-fs-pre.target?

So I created a systemd mount unit for /home, which only runs after
> dm-integrity. Great - I enabled it and it appeared to work!
>

You removed the /home entry from fstab, right?


> Only problem, startplasma-wayland now dumped me at a blank screen.
>
> Now for the weirdo. I disabled it, thinking I would have to log in as
> root, mount /home, and go from there. Except that, when I logged in,
> /home was mounted and startplasma-wayland worked!
>

That sounds like /home was still on fstab...


> systemctl tells me mount.home is disabled, but also tells me that it ran
> and mounted the drive.
>

That should be home.mount. What does systemctl status home.mount says?


> So what on earth is going on, and more importantly, what am I
> misunderstanding or doing wrong. I would very much like to know why it's
> working, when I think it shouldn't be!
>

Regards.
-- 
Dr. Canek Peláez Valdés
Profesor de Carrera Asociado C
Departamento de Matemáticas
Facultad de Ciencias
Universidad Nacional Autónoma de México

[gentoo-user] systemd mount - what on earth is it doing ...

[antlists]

This is weird ...

My /home directory is set up somewhat strangely, as in I've got two raw 
partitions, I've put dm-integrity on them, raided them together, then 
put lvm on top of that.


Which got me into a bind with fstab. I've created a systemd service, 
which fires up dm-integrity on those two partitions. But I get the 
impression it doesn't run until fstab completes. Catch-22 - fstab tries 
to mount /home, but it can't until dm-integrity has made the volume appear!


So I created a systemd mount unit for /home, which only runs after 
dm-integrity. Great - I enabled it and it appeared to work!


Only problem, startplasma-wayland now dumped me at a blank screen.

Now for the weirdo. I disabled it, thinking I would have to log in as 
root, mount /home, and go from there. Except that, when I logged in, 
/home was mounted and startplasma-wayland worked!


systemctl tells me mount.home is disabled, but also tells me that it ran 
and mounted the drive.


So what on earth is going on, and more importantly, what am I 
misunderstanding or doing wrong. I would very much like to know why it's 
working, when I think it shouldn't be!


Cheers,
Wol