Re: [systemd-devel] tentative state and unmount on mapper
On Mon, May 18, 2015 at 3:59 PM, Umut Tezduyar Lindskog wrote: > Hi, > > There have been few discussions about tentative state and unmounting > and I am experiencing different problem in the same device logic. > > I am at 219 + 628c89cc + 496068a8 + 5259bcf6 > > I have 2 mounts (one is bind mount) on mapper device. > > /proc/self/mountinfo: > 47 37 254:0 / /var/spool/storage/SD_DISK > rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 > /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered > 49 37 254:0 / /var/spool/storage/areas/SD_DISK/root > rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 > /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered > > systemctl -t device --all | grep map: > dev-mapper-mmcblk0p1.device loaded activating tentative > /dev/mapper/mmcblk0p1 > > As soon as I unmount the bind mount, systemd picks up the change in > /proc/self/mountinfo and changes the "tentative" device to "dead" and > due to that all file systems BindsTo to the device are being > unmounted. Application which mounted the partitions is not getting a > chance to unmount the fs. > > Should I enumerate available mount units to see if anyone else has > been mounted on the device that is about to be set as DEVICE_DEAD in > device_update_found_one()? For the achieve purpose, Lennart has implemented something similar at 394763f6 and fcd8b266. Things have worked for me. But the proper solution is enabling udev awareness for DM/LVM. Umut > > Umut ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
On Tue, 19.05.15 12:43, Martin Pitt (martin.p...@ubuntu.com) wrote: > Lennart Poettering [2015-05-19 12:28 +0200]: > > > Note that it's a permanent state in containers where you don't > > > actually have udev. > > > > NO! > > > > Martin, as mentioned earlier: current systemd will not bother with > > device units at all in containers, and they hence will not be in > > "tentative" state either. > > Ok ok :) -- FTR, I can't personally stop people from misconfiguring their > containers to have writable /sys, I'm just the messenger here. > > But we've seen that there are other situations where exactly the same > situation applies (plan9 in VM, device mapper, etc.): I. e. we have a > real-iron machine or VM (writable /sys) with devices which aren't in > /dev, and thus end up being tentative. Making a container with > writable /sys is just a convenient way to reproduce/test this, I'm not > saying it's an actual use case we need to optimize for. > > So please consider my previous reply to replace "container" with > "situations with a mounted device not being in /dev". Again, devices should stay in "tentative" state only very shortly. If your device stays in "tentative" state continously after boot, then that's something to fix in your device layer. i.e. the plan9 or device mapper devices need to be fixed so that they show up propery in udev. Yes, we shouldn't unmount "tentative" devices prematurely, since the time windows exists, but to say this clearly: if you device stays in that state for a long time then that's indication that there's something wrong with your device setup, and not with systemd... You are probably missing udev rules that mark your devices as ready for systemd to pick it up. And you should fix *that*, and not rely on systemd not being too agressive with "tentative" devices. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
Lennart Poettering [2015-05-19 12:28 +0200]: > > Note that it's a permanent state in containers where you don't > > actually have udev. > > NO! > > Martin, as mentioned earlier: current systemd will not bother with > device units at all in containers, and they hence will not be in > "tentative" state either. Ok ok :) -- FTR, I can't personally stop people from misconfiguring their containers to have writable /sys, I'm just the messenger here. But we've seen that there are other situations where exactly the same situation applies (plan9 in VM, device mapper, etc.): I. e. we have a real-iron machine or VM (writable /sys) with devices which aren't in /dev, and thus end up being tentative. Making a container with writable /sys is just a convenient way to reproduce/test this, I'm not saying it's an actual use case we need to optimize for. So please consider my previous reply to replace "container" with "situations with a mounted device not being in /dev". Sorry for the confusion! Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
On Tue, 19.05.15 10:30, Martin Pitt (martin.p...@ubuntu.com) wrote: > Hello Umut, > > Umut Tezduyar Lindskog [2015-05-19 8:23 +0200]: > > On Mon, May 18, 2015 at 11:02 PM, Lennart Poettering > > wrote: > > > The "tentative" state is nothing the system should continously leave > > > devices in. It's a state only used for very short time windows, before > > > udev is up, or when a pseudo device (like a loopback block device) is > > > created and immediately mounted. If you have booted up and see a > > > device in "tentative" state, then something is really *wrong*. > > Note that it's a permanent state in containers where you don't > actually have udev. NO! Martin, as mentioned earlier: current systemd will not bother with device units at all in containers, and they hence will not be in "tentative" state either. If you run systemd git in a container and try to enqueue a job for a device you get this: # systemctl start dev-foobar.device Operation on or unit type of dev-foobar.device not supported on this system. # Moreover, systemd will not generate any .device dependencies either in this case. All this depends on /sys being mounted read-only. And that's the only scheme we support with systemd in containers. If you mount /sys writable anyway, then the fucked up .device situation is the least of your problems really. To make this clear: if *zero* interest in making systemd work in containers where /sys is writable. This is out of focus for us really. Or to say this a different way: if you leave /sys writable in a container, then we assume that you run in a scheme where /sys (and the related uevent netlink stuff) is fully virtualized, like it might be on some future kernel, where .device units and udevd would then make sense. But on the current kernel that's not the case, and to indicate that to systemd in the container you have to mount /sys read-only. > My very first patch avoided creating these device > units at all, to simplify state handling; but Lennart nack'ed this as > we want devices/mounts to exist uniformly on real iron and > containers, NO! As mentioned before and above: the way I see it .device units should *not* exist in containers, as the kernel doesn't virtualize devices for them. > which is certainly a valid point. So if we need the .devices at all, > they need to be "tentative", as they can't be "plugged" (not present > in the container /dev) nor "dead" (as that would immediately unmount > everything). NO! There will not be "tentative" nor "plugged" device units in containers, because there will be none at all! Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
On Tue, 19.05.15 08:23, Umut Tezduyar Lindskog (u...@tezduyar.com) wrote: > - Isn't there a race in that "short time window" that if one tries to > unmount stuff, things might go south! Hmm? can you elaborate? The differene between "tentative" and "plugged" is mostly about unmount-on-unplug really. > - If tentative is just a temporary state, than maybe we shouldn't send > the StartupFinished signal until device goes to plugged or dead > state. StartupFinished only cares about jobs really. It isn't suitable as a general scheme to decide whether startup is really complete. Or, to turn this around, you can already makeStartupFinished work the way you want, all you need to do is enqueue a job for the device: that job won't be considered done before the device actually showed up and entered "plugged" state. > - Seems like poky is enabling udev rules for DM. Maybe we should add > required switches on README file to make DM work. So far I found > CONFIG_DM_UEVENT on kernel and some switches on lvm, > --enable-udev_sync, --enable-udev_rules, --with-udev-prefix=. I cannot really help with DM issues, sorry. This is something to discuss with the DM folks... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
Hello Umut, Umut Tezduyar Lindskog [2015-05-19 8:23 +0200]: > On Mon, May 18, 2015 at 11:02 PM, Lennart Poettering > wrote: > > The "tentative" state is nothing the system should continously leave > > devices in. It's a state only used for very short time windows, before > > udev is up, or when a pseudo device (like a loopback block device) is > > created and immediately mounted. If you have booted up and see a > > device in "tentative" state, then something is really *wrong*. Note that it's a permanent state in containers where you don't actually have udev. My very first patch avoided creating these device units at all, to simplify state handling; but Lennart nack'ed this as we want devices/mounts to exist uniformly on real iron and containers, which is certainly a valid point. So if we need the .devices at all, they need to be "tentative", as they can't be "plugged" (not present in the container /dev) nor "dead" (as that would immediately unmount everything). > - Isn't there a race in that "short time window" that if one tries to > unmount stuff, things might go south! Correct, or a permanent situation in containers (with writable /sys at least, as a r/o sys is handled in a special case). Note that I just sent an updated patch for this: http://lists.freedesktop.org/archives/systemd-devel/2015-May/032030.html Would be nice if you could give it a try. > - Seems like poky is enabling udev rules for DM. Maybe we should add > required switches on README file to make DM work. So far I found > CONFIG_DM_UEVENT on kernel and some switches on lvm, > --enable-udev_sync, --enable-udev_rules, --with-udev-prefix=. That's still a preferable fix for non-containers of course. Thanks, Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) signature.asc Description: Digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
On Mon, May 18, 2015 at 11:02 PM, Lennart Poettering wrote: > On Mon, 18.05.15 15:59, Umut Tezduyar Lindskog (u...@tezduyar.com) wrote: > >> Hi, >> >> There have been few discussions about tentative state and unmounting >> and I am experiencing different problem in the same device logic. >> >> I am at 219 + 628c89cc + 496068a8 + 5259bcf6 >> >> I have 2 mounts (one is bind mount) on mapper device. >> >> /proc/self/mountinfo: >> 47 37 254:0 / /var/spool/storage/SD_DISK >> rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 >> /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered >> 49 37 254:0 / /var/spool/storage/areas/SD_DISK/root >> rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 >> /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered >> >> systemctl -t device --all | grep map: >> dev-mapper-mmcblk0p1.device loaded activating tentative >> /dev/mapper/mmcblk0p1 >> >> As soon as I unmount the bind mount, systemd picks up the change in >> /proc/self/mountinfo and changes the "tentative" device to "dead" and >> due to that all file systems BindsTo to the device are being >> unmounted. Application which mounted the partitions is not getting a >> chance to unmount the fs. >> >> Should I enumerate available mount units to see if anyone else has >> been mounted on the device that is about to be set as DEVICE_DEAD in >> device_update_found_one()? > > The right fix is to ensure you ship the right udev rules for your DM > setup, so that the devices are properly announced by udev. Note that > DM/LVM/... might require compile switches to be specified to enable > proper udev support. > > The "tentative" state is nothing the system should continously leave > devices in. It's a state only used for very short time windows, before > udev is up, or when a pseudo device (like a loopback block device) is > created and immediately mounted. If you have booted up and see a > device in "tentative" state, then something is really *wrong*. > > Lennart - Isn't there a race in that "short time window" that if one tries to unmount stuff, things might go south! - If tentative is just a temporary state, than maybe we shouldn't send the StartupFinished signal until device goes to plugged or dead state. - Seems like poky is enabling udev rules for DM. Maybe we should add required switches on README file to make DM work. So far I found CONFIG_DM_UEVENT on kernel and some switches on lvm, --enable-udev_sync, --enable-udev_rules, --with-udev-prefix=. Umut > > -- > Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
On Mon, 18.05.15 15:59, Umut Tezduyar Lindskog (u...@tezduyar.com) wrote: > Hi, > > There have been few discussions about tentative state and unmounting > and I am experiencing different problem in the same device logic. > > I am at 219 + 628c89cc + 496068a8 + 5259bcf6 > > I have 2 mounts (one is bind mount) on mapper device. > > /proc/self/mountinfo: > 47 37 254:0 / /var/spool/storage/SD_DISK > rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 > /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered > 49 37 254:0 / /var/spool/storage/areas/SD_DISK/root > rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 > /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered > > systemctl -t device --all | grep map: > dev-mapper-mmcblk0p1.device loaded activating tentative > /dev/mapper/mmcblk0p1 > > As soon as I unmount the bind mount, systemd picks up the change in > /proc/self/mountinfo and changes the "tentative" device to "dead" and > due to that all file systems BindsTo to the device are being > unmounted. Application which mounted the partitions is not getting a > chance to unmount the fs. > > Should I enumerate available mount units to see if anyone else has > been mounted on the device that is about to be set as DEVICE_DEAD in > device_update_found_one()? The right fix is to ensure you ship the right udev rules for your DM setup, so that the devices are properly announced by udev. Note that DM/LVM/... might require compile switches to be specified to enable proper udev support. The "tentative" state is nothing the system should continously leave devices in. It's a state only used for very short time windows, before udev is up, or when a pseudo device (like a loopback block device) is created and immediately mounted. If you have booted up and see a device in "tentative" state, then something is really *wrong*. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] tentative state and unmount on mapper
Hey Umut, Umut Tezduyar Lindskog [2015-05-18 15:59 +0200]: > I have 2 mounts (one is bind mount) on mapper device. > > /proc/self/mountinfo: > 47 37 254:0 / /var/spool/storage/SD_DISK > rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 > /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered > 49 37 254:0 / /var/spool/storage/areas/SD_DISK/root > rw,nosuid,nodev,noexec,noatime,nodiratime shared:18 - ext4 > /dev/mapper/mmcblk0p1 rw,journal_checksum,commit=1,data=ordered > > systemctl -t device --all | grep map: > dev-mapper-mmcblk0p1.device loaded activating tentative > /dev/mapper/mmcblk0p1 > > As soon as I unmount the bind mount, systemd picks up the change in > /proc/self/mountinfo and changes the "tentative" device to "dead" and > due to that all file systems BindsTo to the device are being > unmounted. Application which mounted the partitions is not getting a > chance to unmount the fs. I stumbled over this as well. This is fixed in the 0002-* patch in http://lists.freedesktop.org/archives/systemd-devel/2015-May/031953.html Direct link: http://lists.freedesktop.org/archives/systemd-devel/attachments/20150518/09d36e6f/attachment-0001.patch This is caused by the device_update_found_one() state transition from "tentative" to "dead" which we must never do as there is no way to know when a tentative device is actually dead. We must only transition to "dead" from "plugged". Testing/feedback appreciated! Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) signature.asc Description: Digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
