Re: [arch-general] Mounting /var early in systemd
On Thursday 13 Dec 2012 10:51:02 you wrote: That's a very good point; probably all of these sockets will end up in /run rather than /var (as they once did). Maybe the issues I'm seeing after boot are not related to sockets being masked as I assumed. I have no idea what else it could be, though. I've got the unit ordered before all the sockets, but NFS is still behaving badly (losing visibility of /home and other mounts), and it works fine if the proper NFS /var is not mounted at all and the /var from the NFS root is used (but then it's not using the right /var). I'm thinking it might be something to do with /var/lib/nfs? Given that /home is mounted significantly later than /var, though, I really don't get why the system ends up losing /home with Device or resource busy. Root and /var remain fine, though... /home is mounted from fstab, and is fine if a new /var is not mounted. Paul
Re: [arch-general] Mounting /var early in systemd
On Wednesday 12 Dec 2012 17:53:23 Tom Gundersen wrote: On Wed, Dec 12, 2012 at 4:02 PM, Paul Gideon Dann pdgid...@gmail.com wrote: Is syslog.socket giving you problems? Could you paste the unit file? On my system the socket is located at /run/systemd/journal/syslog, so After=-.mount is correct. That's a very good point; probably all of these sockets will end up in /run rather than /var (as they once did). Maybe the issues I'm seeing after boot are not related to sockets being masked as I assumed. I have no idea what else it could be, though. I've got the unit ordered before all the sockets, but NFS is still behaving badly (losing visibility of /home and other mounts), and it works fine if the proper NFS /var is not mounted at all and the /var from the NFS root is used (but then it's not using the right /var). Any idea if systemd does something else in /var in very early boot? Paul
Re: [arch-general] Mounting /var early in systemd
On Wednesday 12 Dec 2012 00:40:43 Tom Gundersen wrote: Sockets in /var should automatically be ordered After=var.mount, so this should in theory just work. How are you mounting /var? I assume an fstab entry would not do in your setting, so I guess you somehow generate a custom var.mount file? Please link to the code so I could have a look. That's what I hoped too. I've tried several approaches. I'm trying a mount unit here, because I was hoping there might be a bit more magic to it. However, it does mean that I had to hardcode the mount path (%H doesn't seem to work), but if I can get this working, I have a oneshot unit that should take care of that. I created a sockets-pre.target unit, ordered before sockets.target, and the following unit is ordered before that, because I was hoping that might help. It doesn't, presumably because the socket units and this unit are all before sockets.target, and all get started at the same time. If the sockets were set after sockets-pre.target, this would probably work. (But in that case they might as well be specified directly in the unit, and the sockets-pre target can be dropped.) [Unit] Description=/var directory for the node DefaultDependencies=false Requires=sockets-pre.target Before=sockets-pre.target [Mount] What=192.168.0.1:/srv/nfs/cluster-store/vars/node07 Where=/var Type=nfs Options=v3,nolock [Install] RequiredBy=local-fs.target Bootchart is available here: http://giddie.homeip.net/screenshots/cluster-node-var-mount-boot-chart.png Thanks, Paul
Re: [arch-general] Mounting /var early in systemd
On Wed, Dec 12, 2012 at 11:10:24AM +, Paul Gideon Dann wrote: On Wednesday 12 Dec 2012 00:40:43 Tom Gundersen wrote: Sockets in /var should automatically be ordered After=var.mount, so this should in theory just work. How are you mounting /var? I assume an fstab entry would not do in your setting, so I guess you somehow generate a custom var.mount file? Please link to the code so I could have a look. That's what I hoped too. I've tried several approaches. I'm trying a mount unit here, because I was hoping there might be a bit more magic to it. However, it does mean that I had to hardcode the mount path (%H doesn't seem to work), but if I can get this working, I have a oneshot unit that should take care of that. I'd sooner use a generator than a oneshot unit to perform a mount. I created a sockets-pre.target unit, ordered before sockets.target, and the following unit is ordered before that, because I was hoping that might help. It doesn't, presumably because the socket units and this unit are all before sockets.target, and all get started at the same time. If the sockets were set after sockets-pre.target, this would probably work. (But in that case they might as well be specified directly in the unit, and the sockets-pre target can be dropped.) If sockets.target is too late, then order it before that. man bootup shows a synchronization point at sysinit.target before jobs for sockets.target are even dispatched. [Unit] Description=/var directory for the node DefaultDependencies=false Requires=sockets-pre.target Before=sockets-pre.target [Mount] What=192.168.0.1:/srv/nfs/cluster-store/vars/node07 Where=/var Type=nfs Options=v3,nolock [Install] RequiredBy=local-fs.target Bootchart is available here: http://giddie.homeip.net/screenshots/cluster-node-var-mount-boot-chart.png Thanks, Paul
Re: [arch-general] Mounting /var early in systemd
On Wed, Dec 12, 2012 at 12:10 PM, Paul Gideon Dann pdgid...@gmail.com wrote: I'm trying a mount unit here, because I was hoping there might be a bit more magic to it. Yes, it is. However, it does mean that I had to hardcode the mount path (%H doesn't seem to work) At the moment %H and friends only work on Exec*, I think it would make sense to extend it to work in more places (including Where=). but if I can get this working, I have a oneshot unit that should take care of that. As Dave said, maybe a generator would be better. I created a sockets-pre.target unit, ordered before sockets.target, and the following unit is ordered before that, because I was hoping that might help. It doesn't, presumably because the socket units and this unit are all before sockets.target, and all get started at the same time. If the sockets were set after sockets-pre.target, this would probably work. (But in that case they might as well be specified directly in the unit, and the sockets-pre target can be dropped.) [Unit] Description=/var directory for the node DefaultDependencies=false Requires=sockets-pre.target Before=sockets-pre.target [Mount] What=192.168.0.1:/srv/nfs/cluster-store/vars/node07 Where=/var Type=nfs Options=v3,nolock [Install] RequiredBy=local-fs.target I'm not able to reproduce this problem. I don't see the need for sockets-pre.target. It should be possible to simply specify your mount in /etc/fstab (obviously this only works in this test case, as the hostname will be hardcoded), and all sockets should get ordered correctly. Could you try that (remember doing systemctl daemon-reload after editing fstab) and then doing systemctl show problematic.socket | grep After to see if it worked? If it did not, please post the socket unit in question. Cheers, Tom
Re: [arch-general] Mounting /var early in systemd
On Wednesday 12 Dec 2012 14:17:34 Tom Gundersen wrote: I'm not able to reproduce this problem. I don't see the need for sockets-pre.target. It should be possible to simply specify your mount in /etc/fstab (obviously this only works in this test case, as the hostname will be hardcoded), and all sockets should get ordered correctly. Could you try that (remember doing systemctl daemon-reload after editing fstab) and then doing systemctl show problematic.socket | grep After to see if it worked? If it did not, please post the socket unit in question. Thanks for your suggestions. I added /var to fstab and rebooted, but it was ordered lower down that specifying Before=sockets.target: http://giddie.homeip.net/screenshots/cluster-node-var-mount-boot-chart- fstab.png # systemctl show syslog.socket | grep After After=-.mount Paul
Re: [arch-general] Mounting /var early in systemd
On Wednesday 12 Dec 2012 08:05:26 Dave Reisner wrote: That's what I hoped too. I've tried several approaches. I'm trying a mount unit here, because I was hoping there might be a bit more magic to it. However, it does mean that I had to hardcode the mount path (%H doesn't seem to work), but if I can get this working, I have a oneshot unit that should take care of that. I'd sooner use a generator than a oneshot unit to perform a mount. Yes, you're absolutely right. I just haven't looked into implementing generators so far. If sockets.target is too late, then order it before that. man bootup shows a synchronization point at sysinit.target before jobs for sockets.target are even dispatched. I was already ordering before sockets.target, but the problem is that the other sockets are also ordered before sockets.target, so my unit isn't necessarily ordered before any of those socket units. I wasn't aware of the bootup manpage. That's incredibly helpful; thank you. I'll try ordering before a few more of those targets and see if I get anywhere. However, based on the bootchart, none of the targets appear before the first socket unit is activated. Paul
Re: [arch-general] Mounting /var early in systemd
On Wednesday 12 Dec 2012 15:03:47 you wrote: I wasn't aware of the bootup manpage. That's incredibly helpful; thank you. I'll try ordering before a few more of those targets and see if I get anywhere. However, based on the bootchart, none of the targets appear before the first socket unit is activated. I couldn't actually figure out how the boot manpage connects with what actually happens. I certainly couldn't get /var to mount by ordering before any of those. Paul
Re: [arch-general] Mounting /var early in systemd
On Wed, Dec 12, 2012 at 4:02 PM, Paul Gideon Dann pdgid...@gmail.com wrote: Thanks for your suggestions. I added /var to fstab and rebooted, but it was ordered lower down that specifying Before=sockets.target: So no socket is After=var.mount. The question is: which one should be? On my system there should be none. http://giddie.homeip.net/screenshots/cluster-node-var-mount-boot-chart- fstab.png # systemctl show syslog.socket | grep After After=-.mount Is syslog.socket giving you problems? Could you paste the unit file? On my system the socket is located at /run/systemd/journal/syslog, so After=-.mount is correct. -t
Re: [arch-general] Mounting /var early in systemd
On Tue, Dec 11, 2012 at 4:36 PM, Paul Gideon Dann pdgid...@gmail.com wrote: I'm migrating a diskless cluster from initscripts to systemd. The problem I have is that the nodes each need to mount their own separate /var (identified by hostname) from the NFS server when they boot. I have never done something like this with systemd, but it should definitely be possible. With initscripts, I used a sysinit_end hook to do the job, which worked pretty well. However, with systemd I've really hit a brick wall. I can't get the unit to start early enough, because systemd creates so many sockets (in /var) so early, and there's no target that is ordered before the socket units are started. This means that the early sockets get masked when the real /var is mounted, and things break. I think the only solution with systemd is to explicitly specify each socket in Before= for my unit, which is likely to break if anything changes in future. It might also be possible to create a generator that automates this, but that's still pretty ugly. Sockets in /var should automatically be ordered After=var.mount, so this should in theory just work. How are you mounting /var? I assume an fstab entry would not do in your setting, so I guess you somehow generate a custom var.mount file? Please link to the code so I could have a look. Right now, I'm thinking the only sensible solution is to forget doing it with systemd and push the mounting of /var up into initcpio, but that's admitting defeat. That should definitely not be necessary. -t
