Re: [systemd-devel] [RFC] iscsid / systemd / dracut integration effort
On Mon, 23.09.13 10:33, The Lee-Man (leeman.dun...@gmail.com) wrote: After=network.target NetworkManager-wait-online.service tgtd.service targetcli.service Please use network-online.target here instead of NetworkManager-wait-online.service. Please see the discussion of this unit on: http://www.freedesktop.org/software/systemd/man/systemd.special.html That said, this service makes local block devices available over the network, right, and it doesn't want to connect to anything on the network, or does it? If so, I really don't see the need for pulling in network-online.target/NetworkManager-wait-online.service here at all. iscsi.service: == After=network.target NetworkManager-wait-online.service iscsid.service This one on the other hand actually wants network connectivity around, right, since it tries to connect to other hosts? For this one using network-online.target sounds appropriate. ConditionPathExists=/etc/iscsi/initiatorname.iscsi [Service] Type=oneshot ExecStart=/sbin/iscsiadm -m node --loginall=automatic ExecStop=/bin/sync Wut?? Is this really necessary? If this is really necessary (which I doubt, appears to be snake oil to me...), then why doesn't iscsiadm do this internally? Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] iscsid / systemd / dracut integration effort
On Tuesday, December 11, 2012 3:26:46 PM UTC-8, Chris Leech wrote: ... Thanks, this got me going in the right direction. These unit files seems to be working much better for me, with the startup and shutdown ordering between iscsi, iscsid, and remote-fs mounts sorted out. I'm not sure about the tgtd/targetcli stuff, not sure what the original need there was. I just wanted to post a thank you for all of this work, as I was attempting to do some of it myself and making slow progress. I am trying to get open-iscsi version 2.0-873-suse working on openSUSE. In my case open-iscsi-based boot is not working and has not been working for a while. In addition, we have an initrd-based boot, and it looks like that will have to change before iscsid-based booting can work correctly. Because of that, I did not try to get iscsi-root volumes working. My goal was to get normal non-root open-iscsi working with systemd as well as it had with SysV-based init (or better). Toward that end, I have 3 unit files that are working on openSUSE 12.3 that I thought I would share. I am not submitting these as patches, since the whole systemd-integration effort is still in flux. Using the systemd configuration below, iscsid is not started until needed. NOTE: I think we need a new mode in iscsid that goes away when it's work is done, since it can now be socket-activated (i.e. it doesn't have to sit there and wait for work). Submitted for your comments: iscsid.socket: == Unit] Description=Open-iSCSI iscsid Socket Documentation=man:iscsid(8) man:iscsiadm(8) [Socket] ListenStream=@ISCSIADM_ABSTRACT_NAMESPACE [Install] WantedBy=sockets.target iscsid.service: === [Unit] Description=Open-iSCSI Documentation=man:iscsid(8) man:iscsiadm(8) After=network.target NetworkManager-wait-online.service tgtd.service targetcli.service [Service] Type=forking PIDFile=/var/run/iscsid.pid ExecStart=/sbin/iscsid [Install] WantedBy=multi-user.target iscsi.service: == [Unit] Description=Login and scanning of iSCSI devices Documentation=man:iscsiadm(8) man:iscsid(8) After=network.target NetworkManager-wait-online.service iscsid.service ConditionPathExists=/etc/iscsi/initiatorname.iscsi [Service] Type=oneshot ExecStart=/sbin/iscsiadm -m node --loginall=automatic ExecStop=/bin/sync ExecStop=/sbin/iscsiadm -m session --logout SuccessExitStatus=21 RemainAfterExit=true [Install] WantedBy=remote-fs.target ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] iscsid / systemd / dracut integration effort
On Tue, Dec 11, 2012 at 03:26:46PM -0800, Chris Leech wrote: Thanks, this got me going in the right direction. These unit files seems to be working much better for me, with the startup and shutdown ordering between iscsi, iscsid, and remote-fs mounts sorted out. I'm not sure about the tgtd/targetcli stuff, not sure what the original need there was. After tgtd was added in Fedora for cases when system accesses targets exported by the same system. So targets would be configured first, and only then accessed by iscsid. after tgtd was added, after targetcli is natural expansion of it. -- Tomasz Torcz Morality must always be based on practicality. xmpp: zdzich...@chrome.pl-- Baron Vladimir Harkonnen ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] iscsid / systemd / dracut integration effort
On 12/12/2012 12:45 PM, Tomasz Torcz wrote: On Tue, Dec 11, 2012 at 03:26:46PM -0800, Chris Leech wrote: Thanks, this got me going in the right direction. These unit files seems to be working much better for me, with the startup and shutdown ordering between iscsi, iscsid, and remote-fs mounts sorted out. I'm not sure about the tgtd/targetcli stuff, not sure what the original need there was. After tgtd was added in Fedora for cases when system accesses targets exported by the same system. So targets would be configured first, and only then accessed by iscsid. after tgtd was added, after targetcli is natural expansion of it. See, I figured target should come after initiator, so target could re-export iscsi-initiator-backed block devices :-) -- Andy ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] iscsid / systemd / dracut integration effort
On Mon, Dec 10, 2012 at 11:56:07PM +, Jóhann B. Guðmundsson wrote: On 12/10/2012 10:08 PM, Chris Leech wrote: 2) Proper ordering of the iscsi.service for non-root filesystems. Any and all feedback and/or help welcome. Looking the unit file for iscsid I do believe I missed this ( basically the same as multipath unit ) since you know it's early boot special, root and all that ;) [Unit] # Before or After lvm2-activation-early.service DefaultDependencies=no Conflicts=shutdown.target [Install] WantedBy=sysinit.target Thanks, this got me going in the right direction. These unit files seems to be working much better for me, with the startup and shutdown ordering between iscsi, iscsid, and remote-fs mounts sorted out. I'm not sure about the tgtd/targetcli stuff, not sure what the original need there was. My non-root test install with /home on iSCSI also has a swap partition on it, and that's still causing me all sorts of trouble if I don't comment it out in fstab. Trying to order all this before swap.target blew up in my face pretty good. - Chris --- iscsi.service --- [Unit] Description=Login and scanning of iSCSI devices Documentation=man:iscsiadm(8) man:iscsid(8) DefaultDependencies=no Conflicts=shutdown.target After=systemd-remount-fs.service network.target iscsid.service iscsiuio.service Before=remote-fs.target ConditionPathExists=/etc/iscsi/initiatorname.iscsi [Service] Type=oneshot ExecStart=/usr/libexec/iscsi_mark_root_nodes ExecStart=/sbin/iscsiadm -m node --loginall=automatic ExecStop=/bin/sync ExecStop=/sbin/iscsiadm -m node --logoutall=automatic RemainAfterExit=true [Install] WantedBy=sysinit.target --- iscsid.service --- [Unit] Description=Open-iSCSI Documentation=man:iscsid(8) man:iscsiadm(8) DefaultDependencies=no Conflicts=shutdown.target After=network.target tgtd.service targetcli.service Before=remote-fs-pre.target [Service] Type=simple ExecStart=/usr/sbin/iscsid -f -n ExecStop=/sbin/iscsiadm -k 0 2 --- iscsid.socket --- [Unit] Description=Open-iSCSI iscsid Socket Documentation=man:iscsid(8) man:iscsiadm(8) [Socket] ListenStream=@ISCSIADM_ABSTRACT_NAMESPACE [Install] WantedBy=sockets.target ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] iscsid / systemd / dracut integration effort
On 12/11/2012 11:26 PM, Chris Leech wrote: On Mon, Dec 10, 2012 at 11:56:07PM +, Jóhann B. Guðmundsson wrote: On 12/10/2012 10:08 PM, Chris Leech wrote: 2) Proper ordering of the iscsi.service for non-root filesystems. Any and all feedback and/or help welcome. Looking the unit file for iscsid I do believe I missed this ( basically the same as multipath unit ) since you know it's early boot special, root and all that ;) [Unit] # Before or After lvm2-activation-early.service DefaultDependencies=no Conflicts=shutdown.target [Install] WantedBy=sysinit.target Thanks, this got me going in the right direction. These unit files seems to be working much better for me, with the startup and shutdown ordering between iscsi, iscsid, and remote-fs mounts sorted out. I'm not sure about the tgtd/targetcli stuff, not sure what the original need there was. You should remove if it's not necessary for iscsi to function and the ordering for iscsi be put in those units instead My non-root test install with /home on iSCSI also has a swap partition on it, and that's still causing me all sorts of trouble if I don't comment it out in fstab. Hmm even if you add noauto after the relevant swap entry in /etc/fstab actually it might be necessary for you to create a .swap unit for it. ( man systemd.swap for details ) Trying to order all this before swap.target blew up in my face pretty good. You should not order it after the swap.target however you can order .swap unit before the swap.target - Chris --- iscsi.service --- [Unit] Description=Login and scanning of iSCSI devices Documentation=man:iscsiadm(8) man:iscsid(8) DefaultDependencies=no Conflicts=shutdown.target After=systemd-remount-fs.service network.target iscsid.service iscsiuio.service Before=remote-fs.target ConditionPathExists=/etc/iscsi/initiatorname.iscsi [Service] Type=oneshot ExecStart=/usr/libexec/iscsi_mark_root_nodes ExecStart=/sbin/iscsiadm -m node --loginall=automatic ExecStop=/bin/sync ExecStop=/sbin/iscsiadm -m node --logoutall=automatic RemainAfterExit=true [Install] WantedBy=sysinit.target --- iscsid.service --- [Unit] Description=Open-iSCSI Documentation=man:iscsid(8) man:iscsiadm(8) DefaultDependencies=no Conflicts=shutdown.target After=network.target tgtd.service targetcli.service Before=remote-fs-pre.target [Service] Type=simple ExecStart=/usr/sbin/iscsid -f -n ExecStop=/sbin/iscsiadm -k 0 2 --- iscsid.socket --- [Unit] Description=Open-iSCSI iscsid Socket Documentation=man:iscsid(8) man:iscsiadm(8) [Socket] ListenStream=@ISCSIADM_ABSTRACT_NAMESPACE [Install] WantedBy=sockets.target So it seems to be in ready enough to be pushed to rawhide and the next step is to boot with systemd.log_level=debug systemd.log_target=kmsg log_buf_len=1M enforcing=0 in the kernel command line to actually see why that's blowing up in your face. One question the iscsid is calling fork() right so why are you so incline to use type simple for it? There is an ordering gotcha when using type simple units which boils to systemd has no way of knowing if what it just fired up is finished running/setup etc. ( unless notified ) so you might find yourself in a race condition. JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [RFC] iscsid / systemd / dracut integration effort
Sorry about the cross-posting, but I think that in order to get this right I'm going to need as much feedback as I can get. I think I've got this working pretty well now (at least for my iBFT configured iSCSI root test case), although the dracut module still needs work. This is based on the iscsid socket activation patch from Tomasz Torcz and the unit files from Tomasz and Jóhann B. Guðmundsson. Attempting to start iscsid using systemd unit files (socket activated or not) was impacting my iscsi-root test setup. Rather than try to preserve the old logic that used the _netdev mount flag to determine when iscsid could be safely shutdown, I've been working on following the guidelines for root storage daemons http://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons. 1) I patched iscsid to set argv[0][0] = '@' if a new --initrd flag is passed on the command line. It's simple, it works, we don't kill iscsid before the root filesystem is unmounted. 2) When starting iscsid from the initrd, I then ran into issues of iscsid not functioning properly after the switch_root. It's not mentioned in the RootStorageDaemons documentation, but if a process left running from the initrd needs further filesystem access at runtime it will need to be able to chroot itself before the switch_root. I patched iscsid and iscsiadm with a newroot command (using plymouth as an example), and used a dracut clenaup hook to call iscsiadm --newroot /sysroot. 3) Related to #2, launching iscsid as a Type=simple service from the initrd (I'm testing this with Fedora 18 beta, where dracut uses systemd in the initrd) results in STDOUT/STDERR access after the switch_root failing with EPIPE. It seems that the connection to the journal process is dead once the initrd instantiated systemd-journald is replaced with the runtime process. I can at least keep iscsid functioning if I run it as Type=forking, but I'm not sure if I'm losing runtime logging or not. I may be able to do something with the StandardOutput and StandardError options in the service file. 4) None of this helps if the sessions are shutdown when we still need them. Rather than the all-or-nothing _netdev mount option checks that have been used in the past, I'm attempting to use the fact that iscsiadm will skip over session that match a node with startup=onboot set when doing a logoutall. The iscsi_mark_root_nodes script is borrowed from the SUSE init scripts in the open-iscsi repos written by Hannes Reinecke. I've made minor changes to export the list of sessions activated in the initramfs, and then mark those as onboot when starting iscsi.service. Things that still need looking into. 1) Command line configured iSCSI parameters. If iscsistart is to be fully removed something needs to setup the node from the command line. Or iscsistart needs to be able to run alongside iscsid. 2) Proper ordering of the iscsi.service for non-root filesystems. Any and all feedback and/or help welcome. - Chris ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] iscsid / systemd / dracut integration effort
On 12/10/2012 10:08 PM, Chris Leech wrote: Sorry about the cross-posting, but I think that in order to get this right I'm going to need as much feedback as I can get. I think I've got this working pretty well now (at least for my iBFT configured iSCSI root test case), although the dracut module still needs work. This is based on the iscsid socket activation patch from Tomasz Torcz and the unit files from Tomasz and Jóhann B. Guðmundsson. Attempting to start iscsid using systemd unit files (socket activated or not) was impacting my iscsi-root test setup. Rather than try to preserve the old logic that used the _netdev mount flag to determine when iscsid could be safely shutdown, I've been working on following the guidelines for root storage daemons http://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons. 1) I patched iscsid to set argv[0][0] = '@' if a new --initrd flag is passed on the command line. It's simple, it works, we don't kill iscsid before the root filesystem is unmounted. 2) When starting iscsid from the initrd, I then ran into issues of iscsid not functioning properly after the switch_root. It's not mentioned in the RootStorageDaemons documentation, but if a process left running from the initrd needs further filesystem access at runtime it will need to be able to chroot itself before the switch_root. I patched iscsid and iscsiadm with a newroot command (using plymouth as an example), and used a dracut clenaup hook to call iscsiadm --newroot /sysroot. 3) Related to #2, launching iscsid as a Type=simple service from the initrd (I'm testing this with Fedora 18 beta, where dracut uses systemd in the initrd) results in STDOUT/STDERR access after the switch_root failing with EPIPE. It seems that the connection to the journal process is dead once the initrd instantiated systemd-journald is replaced with the runtime process. I can at least keep iscsid functioning if I run it as Type=forking, but I'm not sure if I'm losing runtime logging or not. I may be able to do something with the StandardOutput and StandardError options in the service file. You should be using Type=forking here 4) None of this helps if the sessions are shutdown when we still need them. Rather than the all-or-nothing _netdev mount option checks that have been used in the past, I'm attempting to use the fact that iscsiadm will skip over session that match a node with startup=onboot set when doing a logoutall. The iscsi_mark_root_nodes script is borrowed from the SUSE init scripts in the open-iscsi repos written by Hannes Reinecke. I've made minor changes to export the list of sessions activated in the initramfs, and then mark those as onboot when starting iscsi.service. Things that still need looking into. 1) Command line configured iSCSI parameters. If iscsistart is to be fully removed something needs to setup the node from the command line. Or iscsistart needs to be able to run alongside iscsid. 2) Proper ordering of the iscsi.service for non-root filesystems. Any and all feedback and/or help welcome. Looking the unit file for iscsid I do believe I missed this ( basically the same as multipath unit ) since you know it's early boot special, root and all that ;) [Unit] # Before or After lvm2-activation-early.service DefaultDependencies=no Conflicts=shutdown.target [Install] WantedBy=sysinit.target JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
