Re: Troubleshooting timedatectl and hostnamectl failed to activate service: timed out
It appears that I can fix this by editing the unit file and changing: ProtectSystem=strict to: ProtectSystem=full I'm not sure why that is but a resolution is good enough for me. Following up on my own thread in case this helps someone else in the future. Best, Sean On Thu, Dec 14, 2023 at 12:02 PM Sean Caron wrote: > Sorry, perhaps apparmor is not completely disabled, but I don't think it's > enforcing. I tried shutting it off completely with: > > systemctl stop apparmor > > And that doesn't seem to have made a difference. > > Best, > > Sean > > > On Thu, Dec 14, 2023 at 11:57 AM Sean Caron wrote: > >> Hi Mantas, >> >> Thanks for the suggestions! I took a look and I'm seeing entries like the >> following in the logs: >> >> Starting Hostname Service... >> systemd-hostnamed.service: Failed to set up mount namespacing: >> /run/systemd/unit-root/: Invalid argument >> systemd-hostnamed.service: Failed at step NAMESPACE spawning >> /lib/systemd/systemd-hostnamed: Invalid argument >> systemd-hostnamed.service: Main process exited, code=exited, >> status=226/NAMESPACE >> systemd-hostnamed.service: Failed with result 'exit-code'. >> Failed to start Hostname Service. >> >> Starting Time & Date Service... >> systemd-timedated.service: Failed to set up mount namespacing: >> /run/systemd/unit-root/: Invalid argument >> systemd-timedated.service: Failed at step NAMESPACE spawning >> /lib/systemd/systemd-timedated: Invalid argument >> systemd-timedated.service: Main process exited, code=exited, >> status=226/NAMESPACE >> systemd-timedated.service: Failed with result 'exit-code'. >> Failed to start Time & Date Service. >> >> Apparmor is disabled on all of our systems. >> >> The /run/systemd/unit-root directory exists on both working and >> nonworking systems and the ownership and permissions are identical on >> working and nonworking systems. >> >> I'm unfortunately not very conversant with everything that systemd does >> behind the scenes with this namespacing stuff. Does this raise any obvious >> flags? Any ideas for how I could further debug and/or resolve this would be >> so greatly appreciated! >> >> Best, >> >> Sean >> >> On Wed, Dec 13, 2023 at 1:22 PM Mantas Mikulėnas >> wrote: >> >>> Activation is not client-side, it's handled automatically by dbus-daemon >>> – which either spawns the service directly or starts it as a systemd >>> service. >>> >>> In this case, check whether your logs show systemd-hostnamed.service >>> attempting to start; either it fails to start (missing libraries? >>> Apparmor?) or dbus-daemon fails to contact systemd (pid1 crashed?). >>> >>> On Wed, Dec 13, 2023, 19:45 Sean Caron wrote: >>> >>>> Hi everyone, >>>> >>>> I'm on Ubuntu 20.04 LTS, kernel version 5.4.0-163-generic, systemd 245 >>>> (245.4-4ubuntu3.22). >>>> >>>> I have some systems where I am receiving the following error messages >>>> when people attempt to use timedatectl or hostnamectl: >>>> >>>> >>>> Failed to query server: Failed to activate service >>>> 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) >>>> >>>> Failed to query system properties: Failed to activate service >>>> 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) >>>> >>>> >>>> I tried setting SYSTEMD_LOG_LEVEL=debug and rerunning the commands and >>>> it didn't really give me anything useful for determining the root cause of >>>> the issue. Here's an example of that output for timedatectl status: >>>> >>>> >>>> Bus n/a: changing state UNSET → OPENING >>>> Bus n/a: changing state OPENING → AUTHENTICATING >>>> Bus n/a: changing state AUTHENTICATING → HELLO >>>> Sent message type=method_call sender=n/a >>>> destination=org.freedesktop.DBus path=/org/freedesktop/DBus >>>> interface=org.freedesktop.DBus member=Hello cookie=1 reply_cookie=0 >>>> signature=n/a error-name=n/a error-message=n/a >>>> Got message type=method_return sender=org.freedesktop.DBus >>>> destination=:1.15318 path=n/a interface=n/a member=n/a cookie=1 >>>> reply_cookie=1 signature=s error-name=n/a error-message=n/a >>>> Bus n/a: changing state HELLO → RUNNING >>>> Sent message type=method_call sender=n/a >>>> destination=org.freedesktop.timedate1 pa
Re: Troubleshooting timedatectl and hostnamectl failed to activate service: timed out
Sorry, perhaps apparmor is not completely disabled, but I don't think it's enforcing. I tried shutting it off completely with: systemctl stop apparmor And that doesn't seem to have made a difference. Best, Sean On Thu, Dec 14, 2023 at 11:57 AM Sean Caron wrote: > Hi Mantas, > > Thanks for the suggestions! I took a look and I'm seeing entries like the > following in the logs: > > Starting Hostname Service... > systemd-hostnamed.service: Failed to set up mount namespacing: > /run/systemd/unit-root/: Invalid argument > systemd-hostnamed.service: Failed at step NAMESPACE spawning > /lib/systemd/systemd-hostnamed: Invalid argument > systemd-hostnamed.service: Main process exited, code=exited, > status=226/NAMESPACE > systemd-hostnamed.service: Failed with result 'exit-code'. > Failed to start Hostname Service. > > Starting Time & Date Service... > systemd-timedated.service: Failed to set up mount namespacing: > /run/systemd/unit-root/: Invalid argument > systemd-timedated.service: Failed at step NAMESPACE spawning > /lib/systemd/systemd-timedated: Invalid argument > systemd-timedated.service: Main process exited, code=exited, > status=226/NAMESPACE > systemd-timedated.service: Failed with result 'exit-code'. > Failed to start Time & Date Service. > > Apparmor is disabled on all of our systems. > > The /run/systemd/unit-root directory exists on both working and nonworking > systems and the ownership and permissions are identical on working and > nonworking systems. > > I'm unfortunately not very conversant with everything that systemd does > behind the scenes with this namespacing stuff. Does this raise any obvious > flags? Any ideas for how I could further debug and/or resolve this would be > so greatly appreciated! > > Best, > > Sean > > On Wed, Dec 13, 2023 at 1:22 PM Mantas Mikulėnas > wrote: > >> Activation is not client-side, it's handled automatically by dbus-daemon >> – which either spawns the service directly or starts it as a systemd >> service. >> >> In this case, check whether your logs show systemd-hostnamed.service >> attempting to start; either it fails to start (missing libraries? >> Apparmor?) or dbus-daemon fails to contact systemd (pid1 crashed?). >> >> On Wed, Dec 13, 2023, 19:45 Sean Caron wrote: >> >>> Hi everyone, >>> >>> I'm on Ubuntu 20.04 LTS, kernel version 5.4.0-163-generic, systemd 245 >>> (245.4-4ubuntu3.22). >>> >>> I have some systems where I am receiving the following error messages >>> when people attempt to use timedatectl or hostnamectl: >>> >>> >>> Failed to query server: Failed to activate service >>> 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) >>> >>> Failed to query system properties: Failed to activate service >>> 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) >>> >>> >>> I tried setting SYSTEMD_LOG_LEVEL=debug and rerunning the commands and >>> it didn't really give me anything useful for determining the root cause of >>> the issue. Here's an example of that output for timedatectl status: >>> >>> >>> Bus n/a: changing state UNSET → OPENING >>> Bus n/a: changing state OPENING → AUTHENTICATING >>> Bus n/a: changing state AUTHENTICATING → HELLO >>> Sent message type=method_call sender=n/a >>> destination=org.freedesktop.DBus path=/org/freedesktop/DBus >>> interface=org.freedesktop.DBus member=Hello cookie=1 reply_cookie=0 >>> signature=n/a error-name=n/a error-message=n/a >>> Got message type=method_return sender=org.freedesktop.DBus >>> destination=:1.15318 path=n/a interface=n/a member=n/a cookie=1 >>> reply_cookie=1 signature=s error-name=n/a error-message=n/a >>> Bus n/a: changing state HELLO → RUNNING >>> Sent message type=method_call sender=n/a >>> destination=org.freedesktop.timedate1 path=/org/freedesktop/timedate1 >>> interface=org.freedesktop.DBus.Properties member=GetAll cookie=2 >>> reply_cookie=0 signature=s error-name=n/a error-message=n/a >>> Got message type=error sender=org.freedesktop.DBus destination=:1.15318 >>> path=n/a interface=n/a member=n/a cookie=3 reply_cookie=2 signature=s >>> error-name=org.freedesktop.DBus.Error.TimedOut error-message=Failed to >>> activate service 'org.freedesktop.timedate1': timed out >>> (service_start_timeout=25000ms) >>> Failed to query server: Failed to activate service >>> 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) >>> Bus n/a: changing state RUNNI
Re: Troubleshooting timedatectl and hostnamectl failed to activate service: timed out
Hi Mantas, Thanks for the suggestions! I took a look and I'm seeing entries like the following in the logs: Starting Hostname Service... systemd-hostnamed.service: Failed to set up mount namespacing: /run/systemd/unit-root/: Invalid argument systemd-hostnamed.service: Failed at step NAMESPACE spawning /lib/systemd/systemd-hostnamed: Invalid argument systemd-hostnamed.service: Main process exited, code=exited, status=226/NAMESPACE systemd-hostnamed.service: Failed with result 'exit-code'. Failed to start Hostname Service. Starting Time & Date Service... systemd-timedated.service: Failed to set up mount namespacing: /run/systemd/unit-root/: Invalid argument systemd-timedated.service: Failed at step NAMESPACE spawning /lib/systemd/systemd-timedated: Invalid argument systemd-timedated.service: Main process exited, code=exited, status=226/NAMESPACE systemd-timedated.service: Failed with result 'exit-code'. Failed to start Time & Date Service. Apparmor is disabled on all of our systems. The /run/systemd/unit-root directory exists on both working and nonworking systems and the ownership and permissions are identical on working and nonworking systems. I'm unfortunately not very conversant with everything that systemd does behind the scenes with this namespacing stuff. Does this raise any obvious flags? Any ideas for how I could further debug and/or resolve this would be so greatly appreciated! Best, Sean On Wed, Dec 13, 2023 at 1:22 PM Mantas Mikulėnas wrote: > Activation is not client-side, it's handled automatically by dbus-daemon – > which either spawns the service directly or starts it as a systemd service. > > In this case, check whether your logs show systemd-hostnamed.service > attempting to start; either it fails to start (missing libraries? > Apparmor?) or dbus-daemon fails to contact systemd (pid1 crashed?). > > On Wed, Dec 13, 2023, 19:45 Sean Caron wrote: > >> Hi everyone, >> >> I'm on Ubuntu 20.04 LTS, kernel version 5.4.0-163-generic, systemd 245 >> (245.4-4ubuntu3.22). >> >> I have some systems where I am receiving the following error messages >> when people attempt to use timedatectl or hostnamectl: >> >> >> Failed to query server: Failed to activate service >> 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) >> >> Failed to query system properties: Failed to activate service >> 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) >> >> >> I tried setting SYSTEMD_LOG_LEVEL=debug and rerunning the commands and it >> didn't really give me anything useful for determining the root cause of the >> issue. Here's an example of that output for timedatectl status: >> >> >> Bus n/a: changing state UNSET → OPENING >> Bus n/a: changing state OPENING → AUTHENTICATING >> Bus n/a: changing state AUTHENTICATING → HELLO >> Sent message type=method_call sender=n/a destination=org.freedesktop.DBus >> path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello >> cookie=1 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a >> Got message type=method_return sender=org.freedesktop.DBus >> destination=:1.15318 path=n/a interface=n/a member=n/a cookie=1 >> reply_cookie=1 signature=s error-name=n/a error-message=n/a >> Bus n/a: changing state HELLO → RUNNING >> Sent message type=method_call sender=n/a >> destination=org.freedesktop.timedate1 path=/org/freedesktop/timedate1 >> interface=org.freedesktop.DBus.Properties member=GetAll cookie=2 >> reply_cookie=0 signature=s error-name=n/a error-message=n/a >> Got message type=error sender=org.freedesktop.DBus destination=:1.15318 >> path=n/a interface=n/a member=n/a cookie=3 reply_cookie=2 signature=s >> error-name=org.freedesktop.DBus.Error.TimedOut error-message=Failed to >> activate service 'org.freedesktop.timedate1': timed out >> (service_start_timeout=25000ms) >> Failed to query server: Failed to activate service >> 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) >> Bus n/a: changing state RUNNING → CLOSED >> >> >> I read that sometimes these issues can be caused by filesystem >> permissions on subdirectories in /var such as /var/tmp or /var/lib/systemd >> but I checked these and compared against a working system and I don't see >> any obvious differences. >> >> I have tried using strace on timedatectl and hostnamectl to try and see >> what's hanging things up but that hasn't really provided any fruitful >> direction, either. >> >> I didn't really know this was occurring until an end user reported it to >> me so I don't necessarily know how long the issue has been occurring or >> have a change in m
Troubleshooting timedatectl and hostnamectl failed to activate service: timed out
Hi everyone, I'm on Ubuntu 20.04 LTS, kernel version 5.4.0-163-generic, systemd 245 (245.4-4ubuntu3.22). I have some systems where I am receiving the following error messages when people attempt to use timedatectl or hostnamectl: Failed to query server: Failed to activate service 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) Failed to query system properties: Failed to activate service 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) I tried setting SYSTEMD_LOG_LEVEL=debug and rerunning the commands and it didn't really give me anything useful for determining the root cause of the issue. Here's an example of that output for timedatectl status: Bus n/a: changing state UNSET → OPENING Bus n/a: changing state OPENING → AUTHENTICATING Bus n/a: changing state AUTHENTICATING → HELLO Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello cookie=1 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a Got message type=method_return sender=org.freedesktop.DBus destination=:1.15318 path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=s error-name=n/a error-message=n/a Bus n/a: changing state HELLO → RUNNING Sent message type=method_call sender=n/a destination=org.freedesktop.timedate1 path=/org/freedesktop/timedate1 interface=org.freedesktop.DBus.Properties member=GetAll cookie=2 reply_cookie=0 signature=s error-name=n/a error-message=n/a Got message type=error sender=org.freedesktop.DBus destination=:1.15318 path=n/a interface=n/a member=n/a cookie=3 reply_cookie=2 signature=s error-name=org.freedesktop.DBus.Error.TimedOut error-message=Failed to activate service 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) Failed to query server: Failed to activate service 'org.freedesktop.timedate1': timed out (service_start_timeout=25000ms) Bus n/a: changing state RUNNING → CLOSED I read that sometimes these issues can be caused by filesystem permissions on subdirectories in /var such as /var/tmp or /var/lib/systemd but I checked these and compared against a working system and I don't see any obvious differences. I have tried using strace on timedatectl and hostnamectl to try and see what's hanging things up but that hasn't really provided any fruitful direction, either. I didn't really know this was occurring until an end user reported it to me so I don't necessarily know how long the issue has been occurring or have a change in mind that could have broken things. I'm not sure if the upgrade from Ubuntu 18 to Ubuntu 20 broke it, or if some security configuration broke it. Or perhaps there is a missing dependency package on the broken systems? Could anyone out there please provide a little bit more guidance on how I might troubleshoot this and determine the root cause of the issue? I really hate to bother folks here but I'm feeling stuck. Thank you! Sean
Re: [systemd-devel] Systemd hang when restarting a service during shutdown
On Tue, Nov 09, 2021 at 09:30:05AM +0100, Sean Nyekjaer wrote: > On Mon, Nov 08, 2021 at 03:00:42PM +0100, Sean Nyekjaer wrote: > > On Mon, Nov 08, 2021 at 03:47:36PM +0200, Uoti Urpala wrote: > > > On Mon, 2021-11-08 at 12:05 +0100, Sean Nyekjaer wrote: > > > > Regarding, > > > > https://github.com/systemd/systemd/issues/21203 > > > > > > > > I think the point of the issue missed when the issue got closed. > > > > > > > > We have a service that is changing configs for systemd-networkd and > > > > issuing a `systemctl restart systemd-networkd`. > > > > An other service is checking uptime and issues a `systemctl reboot`, > > > > when our max uptime have been exeeced. > > > > If restart of systemd-networkd happens while a reboot is in progress, > > > > the system will hang "forever" (and continue to pet the watchdog). > > > > > > The issue shows you using "systemctl start systemd-reboot". That is not > > > the right way to reboot. Use "systemctl reboot" instead. I suspect this > > > is related to why the reboot may stop partway: your command does not > > > start the reboot tasks in "irreversible" mode, which means that any > > > following contrary command, such as explicitly (re)starting a unit that > > > was going to be shut down, is going to implicitly cancel the > > > conflicting reboot action. > > > > > > You should also be using "try-restart" instead of "restart". If your > > > intent is to change configs, you want to say "make sure old configs are > > > not in use" rather than "enforce that the service is running now". (I > > > think making the "restart" command have "start" semantics was a design > > > mistake, and the "try-restart"/"restart" pair would have been better > > > named "restart"/"start-or-restart".) > > > > I can reproduce it with "systemctl reboot"... > > > > I'll try the "try-restart" option, and report back > > > > Thanks > > > > "try-restart" does not fix it for the example above. > But it does with the original issue where we first noticed this... > > https://github.com/systemd/systemd/issues/4955#issuecomment-506234412 > > $ mkdir /run/systemd/network/ > $ vi /run/systemd/network/something.network > corrupt the file! make it look something like > """ > [Match] > Name=eth* > > [Net > """ > $ systemctl try-restart systemd-networkd; systemctl reboot > > Looks like a fix / workaround ^^ Scratch that... Still hangs
Re: [systemd-devel] Systemd hang when restarting a service during shutdown
On Mon, Nov 08, 2021 at 03:00:42PM +0100, Sean Nyekjaer wrote: > On Mon, Nov 08, 2021 at 03:47:36PM +0200, Uoti Urpala wrote: > > On Mon, 2021-11-08 at 12:05 +0100, Sean Nyekjaer wrote: > > > Regarding, > > > https://github.com/systemd/systemd/issues/21203 > > > > > > I think the point of the issue missed when the issue got closed. > > > > > > We have a service that is changing configs for systemd-networkd and > > > issuing a `systemctl restart systemd-networkd`. > > > An other service is checking uptime and issues a `systemctl reboot`, > > > when our max uptime have been exeeced. > > > If restart of systemd-networkd happens while a reboot is in progress, > > > the system will hang "forever" (and continue to pet the watchdog). > > > > The issue shows you using "systemctl start systemd-reboot". That is not > > the right way to reboot. Use "systemctl reboot" instead. I suspect this > > is related to why the reboot may stop partway: your command does not > > start the reboot tasks in "irreversible" mode, which means that any > > following contrary command, such as explicitly (re)starting a unit that > > was going to be shut down, is going to implicitly cancel the > > conflicting reboot action. > > > > You should also be using "try-restart" instead of "restart". If your > > intent is to change configs, you want to say "make sure old configs are > > not in use" rather than "enforce that the service is running now". (I > > think making the "restart" command have "start" semantics was a design > > mistake, and the "try-restart"/"restart" pair would have been better > > named "restart"/"start-or-restart".) > > I can reproduce it with "systemctl reboot"... > > I'll try the "try-restart" option, and report back > > Thanks > "try-restart" does not fix it for the example above. But it does with the original issue where we first noticed this... https://github.com/systemd/systemd/issues/4955#issuecomment-506234412 $ mkdir /run/systemd/network/ $ vi /run/systemd/network/something.network corrupt the file! make it look something like """ [Match] Name=eth* [Net """ $ systemctl try-restart systemd-networkd; systemctl reboot Looks like a fix / workaround /Sean
Re: [systemd-devel] Systemd hang when restarting a service during shutdown
On Mon, Nov 08, 2021 at 03:47:36PM +0200, Uoti Urpala wrote: > On Mon, 2021-11-08 at 12:05 +0100, Sean Nyekjaer wrote: > > Regarding, > > https://github.com/systemd/systemd/issues/21203 > > > > I think the point of the issue missed when the issue got closed. > > > > We have a service that is changing configs for systemd-networkd and > > issuing a `systemctl restart systemd-networkd`. > > An other service is checking uptime and issues a `systemctl reboot`, > > when our max uptime have been exeeced. > > If restart of systemd-networkd happens while a reboot is in progress, > > the system will hang "forever" (and continue to pet the watchdog). > > The issue shows you using "systemctl start systemd-reboot". That is not > the right way to reboot. Use "systemctl reboot" instead. I suspect this > is related to why the reboot may stop partway: your command does not > start the reboot tasks in "irreversible" mode, which means that any > following contrary command, such as explicitly (re)starting a unit that > was going to be shut down, is going to implicitly cancel the > conflicting reboot action. > > You should also be using "try-restart" instead of "restart". If your > intent is to change configs, you want to say "make sure old configs are > not in use" rather than "enforce that the service is running now". (I > think making the "restart" command have "start" semantics was a design > mistake, and the "try-restart"/"restart" pair would have been better > named "restart"/"start-or-restart".) I can reproduce it with "systemctl reboot"... I'll try the "try-restart" option, and report back Thanks /Sean
[systemd-devel] Systemd hang when restarting a service during shutdown
Hi, Regarding, https://github.com/systemd/systemd/issues/21203 I think the point of the issue missed when the issue got closed. We have a service that is changing configs for systemd-networkd and issuing a `systemctl restart systemd-networkd`. An other service is checking uptime and issues a `systemctl reboot`, when our max uptime have been exeeced. If restart of systemd-networkd happens while a reboot is in progress, the system will hang "forever" (and continue to pet the watchdog). This is not a thing that eventually will timeout and reboot the system... /Sean
[systemd-devel] Are there any circumstances under which we would *expect* init.scope to not exist?
Hi there, I have (what I think will be) an easy question: Are there any circumstances or configuration behaviors under which we would expect the init.scope cgroup to not get created (still on v1, if applicable). I suspect the answer is no, but I wanted to verify. Context if you care: we've got a source based distribution (yocto project based. 3.0, specifically) running systemd 243 (not supported, I know) and I've just discovered that init.scope is not created on our systems. I assume that this is a bug and something that we've broken, but it seemed easy enough to ask if there are any circumstances where it might be expected before I get out my shovel and start digging. Thanks! -Sean McKay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] What's the best way to generate .conf files on startup?
I wondered… I saw the changed text in the first paragraph last night and wondered how I missed that.Thank you very much!-Sean From: Lennart PoetteringSent: Wednesday, February 24, 2021 9:33 AMTo: Sean McKayCc: systemd-devel@lists.freedesktop.orgSubject: Re: [systemd-devel] What's the best way to generate .conf files on startup? On Di, 23.02.21 13:49, Sean McKay (insanescient...@gmail.com) wrote: > Much obliged!> > I’ll see if I can carve out a little bit of time to put together a PR for> the documentation with the wording I would have expected, and we can see> what the maintainers think of my wording and go from there. Already done: https://github.com/systemd/systemd/commit/3acf00a5a4ff656e2799f7f3e2544891b09bbc35 Lennart --Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] What's the best way to generate .conf files on startup?
Much obliged!I’ll see if I can carve out a little bit of time to put together a PR for the documentation with the wording I would have expected, and we can see what the maintainers think of my wording and go from there.Thanks again!-Sean From: Lennart PoetteringSent: Tuesday, February 23, 2021 1:32 AMTo: Sean McKayCc: systemd-devel@lists.freedesktop.orgSubject: Re: [systemd-devel] What's the best way to generate .conf files on startup? On Mo, 22.02.21 23:22, Sean McKay (insanescient...@gmail.com) wrote: > Hi all,> > I've been looking to dynamically create .conf files at boot depending on> the hardware that I'm running on (to set MemoryMax, if it's relevant). I'd> assumed that the proper way to do this would be by using a generator, since> the .conf file won't automatically be loaded and would require triggering> the system manager to reload all configuration. And it didn't seem prudent> to call that *during* boot.> > Then I ran across this little snippet in the man page for> systemd.generator, which seems to imply the opposite:> Generators should only be used to generate unit files and symlinks to them,> not any other kind of configuration. Due to the lifecycle logic mentioned> above, generators are not a good fit to generate dynamic configuration for> other services. If you need to generate dynamic configuration for other> services, do so in normal services you order before the service in question.> > What I'm not clear on is whether this refers solely to configuration used> by the daemon itself (to use sshd as a well known example - eg:> /etc/ssh/sshd_config) or if it also refers to drop in .conf files (ie:> something in /run/systemd/system/ssh.service.d/)> Put differently, is a drop in considered to be a unit file or to be> configuration (for the purposes of the above helptext)? It's considered a unit file. I figure we should improve the docs onthat, to make this clearer. > Would the recommended solution in this case be for me to use a generator to> create the relevant .conf file(s) for MemoryMax? Or would it be better to> use a normal service (with proper ordering against the ones it's modifying)> to generate those .conf files and call daemon-reload during boot? If the> latter, are there any expected risks associated with calling daemon-reload> during boot? Doing this with a generator sounds perfect to me. Lennart --Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] What's the best way to generate .conf files on startup?
Hi all, I've been looking to dynamically create .conf files at boot depending on the hardware that I'm running on (to set MemoryMax, if it's relevant). I'd assumed that the proper way to do this would be by using a generator, since the .conf file won't automatically be loaded and would require triggering the system manager to reload all configuration. And it didn't seem prudent to call that *during* boot. Then I ran across this little snippet in the man page for systemd.generator, which seems to imply the opposite: Generators should only be used to generate unit files and symlinks to them, not any other kind of configuration. Due to the lifecycle logic mentioned above, generators are not a good fit to generate dynamic configuration for other services. If you need to generate dynamic configuration for other services, do so in normal services you order before the service in question. What I'm not clear on is whether this refers solely to configuration used by the daemon itself (to use sshd as a well known example - eg: /etc/ssh/sshd_config) or if it also refers to drop in .conf files (ie: something in /run/systemd/system/ssh.service.d/) Put differently, is a drop in considered to be a unit file or to be configuration (for the purposes of the above helptext)? Would the recommended solution in this case be for me to use a generator to create the relevant .conf file(s) for MemoryMax? Or would it be better to use a normal service (with proper ordering against the ones it's modifying) to generate those .conf files and call daemon-reload during boot? If the latter, are there any expected risks associated with calling daemon-reload during boot? Thanks! -- ~Sean McKay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to figure out what's causing systemd to start printing messages partway through boot?
Hi Lennart, I've had debug logging on for a while (it's mentioned in my first email, but I concede that was rather long). Unfortunately, I will admit to not being particularly familiar with dbus or the signals I should be looking for. Is there a direction you could point me in (either documentation, or even in the code itself) that would help me familiarize myself with them so I can follow what's happening to try to figure out what could be changing that value? The part that I really don't understand is that the only place that I can find where the value seems to be modified is in the manager_set_show_status function. That function has a debug log in it, but when I enable debug logging, it's nowhere to be found in the journal despite the value being toggled from auto to temporary. Cheers! -Sean -Original Message- From: Lennart Poettering Sent: Wednesday, May 6, 2020 8:09 AM To: McKay, Sean Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] How to figure out what's causing systemd to start printing messages partway through boot? On Mi, 29.04.20 23:26, McKay, Sean (sean.mc...@hpe.com) wrote: > Do you have any guidance on what I should look at to determine what is > causing the show_status variable in systemd to get flipped to true > without any apparent unit failures? Is there an easy way to find that > information? Maybe something asks systemd to? Consider enabling debug logging with "systemd.log_level=debug" on the kernel cmdline. It will then print information about incoming dbus msgs which might cause this, as well as signals it receives that might cause it. Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to figure out what's causing systemd to start printing messages partway through boot?
Dug a little deeper and found the "(Enabling | Disabling) showing of status." debug log, which shows up exactly once in my journalctl output... but only on one type of system. I don't see it on the other (still trying to figure out if I could be hitting some sort of rate limiting). Nothing around it seems like a failure, but I'm still digging. Any suggestions are still welcome Thanks! -Sean From: systemd-devel On Behalf Of McKay, Sean Sent: Wednesday, April 29, 2020 4:26 PM To: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] How to figure out what's causing systemd to start printing messages partway through boot? I apparently failed to finish a sentence. Fixed. My apologies! From: McKay, Sean Sent: Wednesday, April 29, 2020 4:15 PM To: systemd-devel@lists.freedesktop.org<mailto:systemd-devel@lists.freedesktop.org> Subject: How to figure out what's causing systemd to start printing messages partway through boot? Hi all, Hopefully quick question: Do you have any guidance on what I should look at to determine what is causing the show_status variable in systemd to get flipped to true without any apparent unit failures? Is there an easy way to find that information? We have a few systems that start printing the manager status messages halfway through bootup (that aren't supposed to). They have 'quiet' set on the kernel command line and they appear to be following that for a good chunk of the boot, but partway through I start getting '[ OK ] Started...' messages without any failure showing up first (which is what I expect and have seen in other scenarios). Running 'systemctl show' after the system has booted, shows that the variable has indeed been flipped. From looking through the code, it appears the only way this is possible should be for a unit to have hit some status other than "OK" (JOB_DONE internally, it would appear) but I don't see any of those in the output. And I'm not sure if there's an equivalent in the journal to search for. I've poked through the journal near where things start showing up and there doesn't appear to be anything in particular that sticks out... so I turned on debug logging and there's a ton of it. And I haven't yet had any luck figuring out what I'm looking for from digging through the source code. So I figured I'd ask. Thanks! -Sean McKay P.S. We're still waaay back on 229. Frantically trying to get things upgraded, but not there yet. I don't think that's relevant (I doubt this is a systemd bug), but it might come up if there's more tooling/verbosity in newer versions. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] How to figure out what's causing systemd to start printing messages partway through boot?
Hi all, Hopefully quick question: Do you have any guidance on what I should look at to determine what is causing the show_status variable in systemd to get flipped to true without any apparent unit failures? Is there an easy way to find that information? We have a few systems that start printing the manager status messages halfway through bootup (that aren't supposed to). They have 'quiet' set on the kernel command line and they appear to be following that for a good chunk of the boot, but partway through I start getting '[ OK ] Started...' messages without any failure showing up first (which is what I expect and have seen in other scenarios). Running 'systemctl show' after the system has booted, shows that the variable has indeed been flipped. From looking through the code, it appears the only way this is possible should be for a unit to have I've poked through the journal near where things start showing up and there doesn't appear to be anything in particular that sticks out... so I turned on debug logging and there's a ton of it. And I haven't yet had any luck figuring out what I'm looking for from digging through the source code. So I figured I'd ask. Thanks! -Sean McKay P.S. We're still waaay back on 229. Frantically trying to get things upgraded, but not there yet. I don't think that's relevant (I doubt this is a systemd bug), but it might come up if there's more tooling/verbosity in newer versions. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to figure out what's causing systemd to start printing messages partway through boot?
I apparently failed to finish a sentence. Fixed. My apologies! From: McKay, Sean Sent: Wednesday, April 29, 2020 4:15 PM To: systemd-devel@lists.freedesktop.org Subject: How to figure out what's causing systemd to start printing messages partway through boot? Hi all, Hopefully quick question: Do you have any guidance on what I should look at to determine what is causing the show_status variable in systemd to get flipped to true without any apparent unit failures? Is there an easy way to find that information? We have a few systems that start printing the manager status messages halfway through bootup (that aren't supposed to). They have 'quiet' set on the kernel command line and they appear to be following that for a good chunk of the boot, but partway through I start getting '[ OK ] Started...' messages without any failure showing up first (which is what I expect and have seen in other scenarios). Running 'systemctl show' after the system has booted, shows that the variable has indeed been flipped. From looking through the code, it appears the only way this is possible should be for a unit to have hit some status other than "OK" (JOB_DONE internally, it would appear) but I don't see any of those in the output. And I'm not sure if there's an equivalent in the journal to search for. I've poked through the journal near where things start showing up and there doesn't appear to be anything in particular that sticks out... so I turned on debug logging and there's a ton of it. And I haven't yet had any luck figuring out what I'm looking for from digging through the source code. So I figured I'd ask. Thanks! -Sean McKay P.S. We're still waaay back on 229. Frantically trying to get things upgraded, but not there yet. I don't think that's relevant (I doubt this is a systemd bug), but it might come up if there's more tooling/verbosity in newer versions. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Is it possible to set a default for MemoryMax?
Thanks for the pointer, Lennart! I've done some initial review of the commit you pointed me to, and it seems like it should be pretty straightforward to use that understanding to implement the other functions. Might take a bit depending on my local management's perspective on the priority of getting this done, but I'll put together a PR as soon as I have a chance. In the meantime, can anyone tell me if there's any semblance of a 'getting started' page for systemd development? The main code changes should be easy enough, but I see that there are a number of tests included with the particular commit I was pointed to in addition to the code change itself. I'd like to make sure that I know what sort of tests are expected in a commit and how to run them (and any other things I might need to do to ensure things are in order before submitting the PR). Thanks! -Sean McKay -Original Message- From: Lennart Poettering Sent: Monday, June 17, 2019 7:55 AM To: McKay, Sean Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Is it possible to set a default for MemoryMax? On Do, 13.06.19 22:06, McKay, Sean (sean.mc...@hpe.com) wrote: > Hi there, > > First off, forgive me if this is documented or discussed somewhere > already. I couldn't find any reference to it in the man pages or in > the mailing list archives, and I'm relatively new to this community so > I could easily be missing something. > > Context that can be skipped if you only care about the technical part > of the question: We're building a customized linux environment using > systemd as the init system, and we've found that some of the daemons > that we're including in said environment aren't particularly good > citizens in regards to their resource consumption (admittedly most of > those are the ones that have been written in house). As a result, > while we try to track down and fix those bugs, we would like to limit > the memory that processes are allowed to consume by default (with > actual values in the .service files overriding that default). While we > understand that this is the sort of thing that's probably best set > individually per-daemon based on an understanding of how the daemon > should behave, it would at least give us a starting point and would > mean that a leaky process would be the one that crashed (when it hit > the memory limit), rather than taking down the whole system or the OOM > killer selecting a different process that's working correctly but > still happens to be using more memory at the time. > > We've been trying to encourage our developers to use memory accounting > and MemoryMax, since we're given to understand that's applied to the > entire cgroup's memory usage for a given daemon's slice. The > documentation, however, doesn't seem to indicate that a default value > for any of the memory variables can be set in the system.conf file, > and some quick experimental testing seems to indicate that's truly the > case. There does seem to be support for setting the memory resource > limit (limitas), but my understanding is that wouldn't apply to > subprocesses that are spawned off by the parent daemon - they'd > inherit the resource limit, but get their own instance of it (and I'm > actually still trying to figure out if it would apply to threads, > given the way I'm given to understand Linux treats threads. For > memory, it would probably still cap things, since they're all in the > same address space, but I'm unclear if that would be true of the other > limits that can be applied). > > Am I missing something and this is possible to set? Or is it something > that's come up and been rejected for some reason? Or has it come up > and not been important relative to other development priorities? Or > has it simply never come up because we're the first ones to want to do > something so foolish? > > Any insight that you could provide would be greatly appreciated! Thanks. See https://github.com/systemd/systemd/pull/12211 It adds this for MemoryLow=. I see no reason why we shouldn't add this for MemoryMax= too. Please consider pinging @cdown about this, and maybe prep a PR that adds this for you. It should be simple now, given that the MemoryLow= case is very very similar. Or maybe MemoryLow= is actually the knob you want to use and thus what you are asking for already exists in current git? Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Is it possible to set a default for MemoryMax?
Hi there, First off, forgive me if this is documented or discussed somewhere already. I couldn't find any reference to it in the man pages or in the mailing list archives, and I'm relatively new to this community so I could easily be missing something. Context that can be skipped if you only care about the technical part of the question: We're building a customized linux environment using systemd as the init system, and we've found that some of the daemons that we're including in said environment aren't particularly good citizens in regards to their resource consumption (admittedly most of those are the ones that have been written in house). As a result, while we try to track down and fix those bugs, we would like to limit the memory that processes are allowed to consume by default (with actual values in the .service files overriding that default). While we understand that this is the sort of thing that's probably best set individually per-daemon based on an understanding of how the daemon should behave, it would at least give us a starting point and would mean that a leaky process would be the one that crashed (when it hit the memory limit), rather than taking down the whole system or the OOM killer selecting a different process that's working correctly but still happens to be using more memory at the time. We've been trying to encourage our developers to use memory accounting and MemoryMax, since we're given to understand that's applied to the entire cgroup's memory usage for a given daemon's slice. The documentation, however, doesn't seem to indicate that a default value for any of the memory variables can be set in the system.conf file, and some quick experimental testing seems to indicate that's truly the case. There does seem to be support for setting the memory resource limit (limitas), but my understanding is that wouldn't apply to subprocesses that are spawned off by the parent daemon - they'd inherit the resource limit, but get their own instance of it (and I'm actually still trying to figure out if it would apply to threads, given the way I'm given to understand Linux treats threads. For memory, it would probably still cap things, since they're all in the same address space, but I'm unclear if that would be true of the other limits that can be applied). Am I missing something and this is possible to set? Or is it something that's come up and been rejected for some reason? Or has it come up and not been important relative to other development priorities? Or has it simply never come up because we're the first ones to want to do something so foolish? Any insight that you could provide would be greatly appreciated! Thanks. -Sean McKay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] negative trust anchors not working with non TLD domain names
On 04/20/2017 07:05 AM, Lennart Poettering wrote: > On Wed, 19.04.17 07:12, Sean Dague (s...@dague.net) wrote: > >> I just upgraded to Ubuntu 17.04 (systemd 232) where systemd-resolved is >> turned on by default, which means DNSSEC validation on by default. > > The DNSSEC code got substantially updated in 233. Any chance you can > retest with something more current? > > Lennart I rebuilt systemd 233 out of debian experimental on a VM, and after installing that in a fresh 17.04 environment, the local lookup case seems to be working fine. I'll go report that to the distro. Are there specific patches that they should be looking at here to fix this behavior, or is it extensive enough that the answer is just that it's going to need a full version bump? -Sean -- Sean Dague http://dague.net ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] negative trust anchors not working with non TLD domain names
I just upgraded to Ubuntu 17.04 (systemd 232) where systemd-resolved is turned on by default, which means DNSSEC validation on by default. My home network has DNS provided by dnsmasq, and for historical reasons I set the domain name on all hosts there to 'dague.pvt'. I tried adding both 'dague.pvt' and 'pvt' to /etc/dnssec-trust-anchors.d/dague.pvt.negative (as well as copying in the list of all the negative trust anchors that exist by default, home, local, the reverse lookup ones). Looking up os3.dague.pvt always returns a SERVEFAIL, it does not seem to be respecting the negative trust anchor, even though the logs seem to be picking it up: Apr 19 07:06:10 ribos.dague.pvt systemd-resolved[16286]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan local private pvt dague.pvt test Apr 19 07:06:25 ribos.dague.pvt systemd-resolved[16286]: Switching to DNS server 10.42.0.3 for interface enp0s25. Apr 19 07:06:25 ribos.dague.pvt systemd-resolved[16286]: DNSSEC validation failed for question os3.dague.pvt IN DS: no-signature Apr 19 07:06:25 ribos.dague.pvt systemd-resolved[16286]: DNSSEC validation failed for question os3.dague.pvt IN SOA: no-signature Apr 19 07:06:25 ribos.dague.pvt systemd-resolved[16286]: DNSSEC validation failed for question os3.dague.pvt IN A: no-signature It did occur to me that there are no non TLD examples in the excluded list except the reverse lookup domains (which I assume are treated specially). Is there something I'm missing with configuration here? Or are non TLD domains not supported for negative trust anchors? And if so, is that a bug or intentional? My current work around is to just turn off DNSSEC, which I'd really like to avoid doing if I could. Thanks in advance, -Sean -- Sean Dague http://dague.net ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] dmsetup or cryptsetup hangs in the initrd
In RedHat 7 I am trying to setup a partition with dmsetup. I have attached a tarball that includes three files: * go.sh is a simple script to install everything * install is the dracut module install * 95dmcrypt.sh is the file I want to run at boot time The module is 99securevm. Obviously, this has all been stripped down to make it as simple as possible. Sorry for the hard coding. If you want to test it, you need to provide a /dev/sdb1 partition that is at least 2G. If you comment out the `exit 0' in go.sh it will setup the partition for you. Note that if 95dmcrypt.sh has either the dmsetup or the 2 cryptsetup lines commented in, it will hang the machine. I am hoping I have just put something in the wrong place. pre-mount is where we previously ran the script from. Cheers, Sean dmtest.tar.gz Description: GNU Zip compressed data ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to debug this strange issue about systemd?
On Fri, 2015-07-17 at 10:43 +0100, Colin Guthrie wrote: See man dracut and the Debugging dracut section. Also see man dracut.cmdline and the rd.break bits. Was it not man dracut.kernel? dracut.kernel == dracut.cmdline here (the latter seems to be the preferred name as it's the title in the manpage) and indeed: lrwxrwxrwx 1 root root 19 Jun 5 17:11 /usr/share/man/man7/dracut.kernel.7.xz - dracut.cmdline.7.xz Thanks very much for all help. The root cause is that 64 bit libraries were not included in the real ext4 file system. But now a new issue appears, Boot failed at [ ok ] Reached target System Initialization. How to debug it? The attachment is the full log message. [FAILED] Failed to start Postfix Mail Transport Agent. See systemctl status postfix.service for details. Starting Command Scheduler... [ OK ] Started Command Scheduler. Starting smfpd.service... [ OK ] Started smfpd.service. Starting /etc/init.d/boot.local Compatibility... [ OK ] Started /etc/init.d/boot.local Compatibility. Starting Wait for Plymouth Boot Screen to Quit... Starting Terminate Plymouth Boot Screen... -- Sean. XinRong Fu Dedicate System Engineer SUSE x...@suse.com (P)+86 18566229618 line SUSE [0.00] Initializing cgroup subsys cpuset [0.00] Initializing cgroup subsys cpu [0.00] Initializing cgroup subsys cpuacct [0.00] Linux version 4.1.0-rc2-7-desktop+ (s...@linux-dunz.site) (gcc version 4.8.3 20140627 [gcc-4_8-branch revision 212064] (SUSE Linux) ) #7 SMP PREEMPT Sat Jul 4 23:39:33 CST 2015 [0.00] Command line: root=UUID=20059b62-2542-4a85-80cf-41da6e0c1137 rootflags=rw rootfstype=ext4 console=ttyS0,115200n8 console=tty0 systemd.unit=default.target systemd.confirm_spawn=auto systemd.show_status=ture systemd.log_level=debug [0.00] e820: BIOS-provided physical RAM map: [0.00] BIOS-e820: [mem 0x-0x0009fbff] usable [0.00] BIOS-e820: [mem 0x0009fc00-0x0009] reserved [0.00] BIOS-e820: [mem 0x000f-0x000f] reserved [0.00] BIOS-e820: [mem 0x0010-0x3ffd] usable [0.00] BIOS-e820: [mem 0x3ffe-0x3fff] reserved [0.00] BIOS-e820: [mem 0xfeffc000-0xfeff] reserved [0.00] BIOS-e820: [mem 0xfffc-0x] reserved [0.00] earlycon: no match for ttyS0,115200n8 [0.00] earlycon: no match for tty0 [0.00] NX (Execute Disable) protection: active [0.00] SMBIOS 2.8 present. [0.00] Hypervisor detected: KVM [0.00] AGP: No AGP bridge found [0.00] e820: last_pfn = 0x3ffe0 max_arch_pfn = 0x4 [0.00] PAT not supported by CPU. [0.00] found SMP MP-table at [mem 0x000f0ec0-0x000f0ecf] mapped at [880f0ec0] [0.00] Scanning 1 areas for low memory corruption [0.00] init_memory_mapping: [mem 0x-0x000f] [0.00] init_memory_mapping: [mem 0x3d80-0x3d9f] [0.00] init_memory_mapping: [mem 0x2000-0x3d7f] [0.00] init_memory_mapping: [mem 0x0010-0x1fff] [0.00] init_memory_mapping: [mem 0x3da0-0x3ffd] [0.00] RAMDISK: [mem 0x3da25000-0x3ffe] [0.00] Allocated new RAMDISK: [mem 0x3b45a000-0x3da2417f] [0.00] Move RAMDISK from [mem 0x3da25000-0x3ffef17f] to [mem 0x3b45a000-0x3da2417f] [0.00] ACPI: Early table checksum verification disabled [0.00] ACPI: RSDP 0x000F0CD0 14 (v00 BOCHS ) [0.00] ACPI: ?ɳ� 0x3FFE1854 8A476AE6 (v251 �??� ? ??��^�? 318ACCAB e8�? 6D99E0DB) [0.00] [ cut here ] [0.00] WARNING: CPU: 0 PID: 0 at mm/early_ioremap.c:136 __early_ioremap.constprop.0+0x11f/0x1f2() [0.00] Modules linked in: [0.00] CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.0-rc2-7-desktop+ #7 [0.00] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20141015_183547-build31 04/01/2014 [0.00] 81a3ad65 81c03ca8 8169d79b 0198 [0.00] 81c03ce8 81066d8a 30783020cfb3c93f [0.00] 0008a478 8163 ca45833a [0.00] Call Trace: [0.00] [8169d79b] dump_stack+0x4c/0x6e [0.00] [81066d8a] warn_slowpath_common+0x8a/0xc0 [0.00] [81066e7a] warn_slowpath_null+0x1a/0x20 [0.00] [81d5fed8] __early_ioremap.constprop.0+0x11f/0x1f2 [0.00] [81d6016b] early_ioremap+0x13/0x15 [0.00] [81d3ca60] __acpi_map_table+0x13/0x18 [0.00] [81692a37] acpi_os_map_iomem+0x27/0x151 [0.00] [81692b6f] acpi_os_map_memory+0xe/0x10 [0.00] [81d75af5
Re: [systemd-devel] How to debug this strange issue about systemd?
On Fri, 2015-07-17 at 06:55 +0300, Andrei Borzenkov wrote: В Wed, 15 Jul 2015 23:03:02 +0800 sean x...@suse.com пишет: Hi All: I am trying to test the latest upstream kernel, But i encounter a strange issue about systemd. When the systemd extracted from initrd image mounts the real root file system hda.img on /sysroot and changes root to the new directory, it can not found /sbin/init and /bin/sh. In fact, These two files exist in the hda.img. How to debug this issue? Why does not it enter emergency mode? If enter emergency mode, maybe this issue become easy. You can stop in dracut just before switch root step and examine environment. At this point root should already be mounted. Thanks very much! How to stop the booting process before switch root? qemu command line: qemu-kvm -D /tmp/qemu-kvm-machine.log -m 1024M -append root=UUID=20059b62-2542-4a85-80cf-41da6e0c1137 rootflags=rw rootfstype=ext4 debug debug_objects console=ttyS0,115200n8 console=tty0 rd.debug rd.shell=1 log_buf_len=1M systemd.unit=emergency.target systemd.log_level=debug systemd.log_target=console -kernel ./qemu_platform/bzImage -hda ./qemu_platform/hda.img -initrd ./qemu_platform/initrd-4.1.0-rc2-7-desktop+ -device e1000,netdev=network0 -netdev user,id=network0 -serial file:/home/sean/work/source/upstream/kernel.org/ttys0.txt ... sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform sudo mount -o loop ./hda.img ./hda sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform ls -l ./hda/sbin/init lrwxrwxrwx 1 sean users 26 Jul 14 22:49 ./hda/sbin/init - ../usr/lib/systemd/systemd Do you have separate /usr? No, Only one disk partition. sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform ls -l ./hda/bin/sh lrwxrwxrwx 1 sean users 4 Oct 26 2014 ./hda/bin/sh - bash sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform lsinitrd ./initrd-4.1.0-rc2-7-desktop+ |grep sbin\/init -rwxr-xr-x 1 root root 1223 Nov 27 2014 sbin/initqueue lrwxrwxrwx 1 root root 26 Jul 14 21:00 sbin/init - ../usr/lib/systemd/systemd sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform lsinitrd ./initrd-4.1.0-rc2-7-desktop+ |grep bin\/sh lrwxrwxrwx 1 root root4 Jul 14 21:00 bin/sh - bash -- Sean. XinRong Fu Dedicate System Engineer SUSE x...@suse.com (P)+86 18566229618 line SUSE ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] How to debug this strange issue about systemd?
Hi All: I am trying to test the latest upstream kernel, But i encounter a strange issue about systemd. When the systemd extracted from initrd image mounts the real root file system hda.img on /sysroot and changes root to the new directory, it can not found /sbin/init and /bin/sh. In fact, These two files exist in the hda.img. How to debug this issue? Why does not it enter emergency mode? If enter emergency mode, maybe this issue become easy. qemu command line: qemu-kvm -D /tmp/qemu-kvm-machine.log -m 1024M -append root=UUID=20059b62-2542-4a85-80cf-41da6e0c1137 rootflags=rw rootfstype=ext4 debug debug_objects console=ttyS0,115200n8 console=tty0 rd.debug rd.shell=1 log_buf_len=1M systemd.unit=emergency.target systemd.log_level=debug systemd.log_target=console -kernel ./qemu_platform/bzImage -hda ./qemu_platform/hda.img -initrd ./qemu_platform/initrd-4.1.0-rc2-7-desktop+ -device e1000,netdev=network0 -netdev user,id=network0 -serial file:/home/sean/work/source/upstream/kernel.org/ttys0.txt Job initrd-switch-root.target/start finished, result=done Accepted new private connection. Got message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/agent interface=org.freedesktop.systemd1.Agent member=Released cookie=1 reply_cookie=0 error=n/a Got disconnect on private connection. Received SIGCHLD from PID 520 (plymouth). Child 520 (plymouth) died (code=exited, status=0/SUCCESS) Child 520 belongs to plymouth-switch-root.service plymouth-switch-root.service: main process exited, code=exited, status=0/SUCCESS plymouth-switch-root.service changed start - dead Job plymouth-switch-root.service/start finished, result=done plymouth-switch-root.service: cgroup is empty ConditionPathExists=/etc/initrd-release succeeded for initrd-switch-root.service. About to execute: /usr/bin/systemctl --no-block --force switch-root /sysroot Forked /usr/bin/systemctl as 527 initrd-switch-root.service changed dead - start Accepted new private connection. Got message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=SwitchRoot cookie=1 reply_cookie=0 error=n/a Sent message type=method_return sender=n/a destination=n/a object=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 error=n/a Serializing state to /run/systemd Switching root. Closing left-over fd 21 Closing left-over fd 22 Closing left-over fd 23 Closing left-over fd 26 Closing left-over fd 27 No /sbin/init, trying fallback Failed to execute /bin/sh, giving up: No such file or directory sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform sudo mount -o loop ./hda.img ./hda sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform ls -l ./hda/sbin/init lrwxrwxrwx 1 sean users 26 Jul 14 22:49 ./hda/sbin/init - ../usr/lib/systemd/systemd sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform ls -l ./hda/bin/sh lrwxrwxrwx 1 sean users 4 Oct 26 2014 ./hda/bin/sh - bash sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform lsinitrd ./initrd-4.1.0-rc2-7-desktop+ |grep sbin\/init -rwxr-xr-x 1 root root 1223 Nov 27 2014 sbin/initqueue lrwxrwxrwx 1 root root 26 Jul 14 21:00 sbin/init - ../usr/lib/systemd/systemd sean@linux-dunz:~/work/source/upstream/kernel.org/qemu_platform lsinitrd ./initrd-4.1.0-rc2-7-desktop+ |grep bin\/sh lrwxrwxrwx 1 root root4 Jul 14 21:00 bin/sh - bash -- Sean. XinRong Fu Dedicate System Engineer SUSE x...@suse.com (P)+86 18566229618 line SUSE -- Sean. XinRong Fu Dedicate System Engineer SUSE x...@suse.com (P)+86 18566229618 line SUSE ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [PATCH] timesyncd: do not keep listening socket open forever
This also makes the source port less predicatable.
---
src/timesync/timesyncd-manager.c | 20 +---
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/src/timesync/timesyncd-manager.c b/src/timesync/timesyncd-manager.c
index 3ae01eb..03cfb24 100644
--- a/src/timesync/timesyncd-manager.c
+++ b/src/timesync/timesyncd-manager.c
@@ -132,6 +132,7 @@ struct ntp_msg {
static int manager_arm_timer(Manager *m, usec_t next);
static int manager_clock_watch_setup(Manager *m);
+static int manager_listen_setup(Manager *m);
static double ntp_ts_short_to_d(const struct ntp_ts_short *ts) {
return be16toh(ts-sec) + (be16toh(ts-frac) / 65536.0);
@@ -184,6 +185,14 @@ static int manager_send_request(Manager *m) {
m-event_timeout = sd_event_source_unref(m-event_timeout);
+if (m-server_socket 0) {
+r = manager_listen_setup(m);
+if (r 0) {
+log_warning(Failed to setup connection socket: %s,
strerror(-r));
+return r;
+}
+}
+
/*
* Set transmit timestamp, remember it; the server will send that back
* as the origin timestamp and we have an indication that this is the
@@ -250,7 +259,6 @@ static int manager_arm_timer(Manager *m, usec_t next) {
int r;
assert(m);
-assert(m-event_receive);
if (next == 0) {
m-event_timer = sd_event_source_unref(m-event_timer);
@@ -610,6 +618,10 @@ static int manager_receive_response(sd_event_source
*source, int fd, uint32_t re
m-pending = false;
m-retry_interval = 0;
+/* Stop listening */
+m-event_receive = sd_event_source_unref(m-event_receive);
+m-server_socket = safe_close(m-server_socket);
+
/* announce leap seconds */
if (NTP_FIELD_LEAP(ntpmsg.field) NTP_LEAP_PLUSSEC)
leap_sec = 1;
@@ -741,12 +753,6 @@ static int manager_begin(Manager *m) {
log_info(Using NTP server %s (%s)., strna(pretty),
m-current_server_name-string);
sd_notifyf(false, STATUS=Using Time Server %s (%s)., strna(pretty),
m-current_server_name-string);
-r = manager_listen_setup(m);
-if (r 0) {
-log_warning(Failed to setup connection socket: %s,
strerror(-r));
-return r;
-}
-
r = manager_clock_watch_setup(m);
if (r 0)
return r;
--
2.1.3
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [PATCH] udev: handle network controllers in nonstandard domains
Onboard network controllers are not always on PCI domain 0.
---
src/udev/udev-builtin-net_id.c | 22 +-
1 file changed, 17 insertions(+), 5 deletions(-)
diff --git a/src/udev/udev-builtin-net_id.c b/src/udev/udev-builtin-net_id.c
index 5719021..c8d3ad3 100644
--- a/src/udev/udev-builtin-net_id.c
+++ b/src/udev/udev-builtin-net_id.c
@@ -35,13 +35,15 @@
* oindex -- on-board device index number
* sslot[ffunction][ddev_id] -- hotplug slot index number
* xMAC-- MAC address
- * pbussslot[ffunction][ddev_id] -- PCI geographical location
+ * [Pdomain]pbussslot[ffunction][ddev_id] -- PCI geographical
location
* pbussslot[ffunction][uport][..][cconfig][iinterface]
* -- USB port number chain
*
* All multi-function PCI devices will carry the [ffunction] number in the
* device name, including the function 0 device.
*
+ * When using PCI geography, The PCI domain is only prepended when it is not 0.
+ *
* For USB devices the full chain of port numbers of hubs is composed. If the
* name gets longer than the maximum number of 15 characters, the name is not
* exported.
@@ -163,6 +165,7 @@ out:
static int dev_pci_slot(struct udev_device *dev, struct netnames *names) {
struct udev *udev = udev_device_get_udev(names-pcidev);
+unsigned long domain;
unsigned int bus;
unsigned int slot;
unsigned int func;
@@ -171,6 +174,7 @@ static int dev_pci_slot(struct udev_device *dev, struct
netnames *names) {
char *s;
const char *attr;
struct udev_device *pci = NULL;
+char *sysname;
char slots[256];
DIR *dir;
struct dirent *dent;
@@ -178,8 +182,10 @@ static int dev_pci_slot(struct udev_device *dev, struct
netnames *names) {
int hotplug_slot = 0;
int err = 0;
-if (sscanf(udev_device_get_sysname(names-pcidev), :%x:%x.%d,
bus, slot, func) != 3)
-return -ENOENT;
+sysname = udev_device_get_sysname(names-pcidev);
+
+if(sscanf(sysname, %lx:%x:%x.%d, domain, bus, slot, func) != 4)
+return -ENOENT;
/* kernel provided multi-device index */
attr = udev_device_get_sysattr_value(dev, dev_id);
@@ -188,7 +194,10 @@ static int dev_pci_slot(struct udev_device *dev, struct
netnames *names) {
/* compose a name based on the raw kernel's PCI bus, slot numbers */
s = names-pci_path;
-l = strpcpyf(s, sizeof(names-pci_path), p%ds%d, bus, slot);
+l = sizeof(names-pci_path);
+if (domain 0)
+l = strpcpyf(s, l, P%ld, domain);
+l = strpcpyf(s, l, p%ds%d, bus, slot);
if (func 0 || is_pci_multifunction(names-pcidev))
l = strpcpyf(s, l, f%d, func);
if (dev_id 0)
@@ -236,7 +245,10 @@ static int dev_pci_slot(struct udev_device *dev, struct
netnames *names) {
if (hotplug_slot 0) {
s = names-pci_slot;
-l = strpcpyf(s, sizeof(names-pci_slot), s%d, hotplug_slot);
+l = sizeof(names-pci_slot);
+if (domain 0)
+l = strpcpyf(s, l, P%ld, domain);
+l = strpcpyf(s, l, s%d, hotplug_slot);
if (func 0 || is_pci_multifunction(names-pcidev))
l = strpcpyf(s, l, f%d, func);
if (dev_id 0)
--
1.8.2.1
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] (no subject)
Here is my new attempt for interface naming for cards in non-zero domains. Oddly enough, I still get an f0 at the end of mine even though it is not PCI multifunction. sean@hanyuu ~ $ udevadm test-builtin net_id /sys/class/net/enP2p32s15f0 2/dev/null | grep PATH ID_NET_NAME_PATH=enP2p32s15 sean@hanyuu ~ $ udevadm info /sys/class/net/enP2p32s15f0 2/dev/null | grep NAME_PATH E: ID_NET_NAME_PATH=enP2p32s15f0 -- Sean McG. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] udev: make net_id more robust
On Monday, June 10, 2013, Kay Sievers k...@vrfy.org wrote: On Fri, Jun 7, 2013 at 11:46 PM, Sean McGovern gsean...@gmail.com wrote: This is definitely not a common case as almost all of the other Linux machines I have access to expose a network controller in domain 0. Yeah, I've only seen domains used on huge SGI machines. 256 PCI buses are quite a lot for a normal machine. :) At least in my case each of the domains has a bus ID unique to amongst all of the domains, but I'm reasonably sure this isn't mandated by the PCI specifications. They don't share the numbers, yeah. 0002:20:0f.0 Ethernet controller: Apple Inc. UniNorth 2 GMAC (Sun GEM) (rev 80) We could add a Pdomain or mdomain in front: enP2p32s15 enm2p32s15 Or something else, we just have to pick one. :) And we would suppress that for all the common domain == 0x cases. Kay OK. Would you like me to cook up a better patch with this suggestion? -- Sean McG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [PATCH] udev: make net_id more robust
Onboard network controllers are not always on PCI domain 0.
---
src/udev/udev-builtin-net_id.c | 14 --
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/src/udev/udev-builtin-net_id.c b/src/udev/udev-builtin-net_id.c
index 5719021..3e77f30 100644
--- a/src/udev/udev-builtin-net_id.c
+++ b/src/udev/udev-builtin-net_id.c
@@ -163,6 +163,7 @@ out:
static int dev_pci_slot(struct udev_device *dev, struct netnames *names) {
struct udev *udev = udev_device_get_udev(names-pcidev);
+unsigned long domain __attribute__((__unused__));
unsigned int bus;
unsigned int slot;
unsigned int func;
@@ -171,6 +172,7 @@ static int dev_pci_slot(struct udev_device *dev, struct
netnames *names) {
char *s;
const char *attr;
struct udev_device *pci = NULL;
+char *sysname;
char slots[256];
DIR *dir;
struct dirent *dent;
@@ -178,8 +180,16 @@ static int dev_pci_slot(struct udev_device *dev, struct
netnames *names) {
int hotplug_slot = 0;
int err = 0;
-if (sscanf(udev_device_get_sysname(names-pcidev), :%x:%x.%d,
bus, slot, func) != 3)
-return -ENOENT;
+sysname = udev_device_get_sysname(names-pcidev);
+
+if(!sysname)
+return -ENOENT;
+
+if(!strlen(sysname))
+return -ENOENT;
+
+if(sscanf(sysname, %lx:%x:%x.%d, domain, bus, slot, func) != 4)
+return -ENOENT;
/* kernel provided multi-device index */
attr = udev_device_get_sysattr_value(dev, dev_id);
--
1.8.2.1
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] udev: make net_id more robust
Ignore this patch then -- I can't change the PCI geography of my older G4 Mac Mini, and without this patch predictable interface naming does not work for it. I'll just keep it locally. -- Sean McGovern --Original Message-- From: Kay Sievers To: McGovern, Sean Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] [PATCH] udev: make net_id more robust Sent: Jun 7, 2013 14:25 On Fri, Jun 7, 2013 at 3:27 PM, Sean McGovern gsean...@gmail.com wrote: Onboard network controllers are not always on PCI domain 0. +sysname = udev_device_get_sysname(names-pcidev); + +if(!sysname) +return -ENOENT; + +if(!strlen(sysname)) +return -ENOENT; None of these checks should be needed. +if(sscanf(sysname, %lx:%x:%x.%d, domain, bus, slot, func) != 4) +return -ENOENT; We only support domain 0, and ignore all other devices so far. With the above change we would just ignore the domain and allow possibly conflicting identical names below different domains? I don't understand what you intend here, please be more specific than more robust in patch descriptions. Thanks, Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] udev: make net_id more robust
This is definitely not a common case as almost all of the other Linux machines I have access to expose a network controller in domain 0. At least in my case each of the domains has a bus ID unique to amongst all of the domains, but I'm reasonably sure this isn't mandated by the PCI specifications. Regardless, here is the information you requested: sean@hanyuu ~ $ uname -a Linux hanyuu 3.8.13-gentoo #7 Thu Jun 6 00:45:03 EDT 2013 ppc 7447A, altivec supported PowerMac10,1 GNU/Linux sean@hanyuu ~ $ /usr/sbin/lspci :00:0b.0 Host bridge: Apple Inc. UniNorth 2 AGP :00:10.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] RV280 [Radeon 9200] (rev 01) 0001:10:0b.0 Host bridge: Apple Inc. UniNorth 2 PCI 0001:10:17.0 Unassigned class [ff00]: Apple Inc. KeyLargo/Intrepid Mac I/O 0001:10:18.0 USB controller: Apple Inc. KeyLargo/Intrepid USB 0001:10:19.0 USB controller: Apple Inc. KeyLargo/Intrepid USB 0001:10:1a.0 USB controller: Apple Inc. KeyLargo/Intrepid USB 0001:10:1b.0 USB controller: NEC Corporation OHCI USB Controller (rev 43) 0001:10:1b.1 USB controller: NEC Corporation OHCI USB Controller (rev 43) 0001:10:1b.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 04) 0002:20:0b.0 Host bridge: Apple Inc. UniNorth 2 Internal PCI 0002:20:0d.0 Unassigned class [ff00]: Apple Inc. UniNorth/Intrepid ATA/100 0002:20:0e.0 FireWire (IEEE 1394): Apple Inc. UniNorth 2 FireWire (rev 81) 0002:20:0f.0 Ethernet controller: Apple Inc. UniNorth 2 GMAC (Sun GEM) (rev 80) sean@hanyuu ~ $ ls -l /sys/bus/pci/devices/ total 0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 :00:0b.0 - ../../../devices/pci:00/:00:0b.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 :00:10.0 - ../../../devices/pci:00/:00:10.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:0b.0 - ../../../devices/pci0001:10/0001:10:0b.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:17.0 - ../../../devices/pci0001:10/0001:10:17.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:18.0 - ../../../devices/pci0001:10/0001:10:18.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:19.0 - ../../../devices/pci0001:10/0001:10:19.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:1a.0 - ../../../devices/pci0001:10/0001:10:1a.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:1b.0 - ../../../devices/pci0001:10/0001:10:1b.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:1b.1 - ../../../devices/pci0001:10/0001:10:1b.1 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0001:10:1b.2 - ../../../devices/pci0001:10/0001:10:1b.2 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0002:20:0b.0 - ../../../devices/pci0002:20/0002:20:0b.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0002:20:0d.0 - ../../../devices/pci0002:20/0002:20:0d.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0002:20:0e.0 - ../../../devices/pci0002:20/0002:20:0e.0 lrwxrwxrwx 1 root root 0 Jun 6 19:48 0002:20:0f.0 - ../../../devices/pci0002:20/0002:20:0f.0 -- Sean McG. --Original Message-- From: Kay Sievers To: gsean...@gmail.com Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] [PATCH] udev: make net_id more robust Sent: Jun 7, 2013 17:20 On Fri, Jun 7, 2013 at 10:58 PM, Sean McGovern gsean...@gmail.com wrote: Ignore this patch then -- I can't change the PCI geography of my older G4 Mac Mini, and without this patch predictable interface naming does not work for it. I'll just keep it locally. Maybe we can add support for it, we would need some prefix though, I guess, not just strip off the domain. How does: ls -l /sys/bus/pci/devices/ and lspci look like on that box? Thanks, Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
