Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Tim Meusel commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Hi. Can someone confirm this is actually fixed in 6.16.0? I wasn't able to find an entry in the release notes and my agents still fail: root@server01 ~ # puppet --version 6.16.0 root@server01 ~ # systemctl status puppet ● puppet.service - Puppet agent Loaded: loaded (/usr/lib/systemd/system/puppet.service; enabled; vendor preset: disabled) Active: active (running) (thawing) since Thu 2020-02-20 13:50:56 CET; 5 months 20 days ago Main PID: 20855 (puppet) IP: 94.9M in, 75.0M out Tasks: 5 (limit: 145001) Memory: 112.7M CPU: 4h 17min 29.438s CGroup: /system.slice/puppet.service ├─20855 /usr/bin/ru
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Damon Maria this fix should go out in the next set of releases this week. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.75032.1591038660028%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Damon Maria commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Experiencing the same on Ubuntu 18.04. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.74387.1590851340029%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Claire Cadman updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Claire Cadman Labels: doc_reviewed Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.73300.1590740640037%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Josh Cooper Fix Version/s: PUP 5.5.21 Fix Version/s: PUP 6.16.0 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.66584.1589991840031%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Mihai Buzgau updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Mihai Buzgau Sprint: NW - 2020-04-01, NW - 2020-04-15, NW - 2020-04-29, NW - 2020-05-13 , NW - 2020-05-27 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.61008.1589356680147%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Luchian Nemes updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Luchian Nemes Release Notes: Bug Fix Release Notes Summary: When two agent runs were taking place at about the same time, under certain circumstances, puppet agent could get in a repeatedly `restart_requested` state. It was fixed by cleaning up the puppet daemon code logic. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.59628.1589265720026%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Merged to 5.5.x in https://github.com/puppetlabs/puppet/commit/939ab7ee958e82974f199f84ccb2d2bd1b2c72fc Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.59429.1589236020084%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Mihai Buzgau updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Mihai Buzgau Sprint: NW - 2020-04-01, NW - 2020-04-15, NW - 2020-04-29 , NW - 2020-05-13 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.49912.1588149660171%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Mihai Buzgau updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Mihai Buzgau Sprint: NW - 2020-04-01, NW - 2020-04-15 , NW - 2020-04-29 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.42534.1587450300117%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title
Tim Meusel commented on PUP-10351
Re: Puppet Agent fails to self-restart on config change or update
I've multiple centos boxes, but most of them have nothing in their journal except the restart message.
root ~ # facter -p os
{
architecture => "x86_64",
family => "RedHat",
hardware => "x86_64",
name => "CentOS",
release => {
full => "7.7.1908",
major => "7",
minor => "7"
},
selinux => {
config_mode => "enforcing",
config_policy => "targeted",
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Mihai Buzgau commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Tim Meusel can you paste the log before the first lock started happening? Can you also provide more information about your environment (eg. are you using PE)? Do you have an insight about what changed that triggered the puppet agent restart, did you upgrade the puppet agent version, what configuration changed? It would be great if you could set your log_level to debug and post the full log of an affected node. Can you provide information about your environment (Mater OS, Agent OS) Are you using any specific module to manage the Puppet Agent configuration? Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.41606.1587373440035%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Tim Meusel commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update I thought to try something new and noticed that I could just start the agent on the cmdline, if the daemon logs `Shutdown/restart in progress (:restart_requested); skipping run`. At least on this box it wasn't required to kill the agent. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.41585.158736927%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Tim Meusel commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Hello people, I see this error on multiple nodes in my environment, at the moment all of them are running puppet 6.12.0 and 6.14.0: Apr 09 12:03:19 puppet-agent[27550]: Starting Puppet client version 6.12.0 Apr 09 12:03:19 puppet-agent[27550]: Could not run: Could not create PID file: /var/run/puppetlabs/agent.pid Apr 09 12:03:19 systemd[1]: puppet.service: main process exited, code=exited, status=1/FAILURE Apr 09 12:03:19 systemd[1]: Unit puppet.service entered failed state. Apr 09 12:03:19 systemd[1]: puppet.service failed. Apr 09 12:03:23 puppet-agent[21586]: Config file /etc/puppetlabs/puppet/puppet.conf changed; triggering re-parse of all config files. Apr 09 13:59:35 puppet-agent[21586]: Shutdown/restart in progress (:restart_requested); skipping run Apr 09 15:59:35 puppet-agent[21586]: Shutdown/restart in progress (:restart_requested); skipping run Apr 09 17:59:35 puppet-agent[21586]: Shutdown/restart in progress (:restart_requested); skipping run Apr 09 19:59:35 puppet-agent[21586]: Shutdown/restart in progress (:restart_requested); skipping run Apr 09 21
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title
Mihai Buzgau commented on PUP-10351
Re: Puppet Agent fails to self-restart on config change or update
HUP is sent to the puppet agent process whenever a service reload is issued. We were able to get a similar log to the one in this ticket by using the following manifests in an agent server environment:
class systemd::daemon_reload {
exec { '/usr/bin/systemctl reload puppet':
refreshonly => true,
}
exec { '/usr/bin/systemctl daemon-reload':
refreshonly => true,
}
}
$str = "[main] server = myhost runinterval = 60
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Mihai Buzgau updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Mihai Buzgau Sprint: NW - 2020-04-01 , NW - 2020-04-15 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.27213.1585727460319%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Luchian Nemes commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Nick Maludy/Frank Wall, could you please provide us the exact steps you're taking to reproduce/get to this state on a clean os? (manifests are also very welcomed) Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.25454.1585646880131%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Nick Maludy commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update I'm using both RHEL 7.7 and CentOS 7.7, yes. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.23657.1585588080276%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update Ah I tested using ubuntu xenial but I see the report is for CentOS 7.7.1908. Nick Maludy are you using that centos too? Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.23633.1585587720076%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Nick Maludy commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update FYI manually restarting the Puppet service seems to fix the problem. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.23585.1585586520091%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Luchian Nemes assigned an issue to Luchian Nemes Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Luchian Nemes Assignee: Luchian Nemes Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.23390.1585554840043%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Nick Maludy commented on PUP-10351 Re: Puppet Agent fails to self-restart on config change or update I'm seeing this on 6.13.0 Would love to help debug this is possible! Any steps i can do debug this? Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.17900.1584970860028%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Mihai Buzgau updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Mihai Buzgau Sprint: PR NW - Triage 2020-04-01 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.15024.1584523680043%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Mihai Buzgau updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Mihai Buzgau Story Points: 2 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.15025.1584523680099%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Josh Cooper Team: Night's Watch Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.4040.1583344500199%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title
Josh Cooper commented on PUP-10351
Re: Puppet Agent fails to self-restart on config change or update
I cannot reproduce this using nightly puppet-agent in cases where I send it HUP, USR1 or edit puppet.conf. This may have been fixed by the state pidfile detection correctly handling LWP on Linux. See PUP-10218.
root@worse-mandate:~# puppet resource package puppet-agent
package { 'puppet-agent':
ensure => '6.13.0.111.g401fef03-1xenial',
provider => 'apt',
}
# systemctl status puppet
● puppet.service - Puppet agent
Loaded: loaded (/lib/systemd/system/puppet.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2020-03-04 17:46:55 UTC; 4min 54s ago
Main PID: 2165 (puppet)
CGroup: /system.slice/puppet.service
└─2165 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent --no-daemonize
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Josh Cooper Epic Link: PUP-7532 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.348520.1583329295000.4039.1583344500155%40Atlassian.JIRA.
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper updated an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Josh Cooper Affects Version/s: PUP 6.13.0 Affects Version/s: PUP 6.10.1 Affects Version/s: PUP 6.11.0 Affects Version/s: PUP 6.12.0 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://grou
Jira (PUP-10351) Puppet Agent fails to self-restart on config change or update
Title: Message Title Josh Cooper moved an issue Puppet / PUP-10351 Puppet Agent fails to self-restart on config change or update Change By: Josh Cooper Key: PA PUP - 3165 10351 Affects Version/s: puppet-agent 6.13.0 Affects Version/s: puppet-agent 6.12.0 Affects Version/s: puppet-agent 6.11.0 Affects Version/s: puppet-agent 6.10.1 Project: Puppet Agent Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
