Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Jorie Tappa updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Jorie Tappa Labels: resolved-issue-added Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: Agent 2017-04-19, Agent 2017-05-03, Agent 2017-05-31, Agent 2017-06-14 , Agent 2017-06-28 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Release Notes: Bug Fix Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title John Duarte assigned an issue to John Duarte Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: John Duarte Assignee: qa John Duarte Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title John Duarte assigned an issue to qa Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: John Duarte Status: Ready for CI Test Assignee: Josh Cooper qa Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title John Duarte updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: John Duarte Sprint: Agent 2017-04-19, Agent 2017-05-03, Agent N+1, Agent 2017-05-31 , Agent 2017-06-14 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Josh Cooper commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Merged to master in https://github.com/puppetlabs/puppet/commit/a14346cdc1e949052d118bf903698924ded72c7d. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Fix Version/s: PUP 5.y Fix Version/s: PUP 5.0.0 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Kenn Hussey updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Kenn Hussey Sprint: Agent 2017-04-19, Agent 2017-05-03, Agent 2017-05-31 N+1 , Agent 2017- 06 05 - 14 31 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Kenn Hussey updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Kenn Hussey Fix Version/s: PUP 5.0.0 Fix Version/s: PUP 5.y Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: Agent 2017-04-19, Agent 2017-05-03, Agent 2017-05-31 , Agent 2017-06-14 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza assigned an issue to Josh Cooper Josh Cooper can you take a look? Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Moses Mendoza Assignee: Josh Cooper Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title John Duarte updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: John Duarte Labels: triaged Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza assigned an issue to Unassigned Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Moses Mendoza Assignee: Moses Mendoza Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Puppet PR: https://github.com/puppetlabs/puppet/pull/5866 Classifier PR: https://github.com/puppetlabs/classifier/pull/584 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Andrey Galkin no worries, thank you!! With that info I now understand and can trace through why you are also experiencing this behavior! As mentioned above, that doesn't mean its not a bug and I hope to have this addressed in short order. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Andrey Galkin commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Hi Moses Mendoza. At the moment, I see the second master in lab environment is missing routes.yaml. So, only the primary master has one. I believe the situation was the same in the original testing as puppetdb-termini apparently does not install one automatically. Sorry for the misleading previous comment. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Hi Andrey Galkin - just got to pick this up again (had to delay for a bit with some other work). One question for you - can you confirm you have `/etc/puppetlabs/puppet/routes.yaml` deployed on both of your masters from the example? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Moses Mendoza Sprint: Agent 2017-04-19, Agent 2017-05-03, Agent N+1 2017-05-31 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Kenn Hussey commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters @moses is this still in progress (in which case it should probably be brought into the current sprint)? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: Agent 2017-04-19, Agent 2017-05-03 , Agent N+1 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Mark Frost commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Moses Mendoza I can confirm that adding the routes.yaml back into our configuration (we had it for Puppet 3, but had purged it during our Puppet 4 migration) alleviates this bug, at least in the form that I reported it in PUP-7231 . For our particular case, this is a perfectly sufficient workaround. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Sidenote I was incorrect in my original assertion that it is the default to not have a `routes.yaml` file in open source with the cache class set to `yaml` - the open source puppetlabs-puppetdb module configures this file by default and sets the cache this way. I think the only way to end up without it is if you've overridden the module default, or have installed the puppetdb package and configured it manually. Andrey Galkin you do not need to remove this file - your configuration is the recommended one. I believe this issue is still, as we've hypothesized, a race condition, and in this multi-master case I think one piece of the puzzle might be the facts getting cached on the first master but not the second, so the second run queries PDB which might not have been updated yet due to its asynchronous nature. Mark Frost you might try adding a routes.yaml file identical to the one on https://docs.puppet.com/puppet/4.10/config_file_routes.html, and see if this alleviates your issue. As noted above it will change it so facts are cached synchronously to a file on the master rather than asynchronously to puppetdb (note the facts will still be saved to puppetdb, just cached on the master - sorry for the confusing behavior here). It's not guaranteed to always solve the problem as we've seen in Andrey Galkin's scenario, and it will also not help if there is clock drift between the agent and master as the master might find the cache to be expired and search puppetdb anyway. As most commenters have suggested, I think ultimately the root cause to all of these issues is the circuitous (and inefficient) path our facts take during the agent catalog request, resulting in a scenario where the facts submitted by the agent aren't guaranteed to be the facts used by the master for compilation, and that's what I'm working on hopefully addressing. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Mark Frost commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters I'm the reporter on PUP-7231 , and we do not use a routes.yaml file in our environment. Like Andrey, though, I'm not sure if the suggestion is to add or remove the routes.yaml, as a fix. But I'm guessing since Andrey and I appear to be on opposite sides (i.e. he's using it already and I'm not), maybe that's not going to work. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Andrey Galkin commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Moses Mendoza Regarding 'settings' error, I switched to use unlimited environment_timeout & API calls for refresh during deployment. So, did not pay much attention. Regarding routes.yaml - yes, it it used exactly as in the example. Do you suggest to remove one? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Andrey Galkin Jacky Leung do either of your masters use `/etc/puppetlabs/puppet/routes.yaml` (see https://docs.puppet.com/puppet/4.10/config_file_routes.html) to override any of the puppet's indirection terminii? If you're using Puppet Enterprise this would be the default I believe, but I don't think it is the default to have this file in open source. Just confirming because in Puppet Enterprise the cache class for facts is overridden to `yaml` which is a synchronous cache write, as opposed to `store_configs` (i.e., PDB) which is an asynchronous cache write. If your cache class is `store_configs` this greatly aligns with the hypothesis of a race condition. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Hi Andrey Galkin - just letting you know we're still working on this. As a sidenote, as I was reviewing the puppet server logs you included and I noticed this warning: Puppet Class 'settings' is already defined; cannot redefine at : Found that unusual and tracked it to https://tickets.puppetlabs.com/browse/PUP-7328 which should be addressed in puppet 4.10.0 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: Agent 2017-04-19 , Agent 2017-05-03 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Jeremy Barlow updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Jeremy Barlow Component/s: Puppet Server Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title John Duarte updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: John Duarte QA Risk Assessment: Needs Assessment Manual Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Moses Mendoza Sprint: Agent Ready for Engineering 2017-04-19 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza assigned an issue to Moses Mendoza Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Moses Mendoza Assignee: Moses Mendoza Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: Agent Grooming Ready for Engineering Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Fix Version/s: PUP 5.0.0 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Story Points: 8 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: Agent Grooming On-Deck Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters thanks Jacky Leung for that detailed investigation! that's helpful Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Jacky Leung commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Hi i can give some update and can confirm it is racing condition. I followed Andrey Galkin suggestion and modified my puppetserver. here is the log on puppetdb Before: puppetserver.docker_default - - [14/Mar/2017:04:31:23 +] "GET /pdb/query/v4/nodes/search-slave-4.srv.fish/facts HTTP/1.1" 200 22984 "-" "Apache-HttpAsyncClient/4.1.1 (Java/1.8.0_91)" puppetserver.docker_default - - [14/Mar/2017:04:31:24 +] "POST /pdb/cmd/v1?checksum=49141ee1c6874add827e8d1c0954cfe39b3634c7&version=5&certname=search-slave-4.srv.fish&command=replace_facts HTTP/1.1" 200 53 "-" "Apache-HttpAsyncClient/4.1.1 (Java/1.8.0_91)" puppetserver.docker_default - - [14/Mar/2017:04:31:24 +] "GET /pdb/query/v4/nodes/search-slave-4.srv.fish/facts HTTP/1.1" 200 22984 "-" "Apache-HttpAsyncClient/4.1.1 (Java/1.8.0_91)" 2017-03-14 04:31:24,516 INFO [p.p.command] [a2c40a11-6973-427c-a86f-a70670a7af42] [replace facts] search-slave-4.srv.fish puppetserver.docker_default - - [14/Mar/2017:04:31:24 +] "POST /pdb/cmd/v1?checksum=9cc4a64fcac878174ac1ee8ae0cc924339984d38&version=9&certname=search-slave-4.srv.fish&command=replace_catalog HTTP/1.1" 200 53 "-" "Apache-HttpAsyncClient/4.1.1 (Java/1.8.0_91)" 2017-03-14 04:31:25,009 INFO [p.p.command] [c9649961-55a7-40e8-9949-6d4851b5bf1c] [replace catalog] search-slave-4.srv.fish puppetserver.docker_default - - [14/Mar/2017:04:31:25 +] "POST /pdb/cmd/v1?checksum=2ee733a04e4a58ae49c10d8a92d680215ceb41dd&version=8&certname=search-slave-4.srv.fish&command
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Jacky Leung commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Andrey Galkin i have this issue in a single puppetserver instance. not sure is https://tickets.puppetlabs.com/browse/PDB-3058 also related as i am currently migrating from puppet3 to puppet4 and i have some issues on storing puppet reports and facts Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: AP Grooming On-Deck Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: AP Grooming On-Deck Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: AP Grooming On-Deck Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Geoff Nichols updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Geoff Nichols Sprint: Puppet Triage AP Grooming Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title John Duarte updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: John Duarte Team: Agent & Platform Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Andrey Galkin commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters ... and definitely, timestamps should not come to play when we generate a new catalog. It's obvious that agent provided data is the actual state. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Andrey Galkin commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters In my testing, all nodes are synced through ntpdate initially and then through constantly running ntpd against the same source. Time drift is no more than 0.01 seconds. Each puppet agent run takes way much time. Regarding timestamp, also related: #PDB-3248 IMO, it does not look right at all to compare timestamps. I understand, it can be a sort of mitigation for highly available PuppetDB setup, but it would obviously be much safer to have UUID of agent + counter (generation) of facts/resources. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Jeremy Barlow commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Henrik Lindberg, I'm drawing blanks on this but maybe you remember better. I had thought that there might be some code in one of the terminii on the server which compares a timestamp from the agent request to the timestamp of facts that had been previously cached and, if the agent timestamp was sufficiently older, would cause the facts cached on the server to be used instead. Does that ring a bell for you? I wonder in the case of this ticket whether there might have been a significant difference in local time configured on the agent node vs. the two master nodes. If the three were all set to roughly the same time and there were no significant delay in processing the catalog request on the master, would the problem no longer be reproducible? A ticket that seems like it may be related is PUP-3438. It would seem if we were to eliminate the behavior of facts cached on the server being used in lieu of the facts actually provided with the catalog request, that this sort of problem wouldn't occur. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Andrey Galkin commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Guys, it's a race condition, I am quite sure for that: I put delay (breakpoint + wait 10 seconds + continue) between extract_facts_from_request_facts() and node_from_request() in Puppet::Resource::Catalog::Compiler then everything works as expected. /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:51 Now, I wonder if that can be reproduced on single server instance as well... Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Andrey Galkin commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Some more info: I did restart of both PuppetServer and PuppetDB. The issue remains. It seems obvious that by some reason, PuppetServer uses old facts for compilation, but then submits new facts to YAML cache and database. I've specially checked that both PostgreSQL and YAML cache at /opt/puppetlabs/server/data/puppetserver/yaml/facts/cacheissue.example.com.yaml contain value which is not present anywhere on agent's host. So, every run switches the values. Current fact from client goes to database, but PuppetServer uses the previous value for compilation. All packages come from PuppetLabs OpenSource repo. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Andrey Galkin commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters I've extended the test to shutdown PuppetServer & Puppet DB on primary master and PostgreSQL replica on secondary. The result is the same. I can provide scripts for complete setup of vagrant environment from scratch with a few clicks and around 10 minutes for processing. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Henrik Lindberg commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters Wild guess: Facts posted on one master, read from the other? Posting facts goes to cache and PDB, PDB write is lazy? Read from master 2 does not have values in cache, hits PDB, and PDB write not yet complete? Reads old written value. henrik Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Jeremy Barlow commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters I don't have any immediate ideas as to what might be causing this. It may be that the value of the 'fact_cache_issue' fact is somehow erroneously getting cached into the environment on the master instead of being processed from the agent's catalog request. If that were happening, it could be that a different value is being cached per JRuby instance on the server, causing the content which is applied to flop back and forth per agent run. I don't have any theories as to why switching from one master to another between agent runs may be causing the problem, though. It might be interesting to see if puppet.conf file on each of the masters - the primary and secondary - has any differences. What is the environment_timeout setting set to for the environment in question on each of the masters? Are both the primary and secondary masters running the same Puppet Server and puppet-agent versions? Henrik Lindberg or Thomas Hallgren - can you think of what might cause this problem? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters thanks for filing Andrey Galkin Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Moses Mendoza commented on PUP-7198 Re: Puppet Server uses facts from the previous run (n-1) after agent switches masters ping Jeremy Barlow Patrick Carlisle for review/awareness Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title Michael Smith updated an issue Puppet / PUP-7198 Puppet Server uses facts from the previous run (n-1) after agent switches masters Change By: Michael Smith Sprint: Puppet Triage Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7198) Puppet Server uses facts from the previous run (n-1) after agent switches masters
Title: Message Title
Andrey Galkin updated an issue
Puppet / PUP-7198
Puppet Server uses facts from the previous run (n-1) after agent switches masters
Change By:
Andrey Galkin
I couldn't believe that, but I could finally reliably reproduce the problem in minimal sample. I've observed that episodically and believe this is a long standing issue - the reason I do not put Critical to this.Ruby fact plugin:{code:ruby}Facter.add('fact_cache_issue') dosetcode dobeginFile.read('/etc/fact_cache_issue')rescueendendend{code}Puppet code:{code:puppet}class factcacheissue {$value = $::facts['fact_cache_issue'] ? {undef => generate('/usr/bin/openssl', 'rand', '-hex', '8'),default => $::facts['fact_cache_issue']}file { '/etc/fact_cache_issue':content => $value,}}node cacheissue.example.com {include factcacheissuefile {'/etc/puppetlabs/puppet/puppet.conf':mode=> '0644',content => '[main]client = falsecertname = cacheissue.example.comserver = puppetback.example.comca_server = puppet.example.comenvironment = productionsrv_domain = example.comuse_srv_records = false',}}{code}1. Setup puppet-agent 1.9.0 package2. Run against CA server puppet.example.com3. During the first initialization, openssl generator is called and puppet server is changed to secondary master (puppetback.example.com)4. Second run goes to secondary master. Master stores facts PuppetDB, but Master does not see new facts during catalog compilations => calls openssl generator again .5. Each next run uses fact set from run #n-1 - *that's very weird*, and produces resource value switch to previous run.6. If puppet server is not switched then everything is correct.Command line output:{noformat}Setting up puppet-agent (1.9.0-1jessie) ...Configuration file '/etc/puppetlabs/puppet/puppet.conf' ==> File on system created by you or by a script. ==> File also in package provided by package maintainer. ==> Using current old file as you requested.Created symlink from /etc/systemd/system/multi-user.target.wants/puppet.service to /lib/systemd/system/puppet.service.Created symlink from /etc/systemd/system/multi-user.target.wants/mcollective.service to /lib/systemd/system/mcollective.service.Created symlink from /etc/systemd/system/multi-user.target.wants/pxp-agent.service to /lib/systemd/system/pxp-agent.service.Removed symlink /etc/systemd/system/multi-user.target.wants/pxp-agent.service.+ PUPPET=/opt/puppetlabs/bin/puppet+ /opt/puppetlabs/bin/puppet resource service puppet ensure=false enable=false provider=systemdNotice: /Service[puppet]/enable: enable changed 'true' to 'false'service { 'puppet': ensure => 'stopped', enable => 'false',}+ systemctl mask puppetCreated symlink from /etc/systemd/system/puppet.service to /dev/null.+ host maint.example.com+ /opt/puppetlabs/bin/puppet agent --test --traceInfo: Creating a new SSL key for cacheissue.example.comInfo: Caching certificate for caInfo: csr_attributes file loading from /etc/puppetlabs/puppet/csr_attributes.yamlInfo: Creating a new SSL certificate request for cacheissue.example.comInfo: Certificate Request fingerprint (SHA256): DA:8A:E9:2C:01:A4:2B:B4:DA:28:66:89:8D:91:9B:4F:76:DF:62:4F:EA:A5:28:68:03:0B:20:92:03:01:70:01Info: Caching certi
