Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Matt Dainty commented on PUP-9792 Re: Merge of lookup_options not working https://github.com/puppetlabs/puppet/pull/7726 Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.101139.1568732040166%40Atlassian.JIRA.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Matt Dainty commented on PUP-9792 Re: Merge of lookup_options not working Eventually I managed to unpick the multiple levels of caching and the following crude diff fixes the behaviour: diff --git a/lib/puppet/pops/lookup/lookup_adapter.rb b/lib/puppet/pops/lookup/lookup_adapter.rb index 4014aea..d531705 100644 --- a/lib/puppet/pops/lookup/lookup_adapter.rb +++ b/lib/puppet/pops/lookup/lookup_adapter.rb @@ -233,12 +233,12 @@ class LookupAdapter < DataAdapter module_name = key.module_name # Retrieve the options for the module. We use nil as a key in case we have no module -if !@lookup_options.include?(module_name) +#if !@lookup_options.include?(module_name) options = retrieve_lookup_options(module_name, lookup_invocation, MergeStrategy.strategy(HASH)) - @lookup_options[module_name] = options -else - options = @lookup_options[module_name]
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title
Matt Dainty commented on PUP-9792
Re: Merge of lookup_options not working
I've managed to get my example to work by changing manifests/site.pp to just the following:
$bar = 'bar'
notice(lookup('foo'))
Basically by ensuring the top scope variable is set before any lookup() is called, it gets the correct merge behaviour. This does suggest that lookup_options is getting cached for the lifetime of the compile. I'm trying to find where in the code this caching is done but I'm not having much luck so far.
Add Comment
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
--
You received this message because you are
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Matt Dainty commented on PUP-9792 Re: Merge of lookup_options not working There's no such option, (on 5.5.16 at least): Error: Could not parse application options: invalid option: --evaluate I'm already using --compile as part of my tests, however I don't get the correct answer with the puppet lookup CLI, nor within a normal catalog compile. Your first comment suggested a lack of cache eviction, my example does suggest the lookup options aren't re-evaluated once the top scope variables are set. Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.99263.1568643240423%40Atlassian.JIRA.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Henrik Lindberg commented on PUP-9792 Re: Merge of lookup_options not working In order to use top scope variables in your hiera.yaml and get the correct answer from puppet lookup CLI, you must use the --evaluate option. Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.97883.1568396640212%40Atlassian.JIRA.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title
Matt Dainty commented on PUP-9792
Re: Merge of lookup_options not working
Sorry, I didn't see your comments. I think I've managed to reduce this down to the following contrived example: hiera.yaml
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: YAML
paths:
- "%{bar}.yaml"
- "common.yaml"
data/common.yaml
---
lookup_options:
foo
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Jorie Tappa commented on PUP-9792 Re: Merge of lookup_options not working Matt Dainty can you please help us with a simple example of how we can reproduce this on our end? Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.25354.1563815103089%40Atlassian.JIRA.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Henrik Lindberg commented on PUP-9792 Re: Merge of lookup_options not working Jorie Tappa Someone needs to set up and try to reproduce his case. I added some notes about things to look at - for example, does this work if the hierarchy does not depend on compilation - something that Matt Dainty could try as well as someone at Puppet. This could also be a bug that has been fixed as he is using Puppet 5.5.14. Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.16749.1563227640192%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Jorie Tappa commented on PUP-9792 Re: Merge of lookup_options not working ping Henrik Lindberg does this ticket need action, or more information from Matt Dainty? Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.16148.1563211800135%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Henrik Lindberg commented on PUP-9792 Re: Merge of lookup_options not working The lookup_options merge is not a deep merge - if overriding, all information for the specific key the options are for must be given. Not sure if that has any bearing here. Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.56918.1561140840156%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Henrik Lindberg commented on PUP-9792 Re: Merge of lookup_options not working IIRC, the lookup_options are looked up once for everything. If the hierarchy then depends on changing variables leading to different paths being looked up, it is supposed to evict the cache, maybe that does not happen for lookup_options. Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.313541.1561115342000.56913.1561140660653%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title
Matt Dainty updated an issue
Puppet / PUP-9792
Merge of lookup_options not working
Change By:
Matt Dainty
*Puppet Version: 5.5.14* *Puppet Server Version: 5.3.8* *OS Name/Version: RHEL 7.x*In our Hiera data, we have made use of lookup_options to control the merge behaviour, so in our catch-all common.yaml there are entries such as:{noformat} ---lookup_options: profile::autofs::mounts:merge: hash {noformat}This works as desired. However we have some exceptions where we want to override this on a per-role basis back to the original "first" behaviour, so as we have a higher priority role level, I added the following in a role/foo.yaml:{noformat} ---lookup_options: profile::autofs::mounts:merge: first {noformat}If I use the "puppet lookup" command against the correct host, this appears to work as desired, using `--explain-options` I get the following result:{noformat} Merged result: {"profile::autofs::mounts" => { "merge" => "first"} }{noformat}For any other hosts I still get the behaviour defined at the common level:{noformat} Merged result: {"profile::autofs::mounts" => { "merge" => "hash"} } {noformat}All good. However this doesn't seem to actually apply to the lookup itself:{noformat} Using merge options from "lookup_options" hashSearching for "profile::autofs::mounts" Merge strategy hash ... {noformat}It only ever picks up the behaviour defined at the common level.*Desired Behavior:*I would expect to see:{noformat} Using merge options from "lookup_options" first hash Searching for "profile::autofs::mounts" ... {noformat} i.e. (no merge strategy displayed here) *Actual Behavior:*One bit of information that might be relevant is that in order for our lookups to work from the CLI, I have to use the --compile option, so basically something like:{noformat} puppet lookup --compile --environment --node profile::autofs::mounts --explain(-options){noformat}We have multiple backends/levels configured in Hiera, a mix of YAML/eYAML and using the HTTP backend too. I noticed there was PUP-7037 which was raised for similar behaviour but was ultimately traced to a typo. I'm confident this isn't down to a typo. Edit: After a bit more experimenting, I was a bit confused about where "hash" should be printed vs other merge strategies, the 'Using merge options from "lookup_options" hash' is pretty much a fixed string, it's the presence/lack of "Merge strategy ..." that is the identifier.
Jira (PUP-9792) Merge of lookup_options not working
Title: Message Title Matt Dainty created an issue Puppet / PUP-9792 Merge of lookup_options not working Issue Type: Bug Affects Versions: PUP 5.5.14 Assignee: Unassigned Components: Hiera & Lookup Created: 2019/06/21 4:09 AM Priority: Normal Reporter: Matt Dainty Puppet Version: 5.5.14 Puppet Server Version: 5.3.8 OS Name/Version: RHEL 7.x In our Hiera data, we have made use of lookup_options to control the merge behaviour, so in our catch-all common.yaml there are entries such as: --- lookup_options: profile::autofs::mounts: merge: hash This works a
