subject:"Jira \(PUP\-7332\) add ability to lookup \"below\" in hierarchy"

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Thomas Hallgren commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
It's unclear to me how the result from this filter would participate in a merge. Let's assume that we have the participants A, F, B, and C where F is the filter. Without the filter, A, B, and C would be merged. What should be merged instead? Is it A, F, and C or just A and F? I.e. does F "take over" all subsequent entries or just the next one? If it's the latter, then I assume that F should be fed the already merged result of B and C? 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Thomas Hallgren commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
A follow-up question is of course if it wouldn't be better if the F took over everything so that the framework stops processing at F. Instead it's up to F to do the rest using a lookup_below. That will give F a chance to override merge strategies and even change the key. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Thomas Hallgren commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
Alternative proposal. 
Combine the ability to reorder layers with the ability to add a filter. Make it clear what it is that a filter acts on by declaring a sub-hierarchy: 
 
 
 
 
 
 
hierarchy: 
 
 
 
 
  - name: Common 
 
 
 
 
path: common.yaml 
 
 
 
 
  
 
 
 
 
  - sub_hierarchy: 
 
 
 
 
  filter:  # optional lookup_key function acting on sub-hierarchy using Context methods. 
 
 
 
 
  hierarchy: 
 
 
 
 
 # normal hierarchy 
 
 
 
 
  
 
 
 
 
  - name: Roles

Jira (PUP-7332) add ability to lookup "below" in hierarchy

2017-03-24 Thread Henrik Lindberg (JIRA)

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Henrik Lindberg commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
I don't see how that solves "default hierarchy" inside of a module. Including the underlying layer more than once is too weird. For every filter I want I must introduce one sub hierarchy. I must be explicit about the lower layer or the filter will not filter data from lower layer(s). 
I also struggle to come up with meaningful use cases where I would need an actual nested hierarchy (except for the default hierarchy in a module, but that cannot be intermixed into the regular hierarchy). 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Thomas Hallgren commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
The default hierarchy of a module will still be consulted when everything else fails, and only then. 
And you don't need to introduce any more hierarchy elements than before. Just organize them differently (which also highlights what the filter is using). I'd consider that a very useful way to declare what it is you are doing. The filters are in fact mechanism that introduces nesting. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Thomas Hallgren commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
Revised version of my alternative proposal. Instead of using the term sub_hierarchy, we should simply use hierarchy for the nested hierarchies as well and allow that a hierarchy has a filter functions. Filter functions can be inserted anywhere in a hierarchy as a single string element (as opposed to the hash elements used for other hierarchy entries). A filter then acts on everything "below" its declaration. 
In essence, this means that a filter doesn't need to act on a sub hierarchy. I may (if declared as the first element in a hierarchy), but it can also be declared in arbitrary places in a flat hierarchy, e.g.: 
 
 
 
 
 
 
hierarchy: 
 
 
 
 
  - name: Common 
 
 
 
 
path: common.yaml 
 
 
 
 
  - my_filter_func 
 
 
 
 
  - name: Filtered data 
 
 
 
 
path: filtered.yaml
 
 
 
 
 
 
 
Logically, this would be exactly the same as declaring: 
 
 
 
 
 
 
hierarchy:

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Henrik Lindberg commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
So, are you saying that if the filter is in the top level hierarchy, then all entries below it in the same hierarchy and all entries in all lower layers are filtered? (assuming that the next layer down is not specifically referenced and therefore ends up last in the top hierarchy in the hiera.yaml in question). 
If that is true, then this proposal is workable. I still think the extra ability to have a sub hierarchy may be overkill. (I understand that this is made possible because it becomes a composition of uniform parts which is nice because of its generality). 
One thing to note - it must be possible to pass options to a filter function (e.g. you want to replace "blue" with "green" and don't want that hardcoded in the function itself). You may also want to call the same filter function with different arguments in different places in the hierarchy, and you therefore need a name for the entry to understand explain output. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title

Thomas Hallgren commented on PUP-7332

Re: add ability to lookup "below" in hierarchy

Yes, that's what I'm saying.
Having sub hierarchies might be overkill, but please consider that their presence solves three problems.

The ability to reorder (PUP-7333), since any hierarchy containing a sub-hierarchy is prevented from continuing with the next layer.

The ability to have more than one filter that acts on diverse parts of the hierarchy.

The ability to add unfiltered parts after the filter.

If options are needed, then the filter function cannot be an array. It must be a hash, and as such, must be uniquely identified from normal hierarchy entries. Perhaps by using the unique key 'filter' for the function. Locations (urls, paths, globs, mapped paths) must be forbidden. While using 'options' as a key might be appealing at first, we might want to consider some other key to avoid that it is assigned the 'options' from the 'defaults' entry.

Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title

Henrik Lindberg commented on PUP-7332

Re: add ability to lookup "below" in hierarchy

I don't understand your comment in #1 - It must naturally be possible to filter what comes from a lower layer - the feature is more or less useless otherwise. Are you saying that it is illegal to reference a layer in a subhierarchy? Thus, if you want to filter a layer you must have your filter in the hiera.yaml's top hierarchy?
The #2 is the feature I think is overkill and that would require allowing lower layer multiple times in different sub hierarchies in order to be of real and general value. I can however accept that you can do this as there may be cases where you are combining some backends in the same layer and you need to filter something locally. The filter acts like a wrapper where you can introduce your own post processing.
The #3 seems useful. If answer to my question about #1 is 'yes', then the combination of filtered sublayer with an unfiltered thing after the filter does not seem possible... If #1 is really "only the top hierarchy, or one of the sub-hierarchies may include the lower layer(s), then it works.
Some details:

Options are required

path(s)/glob(s)/mapped-paths not supported (error)

uri support is useful - for example, may filter by talking to a service and need a URL. Seems strange if not the same param ("uri") for this.

I think using a different name for "options" is more confusing than applying the general rule. Also think filter functions are more likely to take parameters than being completely hardcoded, having to pass an empty options hash to override defaults is therefore a very rare operation.

Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Thomas Hallgren commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
Of course you should filter what comes from a lower layer. That's why I propose that it works the way it does. A sub-hierarchy takes over the responsibility for the lower layers from it's parent. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Henrik Lindberg commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
So if I have a sub hierarchy then it automatically filters the lower layer(s)? That was surprising. What if I have two sub hierarchies ? Confused. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Thomas Hallgren commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
This is covered in my original proposal (starts with "Alternative proposal."). See #3 above "pros" and also what I mention as a caveat. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy

Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Henrik Lindberg commented on  PUP-7332 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: add ability to lookup "below" in hierarchy  
 
 
 
 
 
 
 
 
 
 
How I think it would work is that you can include the lower layer(s) anywhere - either in the top hierarchy in the hiera.yaml, or in a sub hierarchy. If it is not included anywhere, it is placed last in the top hierarchy. You are only allowed to include the lower level(s) once in a hiera.yaml. 
Is this correct? Then it works for me. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-7332) add ability to lookup "below" in hierarchy