Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Henrik Lindberg updated an issue Puppet / PUP-1974 Mechanism for securing sensitive catalog data Change By: Henrik Lindberg Scrum Team: Language Scrum Team/s: Client Platform,Language,Puppet Server,PuppetDB Epic Status: To Do Epic Name: Theme: Sensitive Data Status: Needs Information Open Workflow: Scrum Team Engineering Epic Workflow Issue Type: Improvement Epic Add Comment
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Henrik Lindberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data meh - you cannot change an Improvement into an Epic (at least not by just changing the type). Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Henrik Lindberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data In order to be able to break things down and handle them by different teams I am changing this Epic into a Theme. Whatever we decide will need to be handled with several tickets in several teams, and each need their own Epic. This to enable a break down and setting of story points. (We will never get there until we do that since it will appear as a huge problem to tackle and the fizzle). I am about to create an Epic that defines how this would work if we have the Secure type I described earlier (although I changed the name to Sensitive, with Classified being a runner-up name-wise. Will add a link that that epic support this theme. Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Kylo Ginsberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data /me puts down light saber I had two more thoughts on this: 1) On the logging side, we need to call out what puppet options we should be considering. E.g. there's log_level of course (presumably all levels). Plus, I was forgetting about show_diff until I read Chris Barker's comments. Plus, and things get a bit tricky here, late in the 3.x series we added http_debug, which outputs full contents of the http requests/responses (so catalogs and reports). 2) I'm thinking we don't want to do full-resource redaction from the catalog or you'd be introducing some weird scenarios, e.g. if a catalog application failed with only one, sensitive, resource, you'd have a failed report and no failed resources, etc. So I'm assuming we want to do something more surgical. We need to specify what. Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Henrik Lindberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data Before deciding exactly what needs to be done - lets look at the problem with what we need to keep track of. A scrub of all logging is probably going to reveal that there are several layers of indirection before something is logged (that plus what is logged at the surface level inside a Resource). That means we will have plain old strings, ints etc that are being processed at some point in the logic, then logged if something is amiss. The next problem is that there will be processing of values that are derived from a sensitive value - say something like "to_lower" on a String. In a ideal solution every value would have a security attribute. I explored if we could perhaps use the Ruby "taint mechanism" to flag particular objects; but it works in reverse to what we want i.e. it gives the ability to reject processing of tainted things, and you can untaint. Everything read with a gets/read is considered tainted. In the infinite wisdom of the Ruby designers, you cannot however taint numbers. Thus, to be able to mark numbers as sensitive these classes have to be monkeypatched, or wrapped, or we need to create derived classes that we use to represent the runtime values. This means having to override all operations since secure(1) + number(2) should yield secure(3). Needless to say, that will be a bit of an effort and will have performance impact. If we do not do this on the individual value level, and instead set just a sensitive meta parameter, then we could perhaps construct a derived SensitiveResource instead of just a Resource as that would give opportunity to deal with resource specific methods. Otherwise we probably need to sprinkle the Resource (and provider code) with lots of is_sensitive? calls. This also seems like a lot of work, and potentially require changes to every type/provider. Next idea is to use a Secure[T, C, placebo] wrapper, where T is the wrapped type and C is a krypto/key reference say "node" to encrypt for the requesting node, and placebo is a replacement value to show instead of the real value, the placebo must be a valid instance of T (i.e. "*" for a password, '' for an int, etc.), if undef is a valid value, then naturally that can be used (and is perhaps the default for placebo). We cannot include this in a catalog given the current catalog format (every data type possible in say yaml, are already taken). What we can do is to add a top level attribute in the catalog called sensitive_resources (next to the existing resource). When we deserialize this, we post process the yaml to construct Resource (or SensitiveResource depending on what we decide is the best) instances and then set the values as Secure|[T, C, placebo] instances. A Secure[T, C, placebo] is serialized with an encrypted real value, and the type, and placebo values in clear text. An operation on the secure instance allows it to be operated on in real value mode. We still need to review code, but by using another runtime value, it is easier to find all places where the actual value is operated on, then we know all the places where we have to present an unencrypted value; when we do so, we can shift to a global "secure mode" that will prevent logging. To be completely secure we would need to make some monkey patches to stdout/stderr etc. but that may break too much. It may be enough to change logging to a different mode (maybe when entering the protected mode, we keep track of the secure instances, and when logging,
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Chris Barker commented on PUP-1974 Re: Mechanism for securing sensitive catalog data Henrik Lindberg Kylo Ginsberg(ren) - I'd say client should be involved, because with sensitive enabled, the agent shouldn't be posting to standard out or to the system logs the diffs of the resource, etc. It could be something shown in a --debug invocation, but otherwise the report returns just the resource changed (if we want to present a checksum of the detected vs intended contents, so the user could still see it was a change to the catalog/manifest that caused the change vs the nodes resource being remediated). The first pass of sensitive / do not log could be around the assumption that a normal agent invocations of 'puppet agent -t' or the daemon runs, don't leave that sensitive information around a host (event logs, message log, etc) that a non privileged user could access easily. The secondary result of this is sensitive data, since its not in the reports, won't show up in PE or other tools report consoles either - we don't have to expect people to change all their report processors all of a sudden to take advantage of this new metaparam, it just returns doesn't show the sensitive component, instead a placemarker is there. Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Kylo Ginsberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data Henrik Lindberg good points. Re the client sending the data in clear in the report: my thought was that since the master is sending the data in clear in the catalog, there's minimal additional exposure to the agent sending the data in clear back in the report. I think the real win (for this initial tier of sensitive data handling) is that the data isn't lying out in the open in pdb. I.e. I'd say that protecting against someone snooping the wire is a much higher bar, outside the scope of this ticket. Re "do not log" on the compiler side, I can see where that might be very difficult. We'd really need to audit somehow all the places where we might log. The agent side might be somewhat simpler, in that the resource arrives fully realized, but similarly we'd need an audit for all the places where we might log. (And btw, if we attempt to address logs, I'd definitely recommend acceptance tests to validate - it's easy to imagine a regression from a developer adding an additional info message somewhere.) Eric Sorenson: thoughts on the above? Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title
Henrik Lindberg commented on PUP-1974
Re: Mechanism for securing sensitive catalog data
Yes, something like that - if it is ok that the agent sends the reports to the server with the sensitive information in clear text. I.e. it is up to the report processing side to filter out the information. If more is needed, it also becomes a client side matter.
The "do not log" on the compiler side is difficult (close to impossible) to handle, the information does not become sensitive until it is set in a resource that has that flag on. It in turn (as most meta parameters) are not set until late in the game (due to how defaults etc. work). It is really only if there are errors pertaining to an attribute at the very end of the compilation / transformation to wire format that this logging would know it is about sensitive data. To handle sensitive stuff in source code, the language would need to support a sensitive clause around the source that deals with sensitive data; then all errors would result in some generic error message ("error when evaluatiing sensitive clause"), or it could encrypt the real error mssage, etc.
Add Comment
This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Kylo Ginsberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data If we restrict this as discussed above (and if so, we should perhaps retitle this ticket), then I'm thinking of breaking this out into three tickets: add a new metaparameter: sensitive? (PUP ticket, language team) use that new metaparameter to not log messages (PUP ticket, language team) use that new metaparameter to filter what goes to pdb (PDB ticket) Henrik Lindberg, Eric Sorenson thoughts? Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Eric Sorenson commented on PUP-1974 Re: Mechanism for securing sensitive catalog data This may be something of interest for Lori Landesman as well, happy to talk through the use-cases here. Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Henrik Lindberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data The more I think about this, isn't this really more of a Puppet DB problem? IIRC, there is already handling of some password attributes - could this be done in a similar way, only for all attributes of a resource if it is marked as sensitive? If we just strip out sensitive resources, they cannot be exported/collected. What do we do when there are errors related to sensitive information - how do you see those? Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Eric Sorenson commented on PUP-1974 Re: Mechanism for securing sensitive catalog data Henrik Lindberg IMO unless there is some reason why implementing the metaparameter would prevent future end-to-end work (i don't think that it would, but it would fit together in a different/better way in future) i think it should not wait. this is a very common request and stopping the first order problem (which I would summarize as "someone viewing a report can see information they should not") would help people today. Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Henrik Lindberg commented on PUP-1974 Re: Mechanism for securing sensitive catalog data If this is only about preventing logging from taking place, then adding a sensitive meta parameter that when set to true prevents logging and removes resource from catalog before storing in puppet db, then that is doable without new catalog format and features related to that. "Securing Sensitive Data" is a complete epic. Should this ticket be used for "prevent logging of sensitive data"? In order to scope this we need to explore how/where logging takes place. At the moment this feels like a 5 point job (or more as it touches many parts of the system), Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Steve Barlow updated an issue Puppet / PUP-1974 Mechanism for securing sensitive catalog data Change By: Steve Barlow Sprint: Language Triage Scrum Team: Language Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Owen Rodabaugh updated an issue Puppet / PUP-1974 Mechanism for securing sensitive catalog data Change By: Owen Rodabaugh CS Priority: Needs Priority Major CS Impact: This has become more and more of an issue as time goes on and customers realize secrets are stored in plaintext. CS Severity: 3 CS Business Value: 5 CS Frequency: 3 Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Owen Rodabaugh updated an issue Puppet / PUP-1974 Mechanism for securing sensitive catalog data Change By: Owen Rodabaugh CS Priority: Needs Priority Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Bill Niestempski updated an issue Puppet / PUP-1974 Mechanism for securing sensitive catalog data Change By: Bill Niestempski Labels: support tse Add Comment This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Eric Sorenson commented on PUP-1974 Re: Mechanism for securing sensitive catalog data There's a pretty simple precedent in the chef world that's simply a common attribute (err, 'metaparameter' in puppet-land) called 'sensitive', which prevents content logging: https://docs.chef.io/resource_common.html#attributes Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Chris Barker updated an issue Puppet / PUP-1974 Mechanism for securing sensitive catalog data Change By: Chris Barker Labels: tse Add Comment This message was sent by Atlassian JIRA (v6.3.7#6337-sha1:2ed701e) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Henrik Lindberg commented on an issue Re: Mechanism for securing sensitive catalog data I think it is reasonable to add requirements on the new Catalog Builder that it should be designed to be capable of handling encrypted attributes of resources. Add Comment Puppet / PUP-1974 Mechanism for securing sensitive catalog data Sensitive information such as passwords or key files contained within Puppet catalogs leaks into locations such as PuppetDB or syslog. This elevates the necessary security that must be enforced on these external systems. It would be valuable to give manifest/module authors the ability to specify resource properties (such as attributes or titles) which... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/opt
Jira (PUP-1974) Mechanism for securing sensitive catalog data
Title: Message Title Patrick Mooney created an issue Puppet / PUP-1974 Mechanism for securing sensitive catalog data Issue Type: Improvement Affects Versions: 3.x Assignee: Kylo Ginsberg Components: Types and Providers Created: 18/Mar/14 7:39 AM Priority: Normal Reporter: Patrick Mooney Sensitive information such as passwords or key files contained within Puppet catalogs leaks into locations such as PuppetDB or syslog. This elevates the necessary security that must be enforced on these external systems. It would be valuable to give manifest/module authors the ability to specify resource properties (such as attributes or titles) which are sensitive. Components downstream from the catalog compiler could then choose how to handle sensitive data. For example, the master could redact such fields from the catalog before sending it to PuppetDB. The agent could be configured to obscure sensitive resource titles from the log when they are acted upon. One possible way to do this would be the addition of a "sensitive" resource type that is compiled into the catalog. Each instance would specific resource fields to be selected and the preferred means of redaction.
