Awesome! Thanks Adrien. I've got some interesting plans for this if it works out and I appreciate you taking the time to figure out what's going on.
Trevor On Mon, Mar 28, 2016 at 6:36 PM, Adrien Thebo <[email protected]> wrote: > Hey Trevor, > > I've been able to reproduce your problem and I see what's going on. Since > this sort of post hoc catalog serialization isn't something that's done a > lot (or at all) there are some sharp edges; I'm going to head into this > code base with a big stick and I'm going to hit things really hard until > they work. It might take me a few days but when I've got things a bit more > sorted out I'll let you know what I've figured out. > > Adrien > > On Thu, Mar 24, 2016 at 6:48 AM Trevor Vaughan <[email protected]> > wrote: > >> Hey Adrian, >> >> The code is pretty simple right now and is as follows: >> >> ### Type >> >> module Puppet >> newtype(:catalog_test) do >> @doc = "Save a micro-catalog" >> >> newparam(:name) do >> isnamevar >> desc "Resource name to trigger off of" >> end >> >> newproperty(:resource_type) do >> desc "The resource type" >> end >> end >> end >> >> ### Provider >> >> Puppet::Type.type(:catalog_test).provide(:micro_catalog) do >> desc 'Save a micro-catalog' >> >> def resource_type >> my_catalog = Puppet::Resource::Catalog.new >> >> target_resource = @resource.catalog.resources.find{|x| >> (x.type == @resource[:resource_type].downcase.to_sym) && >> (x.name == @resource[:name]) >> } >> >> if target_resource >> >> @resource.catalog.relationship_graph.dependencies(target_resource).each do >> |res| >> my_catalog.add_resource(res) >> end >> >> my_catalog.add_resource(@resource) >> >> >> @resource.catalog.relationship_graph.dependents(target_resource).each do >> |res| >> my_catalog.add_resource(res) >> end >> end >> >> # Tinker Point >> require 'pry' >> binding.pry >> return @resource[:resource_type] >> end >> end >> >> ### Test File >> >> file { '/tmp' : ensure => 'directory' } >> >> file { '/tmp/foo1': content => 'blah' } >> >> file { '/tmp/stack': ensure => 'directory' } >> >> file { '/tmp/stack/test1': content => 'test' } >> >> file { '/usr/test': >> content => 'test', >> require => File['/tmp/stack'] >> } >> >> file { '/var/trigger': >> content => 'test', >> notify => File['/tmp/stack/test1'] >> } >> >> catalog_test { '/var/trigger': >> resource_type => 'file' >> } >> >> Thanks for the help, >> >> Trevor >> >> On Wed, Mar 23, 2016 at 2:43 PM, Adrien Thebo <[email protected]> >> wrote: >> >>> The exact issues that you may run into depend on how the catalog was >>> generated and what it's storing; in my scenario I'm using the RAL to >>> collect Puppet::Type instances, I'm converting them to Puppet::Resource >>> instances, and then inserting those into the catalog. Trying to >>> individually convert resources in the catalog might be messy because you'll >>> need to delete the type you're converting and then add the Resource >>> instance; it's doable but there are some logistics around that. >>> >>> Do you have code that you can provide to demonstrate what you're doing? >>> Given a RAL catalog you should be able to get a resource catalog in some >>> form, as for reactivating a catalog I'm not sure what you mean by >>> "reactivate". And lastly, the "resource catalog" is effectively the only >>> means of serializing a catalog right now; if you want to store a catalog >>> you'll almost certainly have to convert your data to a resource catalog, or >>> figure out how you can make Puppet::Type instances serializable. >>> >>> On Tue, Mar 22, 2016 at 8:43 AM Trevor Vaughan <[email protected]> >>> wrote: >>> >>>> Hey Adrian, >>>> >>>> Thanks for the response. I tried the >>>> Puppet::Resource::Catalog#to_resource method but ended up with an error on >>>> 'copy_as_resource' not found. >>>> >>>> I also tried the 'to_resource' method on all of the resource in my >>>> catalog individually but that didn't seem to get me anywhere either. >>>> >>>> Looking at the links that you posted, I suppose what I'm trying to do >>>> is actually save a RAL catalog and then reactivate it via the command line >>>> at a later point. Do you know if this is possible? I don't really need a >>>> compiler-level resource catalog since this won't be shipped off of the >>>> local system. >>>> >>>> Thanks, >>>> >>>> Trevor >>>> >>>> On Mon, Mar 21, 2016 at 11:16 AM, Adrien Thebo <[email protected]> >>>> wrote: >>>> >>>>> I've been working on a project with similar goals and ran into the >>>>> same thing. The core of the issue is that there are two catalogs, the >>>>> resource catalog and the RAL catalog. The resource catalog is what the >>>>> Puppet compiler emits and contains Puppet::Resource instances; this is >>>>> what's actually sent to the agent and what you'll see if you run `puppet >>>>> master --compile`. The RAL catalog is generated when the catalog is >>>>> actually being applied ( >>>>> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/configurer.rb#L106), >>>>> and converts Puppet::Resource instances to Puppet::Type instances. >>>>> Puppet::Resource instances are meant to be serialized, but Puppet::Type >>>>> instances aren't - I'm pretty sure that the error you're seeing is because >>>>> Puppet is trying to serialize Puppet::Type instances that don't have the >>>>> correct method defined. >>>>> >>>>> There is a document in the Puppet source ( >>>>> https://github.com/puppetlabs/puppet/blob/master/docs/catalogs.md) >>>>> that contains a bit more information about the different types of >>>>> catalogs. >>>>> >>>>> That being said there's a couple of ways of converting things around >>>>> to a more easily serializable format. >>>>> >>>>> One option is to individually convert Puppet::Type instances to >>>>> Puppet::Resource instances as you find them; I've been doing them with >>>>> something like this: >>>>> >>>>> Puppet::Type.type(:service).instances.map { |res| res.to_resource } >>>>> >>>>> Alternately, if you have an existing catalog and you want to convert >>>>> and serialize that wholesale, you should be able to use >>>>> `Puppet::Resource::Catalog#to_resource` ( >>>>> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/resource/catalog.rb#L490) >>>>> to do this. I haven't tested this out but it seems like it should work. >>>>> >>>>> There are a couple of caveats in this - for instance things get a >>>>> little bit screwy with some resource types such as files, but if you run >>>>> into issues with too many parameters being generated/not enough parameters >>>>> being generated, let me know. >>>>> >>>>> If you make progress in this area let me know; since I'm doing some >>>>> poking around in this area I would be interested in seeing what you find >>>>> out as well. >>>>> >>>>> >>>>> >>>>> On Sat, Mar 19, 2016 at 12:04 PM Trevor Vaughan < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> I'm doing some experiments with extracted catalog snippets but seem >>>>>> to be having issues on saving the new catalog after the fact. >>>>>> >>>>>> I have a Puppet::Resource::Catalog object that has content and >>>>>> relationships but when I attempt to save it using 'to_pson' I get an >>>>>> "undefined method `to_pson_data_hash`" for any given resource in the >>>>>> catalog. >>>>>> >>>>>> As an aside, if I just run @resource.catalog.to_pson from within a >>>>>> provider, I end up with the same error. >>>>>> >>>>>> Any help would be appreciated. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Trevor >>>>>> >>>>>> -- >>>>>> Trevor Vaughan >>>>>> Vice President, Onyx Point, Inc >>>>>> (410) 541-6699 >>>>>> >>>>>> -- This account not approved for unencrypted proprietary information >>>>>> -- >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Puppet Developers" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUnc589eJ3nujpF0G38%3D6hAHOMHVShU7xRU9KfzS%2BTjQg%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUnc589eJ3nujpF0G38%3D6hAHOMHVShU7xRU9KfzS%2BTjQg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>>> Adrien Thebo | Puppet Labs >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Puppet Developers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/puppet-dev/CALVJ9SKOK4HyZyOMYT%3DF31m7GxdL0jVvmYvKhzUAPBb5qhn0wg%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/puppet-dev/CALVJ9SKOK4HyZyOMYT%3DF31m7GxdL0jVvmYvKhzUAPBb5qhn0wg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>> >>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> Trevor Vaughan >>>> Vice President, Onyx Point, Inc >>>> (410) 541-6699 >>>> >>>> -- This account not approved for unencrypted proprietary information -- >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Puppet Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU7jp3SBnL2%3DOpa3abixQwPuUvWDgh6VEGRNntGjO8zsw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU7jp3SBnL2%3DOpa3abixQwPuUvWDgh6VEGRNntGjO8zsw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> Adrien Thebo | Puppet Labs >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJ6OjCnWG5TVqOMNFJF8Jc3kXvhGP9wgubg2CwET1Fy%3DA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJ6OjCnWG5TVqOMNFJF8Jc3kXvhGP9wgubg2CwET1Fy%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >> >> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Trevor Vaughan >> Vice President, Onyx Point, Inc >> (410) 541-6699 >> >> -- This account not approved for unencrypted proprietary information -- >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWBPxK5fSfB5CGFZWyWc_PN7htatdChp4S%3DbZ9PayEiSg%40mail.gmail.com >> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWBPxK5fSfB5CGFZWyWc_PN7htatdChp4S%3DbZ9PayEiSg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > Adrien Thebo | Puppet Labs > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJe2bkJT9W07kRO4YcOtPAm35BqMYFJVvWKxuVgQoq8xQ%40mail.gmail.com > <https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJe2bkJT9W07kRO4YcOtPAm35BqMYFJVvWKxuVgQoq8xQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Trevor Vaughan Vice President, Onyx Point, Inc (410) 541-6699 -- This account not approved for unencrypted proprietary information -- -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU%2B5V%3DSfJoSnjBJeuFL2oTm_eO3btTq0NV_wakA%2B0FDAA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
