Re: [Puppet-dev] Re: [Puppet Users] Plugins in modules with environments and the puppetmaster libdir
On Feb 9, 2010, at 6:55 AM, Nigel Kersten wrote: On Mon, Feb 8, 2010 at 2:45 PM, Luke Kanies wrote: On Feb 5, 2010, at 8:47 AM, Nigel Kersten wrote: On Fri, Feb 5, 2010 at 3:29 AM, Thomas Bellman wrote: Nigel Kersten wrote: So facter plugins are kind of different, as they're not actually required to be in the puppetmaster libdir. Say this was a type/provider, and you wanted to add a new parameter, but only roll it out to your testing environments. Functions also have this limitation, by the way. What do you do then? If the version in the puppetmaster libdir doesn't accept that parameter, the manifest compilation will fail for clients who *are* getting a version of the plugin that supports that parameter. You lose... Well, there are a couple of things you can do to work around this limitation. For one thing, you could run a second puppetmaster process, perhaps on another port, or listening on another IP address, or perhaps even on another machine. Another thing you can do, is to run puppet with local manifests, instead of puppetd connecting to puppetmasterd, when developing the new version of the plugin. That's what I do. It does get a bit iffy if your manifests want to fetch files from the puppetmaster (i.e. that aren't in the "modules" namespace) though, or otherwise need to access files that are only available on the puppetmaster. Good news, though, is that the upcoming "Rowlf" version of Puppet is supposed to solve this for at least resource types. At least Luke has posted patches to the devel list that I gather will do that. But it will probably still be a problem for custom functions (I have somewhat volunteered to take a look at it, but I'm unlikely to find the time before Rowlf is supposed to be out). + puppet-dev Luke, how is this going to be solved in practice? The puppetmasterd will automatically add modules/*/lib dirs to it's own libdir in the context/environment of the current client? Hmm, not quite true yet. At this point, rowlf has a bunch of refactoring around just the language stuff, not the pure-ruby stuff. We're on the path to converging the language resource types and the pure ruby Puppet::Type classes, but we're still a ways away. Part of that convergence will be fixing this problem, but I don't expect to see a real long-term fix until at least the release after rowlf - mostly just because we don't want to delay rowlf too much further. Can you give us an overview of the way this is intended to work? Not sure exactly how to answer this question, since there's a lot there. The convergence of the two classes (Puppet::Type and Puppet::Resource::Type) is complex to describe. Basically, current resource types are subclasses of Puppet::Type, and resources are instances of those subclasses. We'll be changing things so that resource types are instances of Puppet::Resource::Type, and resources are instances of Puppet::Resource. In terms of observable differences, other than the constants involved there won't really be any - it'll just make contribution, maintenance, and custom development much easier. However, it'll also enable us to do a lot of things we currently can't, like maintain non-global (i.e., environment-specific) resource types. The refactoring I did to enable the pure-ruby DSL was the main step toward this: When you use 'define' in the language, you're creating an environment-specific resource type, it's just not as functional as a builtin resource type. I think the next major release will normalize the functionality between the two, and hopefully also make it possible to use existing providers with new-style resource types. Then it's just a question of porting over the existing types to use the new class structure. I'm thinking that in the meantime I may need to encode plugin versions in their names, so when a client's manifest contains a given plugin, it's always going to refer to that version, and I script something that collates all the lib/... directories into the puppetmasterd libdir. I feel dirty. Yeah, that's ugly. I think it'd be way easier to run environment- specific masters on a different port or server, wouldn't it? Not for the number of environments we have... Ah. BTW, if this is a big priority to someone, I think it's approachable today - far more than six months ago, with recent refactoring - and I'm happy to work with someone who's committed to getting it fixed. And I know this has been coming up a lot recently, so it's getting pushed up on my priority list. What would you need from those of us who are interested in working on this with you Luke? Code? :) -- Finn's Law: Uncertainty is the final test of innovation. - Luke Kanies -|- http://reductivelabs.com -|- +1(615)594-8199 -- You received this message because you are subscribed to the Google Groups "Puppet Users
Re: [Puppet-dev] Re: [Puppet Users] Plugins in modules with environments and the puppetmaster libdir
On Mon, Feb 8, 2010 at 2:45 PM, Luke Kanies wrote: > On Feb 5, 2010, at 8:47 AM, Nigel Kersten wrote: > > On Fri, Feb 5, 2010 at 3:29 AM, Thomas Bellman >> wrote: >> >>> Nigel Kersten wrote: >>> >>> So facter plugins are kind of different, as they're not actually required to be in the puppetmaster libdir. Say this was a type/provider, and you wanted to add a new parameter, but only roll it out to your testing environments. >>> >>> Functions also have this limitation, by the way. >>> >>> What do you do then? If the version in the puppetmaster libdir doesn't accept that parameter, the manifest compilation will fail for clients who *are* getting a version of the plugin that supports that parameter. >>> >>> You lose... >>> >>> Well, there are a couple of things you can do to work around this >>> limitation. For one thing, you could run a second puppetmaster >>> process, perhaps on another port, or listening on another IP address, >>> or perhaps even on another machine. >>> >>> Another thing you can do, is to run puppet with local manifests, >>> instead of puppetd connecting to puppetmasterd, when developing >>> the new version of the plugin. That's what I do. It does get a >>> bit iffy if your manifests want to fetch files from the puppetmaster >>> (i.e. that aren't in the "modules" namespace) though, or otherwise >>> need to access files that are only available on the puppetmaster. >>> >>> >>> Good news, though, is that the upcoming "Rowlf" version of Puppet is >>> supposed to solve this for at least resource types. At least Luke has >>> posted patches to the devel list that I gather will do that. But it >>> will probably still be a problem for custom functions (I have somewhat >>> volunteered to take a look at it, but I'm unlikely to find the time >>> before Rowlf is supposed to be out). >>> >> >> + puppet-dev >> >> Luke, how is this going to be solved in practice? The puppetmasterd >> will automatically add modules/*/lib dirs to it's own libdir in the >> context/environment of the current client? >> > > Hmm, not quite true yet. > > At this point, rowlf has a bunch of refactoring around just the language > stuff, not the pure-ruby stuff. > > We're on the path to converging the language resource types and the pure > ruby Puppet::Type classes, but we're still a ways away. Part of that > convergence will be fixing this problem, but I don't expect to see a real > long-term fix until at least the release after rowlf - mostly just because > we don't want to delay rowlf too much further. Can you give us an overview of the way this is intended to work? > > > I'm thinking that in the meantime I may need to encode plugin versions >> in their names, so when a client's manifest contains a given plugin, >> it's always going to refer to that version, and I script something >> that collates all the lib/... directories into the puppetmasterd >> libdir. >> >> I feel dirty. >> > > Yeah, that's ugly. I think it'd be way easier to run environment-specific > masters on a different port or server, wouldn't it? > Not for the number of environments we have... > > BTW, if this is a big priority to someone, I think it's approachable today > - far more than six months ago, with recent refactoring - and I'm happy to > work with someone who's committed to getting it fixed. > > And I know this has been coming up a lot recently, so it's getting pushed > up on my priority list. > What would you need from those of us who are interested in working on this with you Luke? > > -- > Men never do evil so completely and cheerfully as when they do it from a > religious conviction. --Blaise Pascal > - > Luke Kanies -|- http://reductivelabs.com -|- +1(615)594-8199 > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To post to this group, send email to puppet-...@googlegroups.com. > To unsubscribe from this group, send email to > puppet-dev+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/puppet-dev?hl=en. > > -- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet-dev] Re: [Puppet Users] Plugins in modules with environments and the puppetmaster libdir
On Feb 5, 2010, at 8:47 AM, Nigel Kersten wrote: On Fri, Feb 5, 2010 at 3:29 AM, Thomas Bellman wrote: Nigel Kersten wrote: So facter plugins are kind of different, as they're not actually required to be in the puppetmaster libdir. Say this was a type/provider, and you wanted to add a new parameter, but only roll it out to your testing environments. Functions also have this limitation, by the way. What do you do then? If the version in the puppetmaster libdir doesn't accept that parameter, the manifest compilation will fail for clients who *are* getting a version of the plugin that supports that parameter. You lose... Well, there are a couple of things you can do to work around this limitation. For one thing, you could run a second puppetmaster process, perhaps on another port, or listening on another IP address, or perhaps even on another machine. Another thing you can do, is to run puppet with local manifests, instead of puppetd connecting to puppetmasterd, when developing the new version of the plugin. That's what I do. It does get a bit iffy if your manifests want to fetch files from the puppetmaster (i.e. that aren't in the "modules" namespace) though, or otherwise need to access files that are only available on the puppetmaster. Good news, though, is that the upcoming "Rowlf" version of Puppet is supposed to solve this for at least resource types. At least Luke has posted patches to the devel list that I gather will do that. But it will probably still be a problem for custom functions (I have somewhat volunteered to take a look at it, but I'm unlikely to find the time before Rowlf is supposed to be out). + puppet-dev Luke, how is this going to be solved in practice? The puppetmasterd will automatically add modules/*/lib dirs to it's own libdir in the context/environment of the current client? Hmm, not quite true yet. At this point, rowlf has a bunch of refactoring around just the language stuff, not the pure-ruby stuff. We're on the path to converging the language resource types and the pure ruby Puppet::Type classes, but we're still a ways away. Part of that convergence will be fixing this problem, but I don't expect to see a real long-term fix until at least the release after rowlf - mostly just because we don't want to delay rowlf too much further. I'm thinking that in the meantime I may need to encode plugin versions in their names, so when a client's manifest contains a given plugin, it's always going to refer to that version, and I script something that collates all the lib/... directories into the puppetmasterd libdir. I feel dirty. Yeah, that's ugly. I think it'd be way easier to run environment- specific masters on a different port or server, wouldn't it? BTW, if this is a big priority to someone, I think it's approachable today - far more than six months ago, with recent refactoring - and I'm happy to work with someone who's committed to getting it fixed. And I know this has been coming up a lot recently, so it's getting pushed up on my priority list. -- Men never do evil so completely and cheerfully as when they do it from a religious conviction. --Blaise Pascal - Luke Kanies -|- http://reductivelabs.com -|- +1(615)594-8199 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
