On Wed, 2010-09-15 at 16:26 -0700, Luke Kanies wrote: > Hi all, > > I've just stuck my proposal for a Catalog Service (which we've been > bandying about internally for a while, and which I've been thinking > about even longer) on the wiki: > > http://projects.puppetlabs.com/projects/puppet/wiki/CatalogServiceArchitecture > > Comments appreciated, and ideas for how you would use it appreciated > even more.
I need to read it again, but the envisioned change looks really interesting. I like this move :) There's something interesting about node dependencies. I understand how all the nodes catalogs form a graph, with one node catalog being a subgraph. However, is there a plan to extend the puppet/ruby DSL to express/enforce such dependencies? I'm also wondering what would happen when serving a catalog of node A which requires some resources from node B which hasn't been yet compiled. Compiling node B might not be an option because we need those node facts (BTW, how do the facts fit in this system?). It would be great if your document could more clearly explain the various possible architectures (ie combined master/catalog service, multiple master to one catalog service, decentralized catalog services if possible...). Also it seems you imply it is needed to run both a master and a catalog service as separate processes (even on the same host), which I thought wasn't necessary (ie indirect directly to the terminus instead of :rest). Regarding queries: I'm not sure I like the second proposed query system where you separate tags and parameters. I prefer the current system where parameters are in the "top namespace" like tags. I remember we also support OR which are not in your proposal. I personally don't use any ORs, and I can understand how it can make everything complex... Regarding back-ends: I have lots of doubts about the performance of the text-file solution, though. One of the nice things of using a RDBMS is that you can use one or another quite independently. Once we'll chose a specific NoSQL or a graph database, I'm afraid we'll have to live a long time with it. I'm not talking about implementation, but actual use of the datastore, which might prove to be painful, if for some reason we find it doesn't work as advertised and we need to switch to the new kid on the block. If possible, I think your document should better enforce the fact back-ends will be plugins (ie abstracted), so that we could let the user chose among RDBMS or NoSQL (provided those plugins exists of course). And finally, you're talking about migration tool. I don't really see how it will be possible to migrate from the current storeconfigs back-end to a newer system since you told us that one of the issue of storeconfigs was it was lacking some necessary information. The only way I would see a migration is: wait all your nodes ask for a catalog, then switch off storeconfigs :) About storing all the revisions of a given catalog, one of the issue would be that you need to implement in the service an option to retrieve one specific version or the list of all versions. This can be easy to do with couchdb because it's how it works, but other NoSQL solutions might not be able to provide an easy way to do this. Of course this is an interesting and powerful feature for auditing changes (especially if there is a corresponding audit tool). Oh and I learnt a new word: paucity. Not sure I'll be able to use it in conversations, but thanks anyway :) -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
