On Wednesday, March 28, 2018 at 12:19:58 AM UTC-5, nick....@countersight.co wrote: > > Hi, > > I'd like to be able to store (and more importantly) retrieve Hiera hash > data from Elasticsearch for my Puppet runs. Does anyone know if this has > already been done? >
I haven't heard of such a project, and a bit of web searching didn't turn one up. If someone has made such a thing, then they don't seem to be saying much about it. > > Can you think of any particular reason why this might not work? > In principle, one ought to be able to build an Hiera back end that draws data from ES, just as there is one based on Postgres. I don't see any special barrier. At its most basic level, the data store aspect of an Hiera back end just needs to provide for looking up values by key, and ES is certainly capable of that. > From a layman's perspective, ES seems like an ideal place to be storing > this data. > > Why? Seriously, if there is more to that opinion than just "ES is a popular tool for searching and retrieving data" then that would be something we could talk about. In any case, ES's focus is on fast and flexible *search*, but Hiera's basic search needs are very simple -- just value lookup by exact key, possibly in multiple runtime-selected tables. Now, if you could actually implement Hiera-style priority lookups and its various merge behaviors directly in ES queries, then maybe you would be onto something, but I don't think that's possible, especially when you start considering how Hiera features such as interpolations affect that. Moreover, although ES is pretty fast as that's judged in its target application space, Hiera's use is, again, not really in that space. From an Hiera perspective, ES would have enormous memory overhead, and, I estimate, non-trivial performance overhead arising at minimum from IPC. Hiera's default YAML back end is tiny in comparison, and can run entirely inside the catalog builder process. That's by no means a reliable analysis of relative efficiency, but the point is that from my perspective, EL for an Hiera back end isn't something I would consider at all. If you want to put your data in a full-fledged DB then have Hiera go directly to that DB -- I see no advantage to putting ES between. With that said, however, I don't hear much about people doing that in practice. So anyway, was this just idle speculation on your part, or are you genuinely trying to design a Puppet infrastructure? If the latter, then do you have specific requirements that the default YAML back end does not meet? For otherwise, I'd certainly recommend starting there. John -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/01326772-6155-4b95-9944-534d78cee3bd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
