On Saturday, March 31, 2018 at 5:59:12 AM UTC-5, nick....@countersight.co wrote: > > Thanks for your response John, > > I appreciate you taking a quick look around to see if anyone else has > already done this. I had come to the same conclusion, that if someone has > already, they mostly likely haven't shared it. > > You raise valid points about EL being generally pretty unsuitable as a > Hiera backend. However, the project I am working on already has an > Elasticsearch instance running in it, so there would be next to no > performance overhead for me. It uses a web interface to write out YAML > files that are fed into a Hiera for a 'puppet apply' run which configures > various aspects of the system. By using Elastic instead of YAML files, I > can eliminate some of the issues surrounding concurrent access, it also > means backups are simplified, as I'd just need to backup ES. >
With an ES instance already running, I agree that you have negligible additional *memory* overhead to consider, but that doesn't do anything about *performance* overhead. Nevertheless, the (speculative) performance impact is not necessarily big; you might well find it entirely tolerable, especially for the kind of usage you describe. It will depend in part on how, exactly, you implement the details. > Is writing a proof-of-concept Hiera backend something that someone with > reasonable coding skills be able to knock out in a few hours? > > It depends on what degree of integration you want to achieve. If you start with the existing YAML back end, and simply hack it to retrieve its target YAML objects from ES instead of from the file system, then yes, I think that could be done in a few hours. It would mean ES offering up relatively few, relatively large chunks of YAML, which I am supposing would be stored as whole objects in the database. I think that would meet your concurrency and backup objectives. If you want a deeper integration, such as having your back end performing individual key lookups in ES, then you might hack up an initial implementation in a few hours, but I would want a lot longer to test it out. I would want someone with detailed knowledge of Hiera and its capabilities to oversee the testing, too, or at least to review it. Even more so to whatever extent you have in mind to implement Hiera prioritization, merging behavior, interpolations, and / or other operations affecting what data Hiera presents to callers. If there is an actual budget for this then I believe Puppet, Inc. offers consulting services, or I'm sure you could find a third-party consultant if you prefer. John -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/0003d42c-d53c-4474-83d4-db4ae5dbb228%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
