Thanks for that John, If I go down this road, I'll post any code that I produce on GitHub and see if anyone else is interested in trying/testing it.
Cheers, Nick On Tuesday, April 3, 2018 at 12:47:37 AM UTC+10, John Bollinger wrote: > > > > On Saturday, March 31, 2018 at 5:59:12 AM UTC-5, [email protected] > wrote: >> >> Thanks for your response John, >> >> I appreciate you taking a quick look around to see if anyone else has >> already done this. I had come to the same conclusion, that if someone has >> already, they mostly likely haven't shared it. >> >> You raise valid points about EL being generally pretty unsuitable as a >> Hiera backend. However, the project I am working on already has an >> Elasticsearch instance running in it, so there would be next to no >> performance overhead for me. It uses a web interface to write out YAML >> files that are fed into a Hiera for a 'puppet apply' run which configures >> various aspects of the system. By using Elastic instead of YAML files, I >> can eliminate some of the issues surrounding concurrent access, it also >> means backups are simplified, as I'd just need to backup ES. >> > > > With an ES instance already running, I agree that you have negligible > additional *memory* overhead to consider, but that doesn't do anything > about *performance* overhead. Nevertheless, the (speculative) > performance impact is not necessarily big; you might well find it entirely > tolerable, especially for the kind of usage you describe. It will depend > in part on how, exactly, you implement the details. > > >> Is writing a proof-of-concept Hiera backend something that someone with >> reasonable coding skills be able to knock out in a few hours? >> >> > It depends on what degree of integration you want to achieve. If you > start with the existing YAML back end, and simply hack it to retrieve its > target YAML objects from ES instead of from the file system, then yes, I > think that could be done in a few hours. It would mean ES offering up > relatively few, relatively large chunks of YAML, which I am supposing would > be stored as whole objects in the database. I think that would meet your > concurrency and backup objectives. > > If you want a deeper integration, such as having your back end performing > individual key lookups in ES, then you might hack up an initial > implementation in a few hours, but I would want a lot longer to test it > out. I would want someone with detailed knowledge of Hiera and its > capabilities to oversee the testing, too, or at least to review it. Even > more so to whatever extent you have in mind to implement Hiera > prioritization, merging behavior, interpolations, and / or other operations > affecting what data Hiera presents to callers. If there is an actual > budget for this then I believe Puppet, Inc. offers consulting services, or > I'm sure you could find a third-party consultant if you prefer. > > > John > > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/ed36ac7d-8e7a-41e5-b961-ab1c430f3637%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
