Thanks for that John, If I go down this road, I'll post any code that I produce on GitHub and see if anyone else is interested in trying/testing it.
Cheers, Nick On Tuesday, April 3, 2018 at 12:47:37 AM UTC+10, John Bollinger wrote: > > > > On Saturday, March 31, 2018 at 5:59:12 AM UTC-5, nick....@countersight.co > wrote: >> >> Thanks for your response John, >> >> I appreciate you taking a quick look around to see if anyone else has >> already done this. I had come to the same conclusion, that if someone has >> already, they mostly likely haven't shared it. >> >> You raise valid points about EL being generally pretty unsuitable as a >> Hiera backend. However, the project I am working on already has an >> Elasticsearch instance running in it, so there would be next to no >> performance overhead for me. It uses a web interface to write out YAML >> files that are fed into a Hiera for a 'puppet apply' run which configures >> various aspects of the system. By using Elastic instead of YAML files, I >> can eliminate some of the issues surrounding concurrent access, it also >> means backups are simplified, as I'd just need to backup ES. >> > > > With an ES instance already running, I agree that you have negligible > additional *memory* overhead to consider, but that doesn't do anything > about *performance* overhead. Nevertheless, the (speculative) > performance impact is not necessarily big; you might well find it entirely > tolerable, especially for the kind of usage you describe. It will depend > in part on how, exactly, you implement the details. > > >> Is writing a proof-of-concept Hiera backend something that someone with >> reasonable coding skills be able to knock out in a few hours? >> >> > It depends on what degree of integration you want to achieve. If you > start with the existing YAML back end, and simply hack it to retrieve its > target YAML objects from ES instead of from the file system, then yes, I > think that could be done in a few hours. It would mean ES offering up > relatively few, relatively large chunks of YAML, which I am supposing would > be stored as whole objects in the database. I think that would meet your > concurrency and backup objectives. > > If you want a deeper integration, such as having your back end performing > individual key lookups in ES, then you might hack up an initial > implementation in a few hours, but I would want a lot longer to test it > out. I would want someone with detailed knowledge of Hiera and its > capabilities to oversee the testing, too, or at least to review it. Even > more so to whatever extent you have in mind to implement Hiera > prioritization, merging behavior, interpolations, and / or other operations > affecting what data Hiera presents to callers. If there is an actual > budget for this then I believe Puppet, Inc. offers consulting services, or > I'm sure you could find a third-party consultant if you prefer. > > > John > > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/ed36ac7d-8e7a-41e5-b961-ab1c430f3637%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
