Hi Trevor,
I started working on the flush part. I have a dev branch (which doesn't pass all tests just now) at https://github.com/raphink/augeasproviders/tree/dev/aug_one_handler Feel free to test it and let me know if that improves your situation with the number of files opened by Augeasproviders. On Monday, February 10, 2014 3:45:02 PM UTC+1, Trevor Vaughan wrote: > > Hi Raphaël, > > 1) I'm a fan of the first bullet. Just having everything run through flush > would be a huge bonus. > > 2) Good to hear! > > 3) Yeah, I definitely know that it depends on the lens. Unfortunately, > I've just had too many issues over time with lenses missing various > features *or* being too loose. > > As far as I'm concerned, I consider Augeas to be a parser before anything else. So I care that my lenses parse as much as possible, and produce valid files. Nowadays, I'd only make lenses strict if there is a need to strictly identify parameters and their entry types. > 4) Unfortunately, I'm referring both to mapping the properties and to the > Augueas language itself. The learning curve is high on creating new custom > types and the learning curve is even higher an creating new Augeas lenses. > Each time I've wanted to create an Augeas lens I ended up frustrated with > the error handling and ease of testing and just wrote it in Ruby because it > was faster. I 100% love the idea, I just find it frustrating in reality. > I understand your frustration. Augeas comes with many lenses already, so most Augeas providers won't need a new lens. On the positive side, I'm planning on organizing a full-fledge Augeas course around Puppetconf (4 days, including Augeas basics, writing facts, functions and providers with the Augeas ruby lib, as well as writing new lenses). Cheers, Raphaël > On Sun, Feb 9, 2014 at 6:09 PM, Raphaël Pinson <[email protected]<javascript:> > > wrote: > >> >> Hi Trevor, >> >> 1) there is an issue open at >> https://github.com/hercules-team/augeasproviders/issues/67. I've been >> thinking about how to fix this and have come up with a few ideas. >> Currently, a new Augeas handler is spawned for every call to augopen, which >> means the file is opened for exists?, create, destroy, and for each >> property getter or setter. That explains the huge amount of file openings >> (and possibly writings, too) you can get with these providers. Solutions >> include: >> >> - using one augeas handler per resource, reusing it for all methods, and >> possibly using flush to save. This is probably a good idea anyway as it >> will avoid applying some properties when others may fail. >> - using one augeas handler for all resources. While this would >> drastically reduce the amount of times files are opened, it is most likely >> a bad idea as this means a single resource breaking the Augeas tree will >> cause all resources to fail saving. >> - "prefetching" all resources from a given target file when a resource >> first requests it. I'm not really sure how that would be implemented, but >> it would at least reduce the amount of file reading. Coupling this with a >> flush method per resource would lead to a nice solution imo. >> >> 2) Augeas providers already report when files fail to parse. The Augeas >> tree clearly indicates which line and character failed to parse, so this >> can easily be improved. >> >> 3) that is a tricky one. As you said, it mostly depends on the lens >> itself. Some lenses are very strict (with all known parameters explicitly >> listed) while others are quite loose. It is often up to the lens authors to >> decide on their policy, but some formats also don't leave much of a choice >> (because of entry types, typically strings vs lists) >> >> 4) I suppose you're referring to mapping properties to the Augeas API? >> >> >> All this said, while Augeas now has most of the lenses required to >> replace all parsedfile providers in the Puppet core, I would still be in >> favor of keeping some kind of parsedfile (or equivalent, such as Adrien's >> new implementation) mechanism in the core for cases that Augeas might have >> a hard time addressing (or devs that would rather not use it). >> >> >> Cheers, >> >> Raphaël Pinson >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/c386ea90-6628-4627-8950-26060bc9eedf%40googlegroups.com >> . >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > > -- > Trevor Vaughan > Vice President, Onyx Point, Inc > (410) 541-6699 > [email protected] <javascript:> > > -- This account not approved for unencrypted proprietary information -- > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/3c4644d5-caee-4bd3-a054-0e198df7ce25%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
