Thanks for the response.. My reply inline... On 26 May 2017 at 17:32, Thomas Gelf <[email protected]> wrote:
> Hi, > > Director/PuppetDB module author here - so I might be biased ;-) > > Am 26.05.2017 um 13:19 schrieb Jayapandian Ponraj: > >> * setup 1: I see that the import of all nodes from puppetdb (1K+) in our >> setup takes a huge amount of time and works only when php is given >> insane amount of memory(1G+). Is this normal ? has anyone tried it? >> > > 1000 nodes isn't much, but it is expensive to query the PuppetDB for all > those details. It imports all facts and all assigned classes. One of > those used to be very slow, I don't remember which one. Eventually the > API query might be improved to benefit from new features in newer > PuppetDB versions - not sure about this. > > I used to keep the "Nodes" import running long time ago for a while > against a production environment. As far as I remember, Import took > nearly 15 minutes all the times, and most time was spent on waiting for > PuppetDB. > > This doesn't hurt Director at all as it usually runs as a background Job > and not through the web frontend. We didn't experience such with a DB > sized similar than your's, but of course this might put some burden on > your PuppetDB. So I would suggest to not use the "Nodes" type of import, > that's more kind of a playground. I do understand that the initial loading can be huge but i have seen director get unresponsive even after the inital load, like when defining sync properties. I would like to have atleast the basic hostalive check across all boxes. In order to achieve this do we have a way to just import nodes without facts and classes. In a way the feature am asking for is filtering at source, which I guess is not available for puppetdb. > > > * setup 2: I tried collecting the exported Host objects from puppetdb >> via director. In this case the import and sync is smooth but am unable >> to FILTER. Is there any documentation listing out the keys available to >> filter on? probably a few examples can help to get started >> > > That's the way to go, I'm also doing so. A customer I work with from > time to time is running it that way, exporting 2,5k hosts with quite > some properties and custom vars (mostly from facts) and additionally > exporting 10-20k single services the same way. Other 60k servives are > then generated via Apply Rules. > > I do not suggest to filter at Sync time, that's a leftover for > compatibility reasons from very old Director versions. Sure, it works - > and the setup mentioned above is doing so. Just, some minor things might > not work as you expect. Like tweaking the filter to get rid of some > hosts would have no influence on the "purge" mechanism. Can you give some examples of what works, like which fields can be used for filters in a typical icinga::object::host object. A simple example will be greatly helpfull. I have tried various filters using "name", "display_name", "title" etc. > > Instead, having filters at the source is the way to go. This is easy for > Import Sources like SQL/LDAP and similar, for PuppetDB this hasn't been > implemented at the time being, as the initial thought was that in a > perfect world it's Puppet that should filter objects by deciding whether > to export them or not. So, to stop monitoring a host or a service, stop > exporting it to PuppetDB and it would vanish. > In our setup we have multiple icinga masters and export host objects with various puppet tags https://docs.puppet.com/puppet/4.10/lang_tags.html We would like to collect host objects based on the tags.. Thats how our setup is right now hence moving to director is an issue without filters.. > > Our world isn't perfect, and people often show up with the desire to > tweak things in Director, as a change in Puppet might take quite some > time - depending on your environment. This would be worth a dedicated > feature request, when creating such please also add some concrete > examples of what you want to achieve and how. That would help to get a > better picture of how your very own perfect world would look like ;-) > > * I see that the preview when using puppetdb import ain't that great. Is >> there any improvement planned in this regard? >> > > Not yet. It should however be more or less usable when working with > exported resources and a defined set of properties. > > Cheers, > Thomas > > NB: In case you're concerned about memory usage, please consider moving > to PHP 7.x. The difference to 5.x when it comes to memory and > performance is impressive. > > -- > Thomas Gelf > Principal Consultant > > NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg > Tel: +49 911 92885-0 | Fax: +49 911 92885-77 > CEO: Julian Hein, Bernd Erk | AG Nuernberg HRB18461 > http://www.netways.de | [email protected] > > ** OSMC 2017 - November - osmc.de ** > ** OSBconf 2017 - September - osbconf.org ** > > _______________________________________________ > icinga-users mailing list > [email protected] > https://lists.icinga.org/mailman/listinfo/icinga-users >
_______________________________________________ icinga-users mailing list [email protected] https://lists.icinga.org/mailman/listinfo/icinga-users
