I don't want to use sqllite... puppetqd doesn't leave me much of a choice since it will default to it even if you tell it to use mysql as your storedconfig DB since thats needed for the centraldb. Ultimately I think puppetqd needs support for keeping a cached copy of various database flavors (mysql, sqllite, etc). If mysql was supported for puppetqd then my remote masters would always have a local cached copy of their stored configs for the servers they manage and we would be able to eliminate the locking issue. They can then asyncronously replicate the data back to the central DB.
# enables storedconfig support, supposed to speed things up storeconfigs = true dbadapter = mysql dbuser = puppet dbpassword = puppet dbserver = centraldb dbsocket = /var/run/mysqld/mysqld.sock # use queuing queue_type = stomp queue_source = stomp://localhost:61613 async_storeconfigs = true On Wed, Jun 9, 2010 at 11:57 AM, Ohad Levy <ohadl...@gmail.com> wrote: > I evaluated this scenario a couple of years ago (before puppetq), and it > was never a feasible solution due to a. latency (e.g. everything is much > slower), b. if wan goes down, your master wont work. > > One of the reasons that I've developed Foreman, is to overcome this > problem, the way I do it is: > - I keep one central database, but don't use it directly with puppet, > rather import the data i need (e.g. for facts see the import_facts script in > foreman extras dir) > - I query the database using puppet functions (e.g. using foreman query > interface) - providing similar functionality to export / collection with > storeconfig. > > in order to avoid the single point of failure, my function caches the > output locally, so if the db (foreman) is not reachable, it uses the last > known cache. > in any case, sqlite will never really work with many processes trying to > lock the file (it is a file after all), so you should really evaluate it > with some other db engine. > > hope this helps, > Ohad > > On Wed, Jun 9, 2010 at 11:50 PM, Christopher Johnston > <chjoh...@gmail.com>wrote: > >> I have about a dozen sites with puppet masters in each. They are >> geographically located with some slow network links between them. The setup >> I was evaluating is to have a centralized stored config server running mysql >> with all of my remote masters sending their locations data back to the >> central server. Now on my fast links remote connectivity is OK if I just >> point a master right to the database (eliminate puppetqd, etc), albeit still >> slower then if I were to have it local but I was hoping I could use a >> stomp+puppetqd to handle caching and asyncronously update my central mysql >> database. In my tests this seems to work however it looks like puppetdq is >> using sqllite to keep a cached copy. If I have 100 servers and I am doing a >> mass update during maintenance window I am seeing timeouts once we hit 5-6 >> servers which is likely sqllite. >> >> Perhaps this solution is not the most viable right now? I am OK with >> running a local mysql instance instead of sqllite if it is supported but I >> would like to get all of my stored config data in one location (using facter >> data, reporting is of great value to us). This would be useful as we can >> plugin foreman on top of that database for a central dashboard instead of >> having dozens of dashboards for each location. >> >> -Chris >> >> >> On Tue, Jun 8, 2010 at 2:40 PM, donavan <dona...@desinc.net> wrote: >> >>> >>> On Jun 7, 1:00 pm, Christopher Johnston <chjoh...@gmail.com> wrote: >>> > Think I found the issue from an strace, the rubygems-sqlit3 rpm was >>> > missing.. still curious on where the logs end up for it though. The >>> fact >>> > that its using sqllite also doesn't help as I think I am running into >>> > locking issues if I try to run a cross environment update. >>> >>> >>> Chris, were you using sqlite or mysql as the backend store when you >>> had issues? >>> >>> Evaluating remote sites with central storeconfigs is on the todo list >>> for me. Currently have it working just peachy with mysql, straight >>> storeconfig no puppetqd. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Puppet Users" group. >>> To post to this group, send email to puppet-us...@googlegroups.com. >>> To unsubscribe from this group, send email to >>> puppet-users+unsubscr...@googlegroups.com<puppet-users%2bunsubscr...@googlegroups.com> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/puppet-users?hl=en. >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-us...@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscr...@googlegroups.com<puppet-users%2bunsubscr...@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-us...@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com<puppet-users%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
