> > > > Besides, I dont think the resource level abstraction in exported resources > > works well with my way of thinking so I just wont use it :P > > Hrm, well, hopefully I can change your mind on this at some point, and > > if not, hopefully we can find a more structured way of sharing data > that will work for both of us. > > I hate the 'query data from db' model, because you lose all dependency > > information - if you use exported resources, we could draw a graph of > the entire network and give you dependency information across the > whole thing, but if you're just pulling data from a database, then you > have no idea who's using the data. > > Do you have a model for these queries that can provide that kind of > dependency information?
I agree dependency info is valuable and there are times you need it. However, in this case it just serves no purpose to me, take for example my mysql servers remembering I am a managed hosting provider and stuff is this complex due to customer demand and flexibility of my offering: - I support: a mysql master. a mysql master with slaves (many), optional a lagging mysql slave. All with optional ssl on the slaves - I back them up: daily, hourly, weekly - depends what you pay me for - I store the backups: weekly, monthly, 3 monthly - depends what you pay me for - I direct the backups centrally but store data regionally - closest storage node to the mysql box - Clients can have many applications some needing ips, some needing hostnames - Clients MOTDs tell them whats their master db host for a specific environment - Monitoring and graphing needs to know where to find DBs. I operate on a regional service center approach. Countries uk and za are served by machines in the UK. Served meaning monitored, graphed, backedup, activemq endpoints etc - Load balancers need to know all the slaves for a client per environment - The Mysql Master DB needs to be able to generate IPTables rules allowing all its slaves to talk to it, as well as all the clients web servers, app servers etc So I need to know mysql master/slave/combos ip and/or hostnames in these places: - all the mysql slaves must know their master and if its SSL, if its SSL need to know the hostname and the ip - random applications like wordpress, custom code, php/ruby/etc apps all need to know the ip/hostname of the master. some who have split setups need to know the slaves too. this data goes into wide range of formats of file - in some cases this might be in the form: pool = db1, db2, db2 - backups need to know where/what to backup. Where the machines are physically, be able to figure out the nearest storage node to these machines. And be able to capture all the crazy policies and retention policies, frequencies etc i support. - lists of ips, ranges etc for iptables Now some of this you can satisfy with exported resources, mostly you wouldnt be able to do this with exported resources - the mysql module has no idea how many or what applications are going to need to talk to it, how can it export its ip/host in a sane manner? Backups, in a simple setup you could have mysql::master::backup just export a backup resource, in my case how could it? the amount of knowlesge related to how i do bakups, decisions that needs to be made etc far exceeds the abilities of the puppet DSL. It gets done in templates on the backup machines. And specifically the ways I backup machines with the module mysql::master vary widely, that module could not export how to back it up. So the approach today is to have some kind of client setup class or extlookup data source for the client and you hardcode all this stuff and if anything changes then you update this hardcoding. I want to be able to answer a lot of the above using queries and not hardcoding: - for customer=acme i want an array of hostnames for machines with class mysql::slave in environment=production - for customer=acme I want all mysql::master machines, i want to know what country they are in and then decide based on that which backup storage node they should talk to. Oh and I might just do a query to figure out what my storage nodes are in country=xx rather than hardcode that - in random php applications config file template I just want to know mysql masters for customer=acme, environment=production, class mysql::master present - on all customer machines I want to put a snippet of text into MOTD, just figure out whats the mysql::master for customer=acme, environment=production (this could be done with exported resources easily too) etc There's a huge use case for just being able to query key bits of information about an architecture. This is just mysql, I can show you >10 other places in my environment that's equally complex, equally unsuited for exported resources and that currently requires data in extlookup (a different type of hardcoding). I am seeking to stream line this and make it all more self organizing and less about needing knowledge of the physical make up of my machines. I let customers bring any machine in any dc in any country to me and I'll manage it. So I end up with kit in LOTS of data centers and the picture of what machines are where and what backs them up, monitors them etc can be insane. One client can have machines in 3 countries, its very confusing. You need to think past artificial borders and traditional approaches of 'i have a backup machine and i can export a single backup resource' to i have regional service centers and machines in countries x y and z belongs to this region and so needs to be allow certain connections for monitoring, backups etc. matching that knowledge. Similarly the service center need to be able to discover all the machines its responsible for. -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
