Re: [Puppet Users] Using puppet to automate tasks (ex: mysql slave setup)
On Mar 10, 2010, at 5:18 AM, Doug Warner wrote:
I'm not sure if puppet can be coerced to do something like this, but
I wanted
to throw it out there as it's a process that seems fairly easy to
automate.
To create new mysql slaves my process goes something like this:
1) run innobackupex [1] on mysql master
2a) copy data from master to slave (takes a decent amount of time
depending on
wire speed and database size; usually on the order of 6-24 hrs)
2b) setup mysql slave w/ data from mysql master to start replicating
binary log
3) import backup using innobackupex
The problem here seems to be that I need to do things on two
different hosts
and only once certain things have happened.
I figured I could easily specify the node that is the master and the
slave in
the manifests, but I'm not sure how far this gets me. I imagine
through the
creative use of exec's and "onlyif"s this should be doable, but I
wanted to
get other people's experiences with automating processes like this.
Hi Doug,
I, um, don't know if this is a good idea, but this might work for you:
http://github.com/reductivelabs/puppet-external-resource
Basically, I've created a simple resource that functions as a kind of
gate for exactly that situation. I've tested it in very simplistic
situations, but you should consider this to be experimental code and
by no means a complete solution. I just made this repo public five
minutes ago and wrote it for a customer last week.
For long delays -- 12 hours -- this will likely be pretty spammy,
because you'll get a lot of 'skipped because of failed dependencies'
reports, and you'll likely want to run it with a timeout of 0 and
verify that it works correctly in that case.
With this, your slave would have something like:
class dbslave {
external_resource { db-backup: check => "..script to verify backup
is complete..." }
db::import { mydb: , require => External_resource[db-backup] }
}
By default, it'll run the external resource check every 10 seconds,
and if it doesn't complete within five minutes then it fails, causing
all dependent resources (in this case, your import) to be skipped.
You can tune both the frequency and timeout, but I haven't tested it
with a timeout of 0; it might just sleep infinitely in that case.
--
There is only one success - to be able to spend your life in your own
way. -- Christopher Morley
-
Luke Kanies -|- http://reductivelabs.com -|- +1(615)594-8199
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
RE: [Puppet Users] Using puppet to automate tasks (ex: mysql slave setup)
In my environment I'd probably do something like this with Func (https://fedorahosted.org/func/). It's primarily used with RedHat based systems. If that's not a problem for you, then you can write scripts on your "overlord" that run commands on any systems running the func daemon and reporting to your "overlord". Your specific issue could be solved mostly with the "Command" and "CopyFile" module or, with a little Python experience, you could write your own module using the available api. It takes some work getting Func implemented but can be done quite easily with an existing puppet infrastructure or with something like Cobbler. byron -Original Message- From: puppet-users@googlegroups.com [mailto:puppet-us...@googlegroups.com] On Behalf Of Doug Warner Sent: Wednesday, March 10, 2010 8:19 AM To: puppet-users@googlegroups.com Subject: [Puppet Users] Using puppet to automate tasks (ex: mysql slave setup) I'm not sure if puppet can be coerced to do something like this, but I wanted to throw it out there as it's a process that seems fairly easy to automate. To create new mysql slaves my process goes something like this: 1) run innobackupex [1] on mysql master 2a) copy data from master to slave (takes a decent amount of time depending on wire speed and database size; usually on the order of 6-24 hrs) 2b) setup mysql slave w/ data from mysql master to start replicating binary log 3) import backup using innobackupex The problem here seems to be that I need to do things on two different hosts and only once certain things have happened. I figured I could easily specify the node that is the master and the slave in the manifests, but I'm not sure how far this gets me. I imagine through the creative use of exec's and "onlyif"s this should be doable, but I wanted to get other people's experiences with automating processes like this. -Doug [1] http://www.percona.com/docs/wiki/percona-xtrabackup:xtrabackup_manual#setting_up_a_new_slave_from_a_backup_in_replication -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Using puppet to automate tasks (ex: mysql slave setup)
I'm not sure if puppet can be coerced to do something like this, but I wanted to throw it out there as it's a process that seems fairly easy to automate. To create new mysql slaves my process goes something like this: 1) run innobackupex [1] on mysql master 2a) copy data from master to slave (takes a decent amount of time depending on wire speed and database size; usually on the order of 6-24 hrs) 2b) setup mysql slave w/ data from mysql master to start replicating binary log 3) import backup using innobackupex The problem here seems to be that I need to do things on two different hosts and only once certain things have happened. I figured I could easily specify the node that is the master and the slave in the manifests, but I'm not sure how far this gets me. I imagine through the creative use of exec's and "onlyif"s this should be doable, but I wanted to get other people's experiences with automating processes like this. -Doug [1] http://www.percona.com/docs/wiki/percona-xtrabackup:xtrabackup_manual#setting_up_a_new_slave_from_a_backup_in_replication signature.asc Description: OpenPGP digital signature
