Re: [gentoo-user] Cross system dependencies
On Sat, 28 Jun 2014 11:36:11 +0200, J. Roeleveld wrote: > I need a way to add dependencies to services which are provided by > different servers. For instance, my mail server uses DNS to locate my > LDAP server which contains the mail aliases. All these are running on > different machines. Currently, I manually ensure these are all started > in the correct sequence, I would like to automate this to the point > where I can start all 3 servers at the same time and have the different > services wait for the dependency services to be available even though > they are on different systems. > > All the dependency systems in the init-systems I could find are all > based on dependencies on the same server. Does anyone know of something > that can already provide this type of dependencies? Or do I need to > write something myself? With systemd you can add ExecStartPre=/some/script to the service's unit file where /some/script waits for the remote services to become available, and possibly return an error if the service does not become available within a set time. -- Neil Bothwick I have a mind like a steel...uh...thingamajig... signature.asc Description: PGP signature
Re: [gentoo-user] Cross system dependencies
On Saturday, June 28, 2014 01:39:41 PM Neil Bothwick wrote: > On Sat, 28 Jun 2014 11:36:11 +0200, J. Roeleveld wrote: > > I need a way to add dependencies to services which are provided by > > different servers. For instance, my mail server uses DNS to locate my > > LDAP server which contains the mail aliases. All these are running on > > different machines. Currently, I manually ensure these are all started > > in the correct sequence, I would like to automate this to the point > > where I can start all 3 servers at the same time and have the different > > services wait for the dependency services to be available even though > > they are on different systems. > > > > All the dependency systems in the init-systems I could find are all > > based on dependencies on the same server. Does anyone know of something > > that can already provide this type of dependencies? Or do I need to > > write something myself? > > With systemd you can add ExecStartPre=/some/script to the service's unit > file where /some/script waits for the remote services to become available, > and possibly return an error if the service does not become available > within a set time. That method works for any init-system and writing a script to check and if necessary fail is my temporary fall-back plan. I was actually hoping for a method that can be used to monitor availability and, if necessary, stop services when the dependencies disappear. -- Joost
Re: [gentoo-user] Cross system dependencies
On 06/28/2014 07:06 PM, J. Roeleveld wrote: > On Saturday, June 28, 2014 01:39:41 PM Neil Bothwick wrote: >> On Sat, 28 Jun 2014 11:36:11 +0200, J. Roeleveld wrote: >>> I need a way to add dependencies to services which are provided by >>> different servers. For instance, my mail server uses DNS to locate my >>> LDAP server which contains the mail aliases. All these are running on >>> different machines. Currently, I manually ensure these are all started >>> in the correct sequence, I would like to automate this to the point >>> where I can start all 3 servers at the same time and have the different >>> services wait for the dependency services to be available even though >>> they are on different systems. >>> >>> All the dependency systems in the init-systems I could find are all >>> based on dependencies on the same server. Does anyone know of something >>> that can already provide this type of dependencies? Or do I need to >>> write something myself? >> With systemd you can add ExecStartPre=/some/script to the service's unit >> file where /some/script waits for the remote services to become available, >> and possibly return an error if the service does not become available >> within a set time. > That method works for any init-system and writing a script to check and if > necessary fail is my temporary fall-back plan. I was actually hoping for a > method that can be used to monitor availability and, if necessary, stop > services when the dependencies disappear. > > -- > Joost > the difficulty is in identifying failed services. local network issue / load issue could mean your services start bouncing. the best way is to have redundancy so it doesn't matter as much having said all of that:: systemd will start servers and buffer network activity - how this works for non local services would be interesting to see. with openrc : you could on the DNS server have a service which is just a batch script that uses watches for pid / program path in "ps" which outputs ACK or NAK to a file in an NFS share say /nfs/monitoring/dns then on the mail server you could have a service that polls /nfs/monitoring/dns for NAK or ACK you can then choose to have this service directly start your dependent services, or if you adjust /etc/init.d/postfix to have depends = "mymonitorDNS" which is an empty shell of a service. your watchdog service could stop / start the empty shell of a script mymonitorDNS, and then postfix depends on mymonitorDNS this would save you from "i've just stopped the mail server for maintenance and my watchdogservice has just restarted it due to a NAK>ACK event" or... you could have a central master machine which has it's own services, watchdog and monitor... i.e. /etc/init.d/thepostfixserver start / depends on thednsserver which just runs # ssh postfixserver '/etc/init.d/postfix start' or... puppet and it's kin
Re: [gentoo-user] Cross system dependencies
On Saturday, June 28, 2014 09:23:17 PM thegeezer wrote: > On 06/28/2014 07:06 PM, J. Roeleveld wrote: > > On Saturday, June 28, 2014 01:39:41 PM Neil Bothwick wrote: > >> On Sat, 28 Jun 2014 11:36:11 +0200, J. Roeleveld wrote: > >>> I need a way to add dependencies to services which are provided by > >>> different servers. For instance, my mail server uses DNS to locate my > >>> LDAP server which contains the mail aliases. All these are running on > >>> different machines. Currently, I manually ensure these are all started > >>> in the correct sequence, I would like to automate this to the point > >>> where I can start all 3 servers at the same time and have the different > >>> services wait for the dependency services to be available even though > >>> they are on different systems. > >>> > >>> All the dependency systems in the init-systems I could find are all > >>> based on dependencies on the same server. Does anyone know of something > >>> that can already provide this type of dependencies? Or do I need to > >>> write something myself? > >> > >> With systemd you can add ExecStartPre=/some/script to the service's unit > >> file where /some/script waits for the remote services to become > >> available, > >> and possibly return an error if the service does not become available > >> within a set time. > > > > That method works for any init-system and writing a script to check and if > > necessary fail is my temporary fall-back plan. I was actually hoping for a > > method that can be used to monitor availability and, if necessary, stop > > services when the dependencies disappear. > > > > -- > > Joost > > the difficulty is in identifying failed services. > local network issue / load issue could mean your services start bouncing. > the best way is to have redundancy so it doesn't matter as much I know that. A proper system for this would have a configurable amount of retries with a wait-time in between. > having said all of that:: > > systemd will start servers and buffer network activity - how this works > for non local services would be interesting to see. It would, but I am not going to migrate my servers to something like systemd without a clear and proven advantage. For me, that currently does not exist. It also would not work as not all the software I run will happily wait while the rest of the stack starts. I would end up in a bigger mess thanks to timeout issues during startup. > with openrc : > you could on the DNS server have a service which is just a batch script > that uses watches for pid / program path in "ps" which outputs ACK or > NAK to a file in an NFS share say /nfs/monitoring/dns Yes, but in order to access the NFS share, I need DNS to be running. Chicken- egg problem. > then on the mail server you could have a service that polls > /nfs/monitoring/dns for NAK or ACK > you can then choose to have this service directly start your dependent > services, or if you adjust /etc/init.d/postfix to have depends = > "mymonitorDNS" which is an empty shell of a service. your watchdog > service could stop / start the empty shell of a script mymonitorDNS, and > then postfix depends on mymonitorDNS > this would save you from "i've just stopped the mail server for > maintenance and my watchdogservice has just restarted it due to a > NAK>ACK event" That is the problem I have with these watchdog services. During boot, I want it to wait. But it needs to understand not to start a service when I stopped it during runtime. Otherwise it could prevent a clean shutdown as well... > or... > you could have a central master machine which has it's own services, > watchdog and monitor... i.e. /etc/init.d/thepostfixserver start / > depends on thednsserver which just runs > # ssh postfixserver '/etc/init.d/postfix start' > > or... > puppet and it's kin Last time I looked at puppet, it seemed too complex for what I need. I will recheck it again. Thanks, Joost
Re: [gentoo-user] Cross system dependencies
On Sun, 29 Jun 2014 08:55:41 +0200, J. Roeleveld wrote: > > or... > > puppet and it's kin > > Last time I looked at puppet, it seemed too complex for what I need. > I will recheck it again. What about something like monit? -- Neil Bothwick Bug: (n.) any program feature not yet described to the marketing department. signature.asc Description: PGP signature
Re: [gentoo-user] Cross system dependencies
On Sunday, June 29, 2014 09:35:33 AM Neil Bothwick wrote: > On Sun, 29 Jun 2014 08:55:41 +0200, J. Roeleveld wrote: > > > or... > > > puppet and it's kin > > > > Last time I looked at puppet, it seemed too complex for what I need. > > I will recheck it again. > > What about something like monit? Hmm... I looked into that before, don't recall why I didn't look into it properly before. Just had a look on the website, it looks usable, will need to check this. Will also replace nagios at the same time, which I find ok, but don't really like it. I might open a new thread at a later stage when I get round to trying it. Thanks, Joost
