Re: [Pacemaker] ClusterMon Resource starting multiple instances of crm_mon

2013-05-13 Thread Florian Crouzat

Le 09/05/2013 16:40, Steven Bambling a écrit :

I'm having some issues with getting some cluster  monitoring setup and
configured on a 3 node multi-state cluster.   I'm using Florian's blog
as an example
http://floriancrouzat.net/2013/01/monitor-a-pacemaker-cluster-with-ocfpacemakerclustermon-andor-external-agent/.



Btw, if any clarification is required on this blog post, let me know ;)

--
Cheers,
Florian Crouzat

___
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


Re: [Pacemaker] ClusterMon Resource starting multiple instances of crm_mon

2013-05-12 Thread Andrew Beekhof

On 10/05/2013, at 8:08 PM, Steven Bambling smbambl...@arin.net wrote:

 
 On May 10, 2013, at 5:35 AM, Steven Bambling smbambl...@arin.net wrote:
 
 
 On May 9, 2013, at 8:05 PM, Andrew Beekhof and...@beekhof.net wrote:
 
 
 On 10/05/2013, at 12:40 AM, Steven Bambling smbambl...@arin.net wrote:
 
 I'm having some issues with getting some cluster  monitoring setup and 
 configured on a 3 node multi-state cluster.   I'm using Florian's blog as 
 an example 
 http://floriancrouzat.net/2013/01/monitor-a-pacemaker-cluster-with-ocfpacemakerclustermon-andor-external-agent/.
 
 When I create the primitive resource it starts on one of my nodes but 
 spawns multiple instances of crm_mon.  I don't see any reason that would 
 cause it to spawn multiple instances, its very odd behavior.
 
 If you run:
 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 
 manually a few times, what happens?  Multiple processes?
 
 Yep for some reason its spawning multiple processes.
 
 root@pgdb3 ~]# ps aux | grep crm_mon
 root 30678  0.0  0.0 103244   856 pts/0S+   05:30   0:00 grep crm_mon
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# ps aux | grep crm_mon
 root 30772  0.0  0.0  82744  2816 ?S05:30   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 30781  0.0  0.0  82744  2668 ?S05:30   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 30784  0.0  0.0  82744  2476 ?S05:30   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 31134  0.0  0.0 103244   856 pts/0S+   05:30   0:00 grep crm_mon
 
 Put the .pid file in the tmp dir only lists 1 pid
 [root@pgdb3 ~]# cat /tmp/ClusterMon_SNMPMon.pid
   30772
 
 I take that back I doubled checked and the SNMPMon resource was still started 
 which was creating the multiple processes.  After I stopped the resource I 
 pkill'd all the crm_mon process and then ran the command again manually.  Now 
 it seems to squash the additional processes and only allows 1 process to be 
 running.
 
 [root@pgdb3 tmp]# ps aux | grep crm_mon
 root 30955  0.0  0.0  82492  2632 pts/0S06:05   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 31991  0.0  0.0 103244   852 pts/0S+   06:05   0:00 grep crm_mon
 [root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 tmp]# ps aux | grep crm_mon
 root 30955  0.0  0.0  82492  2632 pts/0S06:05   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 32545  0.0  0.0 103244   856 pts/0S+   06:06   0:00 grep crm_mon

Its possibly due to a stale pid file.
I'm believe the following patches should fix the problem.

+ Andrew Beekhof (12 days ago) 479c5cc: Fix: crm_mon: Check if a process can be 
daemonized before forking so the parent can report an error 
+ Andrew Beekhof (12 days ago) e549770: Fix: crm_mon: Ensure stale pid files 
are updated when a new process is started 

https://github.com/beekhof/pacemaker/commit/e549770
https://github.com/beekhof/pacemaker/commit/479c5cc

Weird that it works from the command line but not the resource agent.
Are the permissions on /tmp/ClusterMon_SNMPMon.pid ok?

 
 STEVE
 
 
 
 
 I was 

Re: [Pacemaker] ClusterMon Resource starting multiple instances of crm_mon

2013-05-12 Thread Andrew Beekhof

On 10/05/2013, at 7:35 PM, Steven Bambling smbambl...@arin.net wrote:

 If this is correct what is the best practice for monitoring additional 
 resource states?
 
 Define additional?
 If the resource fails we'll normally recover it automatically.
 An example of an additional resource would be a vip using ( IPaddr2 ).  Also 
 I have a multi-state pgsql resource, so if the resource fails it will either 
 try to restart or promote another node in the cluster to Master.

As long as each resource has a monitor action defined, then the cluster is 
keeping an eye on them.
ClusterMon is for situations when you want to watch pacemaker watching the 
resources...
___
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


Re: [Pacemaker] ClusterMon Resource starting multiple instances of crm_mon

2013-05-10 Thread Steven Bambling

On May 9, 2013, at 8:05 PM, Andrew Beekhof and...@beekhof.net wrote:

 
 On 10/05/2013, at 12:40 AM, Steven Bambling smbambl...@arin.net wrote:
 
 I'm having some issues with getting some cluster  monitoring setup and 
 configured on a 3 node multi-state cluster.   I'm using Florian's blog as an 
 example 
 http://floriancrouzat.net/2013/01/monitor-a-pacemaker-cluster-with-ocfpacemakerclustermon-andor-external-agent/.
 
 When I create the primitive resource it starts on one of my nodes but spawns 
 multiple instances of crm_mon.  I don't see any reason that would cause it 
 to spawn multiple instances, its very odd behavior.
 
 If you run:
 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 
 manually a few times, what happens?  Multiple processes?

Yep for some reason its spawning multiple processes.

root@pgdb3 ~]# ps aux | grep crm_mon
root 30678  0.0  0.0 103244   856 pts/0S+   05:30   0:00 grep crm_mon
[root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 ~]# ps aux | grep crm_mon
root 30772  0.0  0.0  82744  2816 ?S05:30   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 30781  0.0  0.0  82744  2668 ?S05:30   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 30784  0.0  0.0  82744  2476 ?S05:30   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 31134  0.0  0.0 103244   856 pts/0S+   05:30   0:00 grep crm_mon

Put the .pid file in the tmp dir only lists 1 pid
[root@pgdb3 ~]# cat /tmp/ClusterMon_SNMPMon.pid
30772

 
 
 I was also looking for some clarification on what this resource provides….it 
 looks to me that it kicks off a crm_mon in daemon mode that will update a 
 .html file and with -E it will run an external script.  But the resource 
 itself doesn't trigger anything if another resource changes state only if 
 the crm_mon process ( monitored with PID ) fails and it has to restart.
 
 Correct, it just updates the html file which you can see in your browser.
 Or, with -E, it can send an email or snmp alert.
 
 If this is correct what is the best practice for monitoring additional 
 resource states?
 
 Define additional?
 If the resource fails we'll normally recover it automatically.
An example of an additional resource would be a vip using ( IPaddr2 ).  Also I 
have a multi-state pgsql resource, so if the resource fails it will either try 
to restart or promote another node in the cluster to Master.

v/r

STEVE

 
 
 v/r
 
 STEVE
 
 
 Below are some additional data points. 
 
 
 Creating the Resource
 
 [root@pgdb2 tmp]# crm configure primitive SNMPMon ocf:pacemaker:ClusterMon \
   params user=root update=30 extra_options=-E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net \
   op monitor on-fail=restart interval=60
 
 
 Manual crm_mon output
 
 Last updated: Thu May  9 10:24:30 2013
 Last change: Thu May  9 10:20:49 2013 via cibadmin on pgdb2.example.com
 Stack: cman
 Current DC: pgdb1.example.com - partition with quorum
 Version: 1.1.8-7.el6-394e906
 3 Nodes configured, unknown expected votes
 6 Resources configured.
 
 
 Node pgdb1.example.com: standby
 Online: [ pgdb2.example.com pgdb3.example.com ]
 
 PG_REP_VIP   (ocf::heartbeat:IPaddr2):   Started pgdb2.example.com
 PG_CLI_VIP   (ocf::heartbeat:IPaddr2):   Started pgdb2.example.com
 Master/Slave Set: msPGSQL [PGSQL]
Masters: [ pgdb2.example.com ]
Slaves: [ pgdb3.example.com ]
Stopped: [ PGSQL:2 ]
 SNMPMon  (ocf::pacemaker:ClusterMon):Started pgdb3.example.com
 
 PS to check for process on pgdb3
 
 [root@pgdb3 tmp]# ps aux | grep crm_mon
 root 16097  0.0  0.0  82624  2784 ?S10:20   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 16099  0.0  0.0  82624  2660 ?S10:20   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 

Re: [Pacemaker] ClusterMon Resource starting multiple instances of crm_mon

2013-05-10 Thread Steven Bambling

On May 10, 2013, at 5:35 AM, Steven Bambling smbambl...@arin.net wrote:

 
 On May 9, 2013, at 8:05 PM, Andrew Beekhof and...@beekhof.net wrote:
 
 
 On 10/05/2013, at 12:40 AM, Steven Bambling smbambl...@arin.net wrote:
 
 I'm having some issues with getting some cluster  monitoring setup and 
 configured on a 3 node multi-state cluster.   I'm using Florian's blog as 
 an example 
 http://floriancrouzat.net/2013/01/monitor-a-pacemaker-cluster-with-ocfpacemakerclustermon-andor-external-agent/.
 
 When I create the primitive resource it starts on one of my nodes but 
 spawns multiple instances of crm_mon.  I don't see any reason that would 
 cause it to spawn multiple instances, its very odd behavior.
 
 If you run:
 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 
 manually a few times, what happens?  Multiple processes?
 
 Yep for some reason its spawning multiple processes.
 
 root@pgdb3 ~]# ps aux | grep crm_mon
 root 30678  0.0  0.0 103244   856 pts/0S+   05:30   0:00 grep crm_mon
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 [root@pgdb3 ~]# ps aux | grep crm_mon
 root 30772  0.0  0.0  82744  2816 ?S05:30   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 30781  0.0  0.0  82744  2668 ?S05:30   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 30784  0.0  0.0  82744  2476 ?S05:30   0:00 
 /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
 /tmp/ClusterMon_SNMPMon.html
 root 31134  0.0  0.0 103244   856 pts/0S+   05:30   0:00 grep crm_mon
 
 Put the .pid file in the tmp dir only lists 1 pid
 [root@pgdb3 ~]# cat /tmp/ClusterMon_SNMPMon.pid
30772

I take that back I doubled checked and the SNMPMon resource was still started 
which was creating the multiple processes.  After I stopped the resource I 
pkill'd all the crm_mon process and then ran the command again manually.  Now 
it seems to squash the additional processes and only allows 1 process to be 
running.

[root@pgdb3 tmp]# ps aux | grep crm_mon
root 30955  0.0  0.0  82492  2632 pts/0S06:05   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 31991  0.0  0.0 103244   852 pts/0S+   06:05   0:00 grep crm_mon
[root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 tmp]# /usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
[root@pgdb3 tmp]# ps aux | grep crm_mon
root 30955  0.0  0.0  82492  2632 pts/0S06:05   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 32545  0.0  0.0 103244   856 pts/0S+   06:06   0:00 grep crm_mon

STEVE

 
 
 
 I was also looking for some clarification on what this resource 
 provides….it looks to me that it kicks off a crm_mon in daemon mode that 
 will update a .html file and with -E it will run an external script.  But 
 the resource itself doesn't trigger anything if another resource changes 
 state only if the crm_mon process ( monitored with PID ) fails and it has 
 to restart.
 
 Correct, it just updates the html file which you can see in your browser.
 Or, with -E, it can send an email or snmp alert.
 
 If this is correct what is the best practice for monitoring additional 
 resource states?
 
 Define additional?
 If the resource fails we'll normally recover it automatically.
 An example of an 

[Pacemaker] ClusterMon Resource starting multiple instances of crm_mon

2013-05-09 Thread Steven Bambling
I'm having some issues with getting some cluster  monitoring setup and 
configured on a 3 node multi-state cluster.   I'm using Florian's blog as an 
example 
http://floriancrouzat.net/2013/01/monitor-a-pacemaker-cluster-with-ocfpacemakerclustermon-andor-external-agent/.

When I create the primitive resource it starts on one of my nodes but spawns 
multiple instances of crm_mon.  I don't see any reason that would cause it to 
spawn multiple instances, its very odd behavior.

I was also looking for some clarification on what this resource provides….it 
looks to me that it kicks off a crm_mon in daemon mode that will update a .html 
file and with -E it will run an external script.  But the resource itself 
doesn't trigger anything if another resource changes state only if the crm_mon 
process ( monitored with PID ) fails and it has to restart.  If this is correct 
what is the best practice for monitoring additional resource states?

v/r

STEVE


Below are some additional data points. 


Creating the Resource

[root@pgdb2 tmp]# crm configure primitive SNMPMon ocf:pacemaker:ClusterMon \
 params user=root update=30 extra_options=-E 
 /usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net \
 op monitor on-fail=restart interval=60


Manual crm_mon output

Last updated: Thu May  9 10:24:30 2013
Last change: Thu May  9 10:20:49 2013 via cibadmin on pgdb2.example.com
Stack: cman
Current DC: pgdb1.example.com - partition with quorum
Version: 1.1.8-7.el6-394e906
3 Nodes configured, unknown expected votes
6 Resources configured.


Node pgdb1.example.com: standby
Online: [ pgdb2.example.com pgdb3.example.com ]

 PG_REP_VIP (ocf::heartbeat:IPaddr2):   Started pgdb2.example.com
 PG_CLI_VIP (ocf::heartbeat:IPaddr2):   Started pgdb2.example.com
 Master/Slave Set: msPGSQL [PGSQL]
 Masters: [ pgdb2.example.com ]
 Slaves: [ pgdb3.example.com ]
 Stopped: [ PGSQL:2 ]
 SNMPMon(ocf::pacemaker:ClusterMon):Started pgdb3.example.com

PS to check for process on pgdb3

[root@pgdb3 tmp]# ps aux | grep crm_mon
root 16097  0.0  0.0  82624  2784 ?S10:20   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 16099  0.0  0.0  82624  2660 ?S10:20   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 16104  0.0  0.0  82624  2448 ?S10:20   0:00 
/usr/sbin/crm_mon -p /tmp/ClusterMon_SNMPMon.pid -d -i 0 -E 
/usr/local/bin/pcmk_snmp_helper.sh -e zen.arin.net -h 
/tmp/ClusterMon_SNMPMon.html
root 16515  0.0  0.0 103244   852 pts/0S+   10:21   0:00 grep crm_mon

Output from corosync.log

May 09 10:20:51 [3100] pgdb3.cha.arin.net   lrmd: info: 
process_lrmd_get_rsc_info:  Resource 'SNMPMon' not found (3 active 
resources)
May 09 10:20:51 [3100] pgdb3.cha.arin.net   lrmd: info: 
process_lrmd_rsc_register:  Added 'SNMPMon' to the rsc list (4 active 
resources)
May 09 10:20:52 [3103] pgdb3.cha.arin.net   crmd: info: 
services_os_action_execute: Managed ClusterMon_meta-data_0 process 16010 
exited with rc=0
May 09 10:20:52 [3103] pgdb3.cha.arin.net   crmd:   notice: 
process_lrm_event:  LRM operation SNMPMon_monitor_0 (call=61, rc=7, 
cib-update=28, confirmed=true) not running
May 09 10:20:52 [3103] pgdb3.cha.arin.net   crmd:   notice: 
process_lrm_event:  LRM operation SNMPMon_start_0 (call=64, rc=0, 
cib-update=29, confirmed=true) ok
May 09 10:20:52 [3103] pgdb3.cha.arin.net   crmd:   notice: 
process_lrm_event:  LRM operation SNMPMon_monitor_6 (call=67, rc=0, 
cib-update=30, confirmed=false) ok

signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org