Re: [ClusterLabs] drbd could not start by pacemaker. strange limited root privileges?
Looks like selinux restrictions. 23.05.2019, 14:22, "László Neduki" :Hi, (I sent a similar question from an other acount 3 days ago, but: - I do not see it on the list. Maybe I should not see my own email? So I created a new account- I have additional infos (but no solution), so I rewrite the question) pacemaker cannot start drbd9 resources. As I see, root has very limited privileges in the drbd resource agent, when it run by the pacemaker. I downloaded the latest pacemaker this week, and I compiled drbd9 rpms also. I hope, You can help me, I do not find the cause of this behaviour. Please see the below test cases: 1. When I create Pacemaker DRBD resource I get errors# pcs resource create DrbdDB ocf:linbit:drbd drbd_resource=drbd_db op monitor interval=60s meta notify=true# pcs resource master DrbdDBClone DrbdDB master-max=1 master-node-max=1 clone-node-max=1 notify=true# pcs constraint location DrbdDBClone prefers node1=INFINITY# pcs cluster stop --all; pcs cluster start --all; pcs statusFailed Actions:* DrbdDB_monitor_0 on node1 'not installed' (5): call=6, status=complete, exitreason='DRBD kernel (module) not available?', last-rc-change='Thu May 23 09:54:09 2019', queued=0ms, exec=58ms* DrbdDB_monitor_0 on node2 'not installed' (5): call=6, status=complete, exitreason='DRBD kernel (module) not available?', last-rc-change='Thu May 23 10:00:22 2019', queued=0ms, exec=71ms 2. when I try to start drbd_db by drbdadm directly, it works well:# modprobe drbd #on each node# drbdadm up drbd_db #on each node# drbdadm primary drbd_db# drbdadm status it shows drbd_db is UpToDate on each nodeI also can promote and mount filesystem well 3. When I use debug-start, it works fine (so the resource syntax sould be correct)# drbdadm statusNo currently configured DRBD found.# pcs resource debug-start DrbdDBMasterError: unable to debug-start a master, try the master's resource: DrbdDB# pcs resource debug-start DrbdDB #on each nodeOperation start for DrbdDB:0 (ocf:linbit:drbd) returned: 'ok' (0)# drbdadm statusit shows drbd_db is UpToDate on each node 4. Pacemaker handle other resources well . If I set auto_promote=yes, and I start (but not promote) the drbd_db by drbdadm, then pacemaker can create filesystem on it well, and also the appserver, database resources. 5. The strangest behaviour for me. Root have very limited privileges whitin the drbd resource agent. If I write this line to the srbd_start() method of /usr/lib/ocf/resource.d/linbit/drbd ocf_log err "lados " $(whoami) $( ls -l /home/opc/tmp/modprobe2.trace ) $( do_cmd touch /home/opc/tmp/modprobe2.trace ) I got theese messeges in log, when I start the cluster # tail -f /var/log/cluster/corosync.log | grep -A 8 -B 3 -i lados ...May 21 15:35:12 drbd(DrbdDB)[31649]: ERROR: lados rootMay 21 15:35:12 [31309] node1 lrmd: notice: operation_finished: DrbdDB_start_0:31649:stderr [ ls: cannot access /home/opc/tmp/modprobe2.trace: Permission denied ]May 21 15:35:12 [31309] node1 lrmd: notice: operation_finished: DrbdFra_start_0:31649:stderr [ touch: cannot touch '/home/opc/tmp/modprobe2.trace': Permission denied ]...and also, when I try to strace the "modprobe -s drbd `$DRBDADM sh-mod-parms`" in drbd resource agent, I only see 1 line in the /root/modprobe2.trace. This meens for me:- root cannot trace the calls in drbdadm (even if root can strace drbdadm outside of pacemaker well)- root can write into files his own directory (/root/modprobe2.trace) 6. Opposit of previous testroot has these privileges outside from pacamaker # sudo su -# touch /home/opc/tmp/modprobe2.trace# ls -l /home/opc/tmp/modprobe2.trace-rw-r--r--. 1 root root 0 May 21 15:44 /home/opc/tmp/modprobe2.trace Thanks: lados. ,___Manage your subscription:https://lists.clusterlabs.org/mailman/listinfo/usersClusterLabs home: https://www.clusterlabs.org/ Sincerely,George Melikov ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] drbd could not start by pacemaker. strange limited root privileges?
Hi Ken, very thanks for SELinux advice ! I forget to check it. After I disabled SELinux temporarily, pacmaker starts to handle the DRBD resources well. (I checked the agent scripts earlier, but they were same as in the scripts subdir of drbd9 source). And I will check my email settings. Thanks again: lados. Ken Gaillot ezt írta (időpont: 2019. máj. 23., Cs, 16:21): > On Thu, 2019-05-23 at 13:21 +0200, László Neduki wrote: > > Hi, > > > > ( > > I sent a similar question from an other acount 3 days ago, but: > > - I do not see it on the list. Maybe I should not see my own email? > > A DRBD message from govom...@gmail.com did make it to the list a week > ago. You should get your own emails from the list server, though your > own mail server or client might filter them. > > > So I created a new account > > - I have additional infos (but no solution), so I rewrite the > > question > > ) > > > > pacemaker cannot start drbd9 resources. As I see, root has very > > limited privileges in the drbd resource agent, when it run by the > > pacemaker. I downloaded the latest pacemaker this week, and I > > compiled drbd9 rpms also. I hope, You can help me, I do not find the > > cause of this behaviour. Please see the below test cases: > > I'm not a DRBD expert, but given the symptoms you describe, my first > thoughts would be that either the ocf:linbit:drbd agent you're using > isn't the right version for your DRBD version, or something like > SELinux is restricting access. > > > 1. When I create Pacemaker DRBD resource I get errors > > # pcs resource create DrbdDB ocf:linbit:drbd drbd_resource=drbd_db op > > monitor interval=60s meta notify=true > > # pcs resource master DrbdDBClone DrbdDB master-max=1 master-node- > > max=1 clone-node-max=1 notify=true > > # pcs constraint location DrbdDBClone prefers node1=INFINITY > > # pcs cluster stop --all; pcs cluster start --all; pcs status > > > > Failed Actions: > > * DrbdDB_monitor_0 on node1 'not installed' (5): call=6, > > status=complete, exitreason='DRBD kernel (module) not available?', > > last-rc-change='Thu May 23 09:54:09 2019', queued=0ms, exec=58ms > > * DrbdDB_monitor_0 on node2 'not installed' (5): call=6, > > status=complete, exitreason='DRBD kernel (module) not available?', > > last-rc-change='Thu May 23 10:00:22 2019', queued=0ms, exec=71ms > > > > 2. when I try to start drbd_db by drbdadm directly, it works well: > > # modprobe drbd #on each node > > # drbdadm up drbd_db #on each node > > # drbdadm primary drbd_db > > # drbdadm status > > it shows drbd_db is UpToDate on each node > > I also can promote and mount filesystem well > > > > 3. When I use debug-start, it works fine (so the resource syntax > > sould be correct) > > # drbdadm status > > No currently configured DRBD found. > > # pcs resource debug-start DrbdDBMaster > > Error: unable to debug-start a master, try the master's resource: > > DrbdDB > > # pcs resource debug-start DrbdDB #on each node > > Operation start for DrbdDB:0 (ocf:linbit:drbd) returned: 'ok' (0) > > # drbdadm status > > it shows drbd_db is UpToDate on each node > > > > 4. Pacemaker handle other resources well . If I set auto_promote=yes, > > and I start (but not promote) the drbd_db by drbdadm, then pacemaker > > can create filesystem on it well, and also the appserver, database > > resources. > > > > 5. The strangest behaviour for me. Root have very limited privileges > > whitin the drbd resource agent. If I write this line to the > > srbd_start() method of /usr/lib/ocf/resource.d/linbit/drbd > > > > ocf_log err "lados " $(whoami) $( ls -l /home/opc/tmp/modprobe2.trace > > ) $( do_cmd touch /home/opc/tmp/modprobe2.trace ) > > > > I got theese messeges in log, when I start the cluster > > > > # tail -f /var/log/cluster/corosync.log | grep -A 8 -B 3 -i lados > > > > ... > > May 21 15:35:12 drbd(DrbdDB)[31649]:ERROR: lados root > > May 21 15:35:12 [31309] node1 lrmd: notice: > > operation_finished:DrbdDB_start_0:31649:stderr [ ls: cannot > > access /home/opc/tmp/modprobe2.trace: Permission denied ] > > May 21 15:35:12 [31309] node1 lrmd: notice: > > operation_finished:DrbdFra_start_0:31649:stderr [ touch: cannot > > touch '/home/opc/tmp/modprobe2.trace': Permission denied ] > > ... > > and also, when I try to strace the "modprobe -s drbd `$DRBDADM sh- > > mod-parms`" in drbd resource agent, I only see 1 line in the > > /root/modprobe2.trace. This meens for me: > > - root cannot trace the calls in drbdadm (even if root can strace > > drbdadm outside of pacemaker well) > > - root can write into files his own directory > > (/root/modprobe2.trace) > > > > 6. Opposit of previous test > > root has these privileges outside from pacamaker > > > > # sudo su - > > # touch /home/opc/tmp/modprobe2.trace > > # ls -l /home/opc/tmp/modprobe2.trace > > -rw-r--r--. 1 root root 0 May 21 15:44 /home/opc/tmp/modprobe2.trace > > > > > > Thanks: lados. > > > > > > ___ > >
Re: [ClusterLabs] drbd could not start by pacemaker. strange limited root privileges?
On Thu, 2019-05-23 at 13:21 +0200, László Neduki wrote: > Hi, > > ( > I sent a similar question from an other acount 3 days ago, but: > - I do not see it on the list. Maybe I should not see my own email? A DRBD message from govom...@gmail.com did make it to the list a week ago. You should get your own emails from the list server, though your own mail server or client might filter them. > So I created a new account > - I have additional infos (but no solution), so I rewrite the > question > ) > > pacemaker cannot start drbd9 resources. As I see, root has very > limited privileges in the drbd resource agent, when it run by the > pacemaker. I downloaded the latest pacemaker this week, and I > compiled drbd9 rpms also. I hope, You can help me, I do not find the > cause of this behaviour. Please see the below test cases: I'm not a DRBD expert, but given the symptoms you describe, my first thoughts would be that either the ocf:linbit:drbd agent you're using isn't the right version for your DRBD version, or something like SELinux is restricting access. > 1. When I create Pacemaker DRBD resource I get errors > # pcs resource create DrbdDB ocf:linbit:drbd drbd_resource=drbd_db op > monitor interval=60s meta notify=true > # pcs resource master DrbdDBClone DrbdDB master-max=1 master-node- > max=1 clone-node-max=1 notify=true > # pcs constraint location DrbdDBClone prefers node1=INFINITY > # pcs cluster stop --all; pcs cluster start --all; pcs status > > Failed Actions: > * DrbdDB_monitor_0 on node1 'not installed' (5): call=6, > status=complete, exitreason='DRBD kernel (module) not available?', > last-rc-change='Thu May 23 09:54:09 2019', queued=0ms, exec=58ms > * DrbdDB_monitor_0 on node2 'not installed' (5): call=6, > status=complete, exitreason='DRBD kernel (module) not available?', > last-rc-change='Thu May 23 10:00:22 2019', queued=0ms, exec=71ms > > 2. when I try to start drbd_db by drbdadm directly, it works well: > # modprobe drbd #on each node > # drbdadm up drbd_db #on each node > # drbdadm primary drbd_db > # drbdadm status > it shows drbd_db is UpToDate on each node > I also can promote and mount filesystem well > > 3. When I use debug-start, it works fine (so the resource syntax > sould be correct) > # drbdadm status > No currently configured DRBD found. > # pcs resource debug-start DrbdDBMaster > Error: unable to debug-start a master, try the master's resource: > DrbdDB > # pcs resource debug-start DrbdDB #on each node > Operation start for DrbdDB:0 (ocf:linbit:drbd) returned: 'ok' (0) > # drbdadm status > it shows drbd_db is UpToDate on each node > > 4. Pacemaker handle other resources well . If I set auto_promote=yes, > and I start (but not promote) the drbd_db by drbdadm, then pacemaker > can create filesystem on it well, and also the appserver, database > resources. > > 5. The strangest behaviour for me. Root have very limited privileges > whitin the drbd resource agent. If I write this line to the > srbd_start() method of /usr/lib/ocf/resource.d/linbit/drbd > > ocf_log err "lados " $(whoami) $( ls -l /home/opc/tmp/modprobe2.trace > ) $( do_cmd touch /home/opc/tmp/modprobe2.trace ) > > I got theese messeges in log, when I start the cluster > > # tail -f /var/log/cluster/corosync.log | grep -A 8 -B 3 -i lados > > ... > May 21 15:35:12 drbd(DrbdDB)[31649]:ERROR: lados root > May 21 15:35:12 [31309] node1 lrmd: notice: > operation_finished:DrbdDB_start_0:31649:stderr [ ls: cannot > access /home/opc/tmp/modprobe2.trace: Permission denied ] > May 21 15:35:12 [31309] node1 lrmd: notice: > operation_finished:DrbdFra_start_0:31649:stderr [ touch: cannot > touch '/home/opc/tmp/modprobe2.trace': Permission denied ] > ... > and also, when I try to strace the "modprobe -s drbd `$DRBDADM sh- > mod-parms`" in drbd resource agent, I only see 1 line in the > /root/modprobe2.trace. This meens for me: > - root cannot trace the calls in drbdadm (even if root can strace > drbdadm outside of pacemaker well) > - root can write into files his own directory > (/root/modprobe2.trace) > > 6. Opposit of previous test > root has these privileges outside from pacamaker > > # sudo su - > # touch /home/opc/tmp/modprobe2.trace > # ls -l /home/opc/tmp/modprobe2.trace > -rw-r--r--. 1 root root 0 May 21 15:44 /home/opc/tmp/modprobe2.trace > > > Thanks: lados. > > > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ -- Ken Gaillot ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
