Re: [ClusterLabs] Error When Creating LVM Resource
That makes sense. I wasn’t yet configuring the constraints on the cluster and was alarmed by the error messages…especially the ones where it seemed like the services weren’t starting anywhere. Eventually, however, that somehow magically resolved itself, so I did went ahead with adding the resource constraints. Here’s what I added: # pcs constraint colocation add gctvanas-vip with gctvanas-fs2o INFINITY with-rsc-role=Master # pcs constraint colocation add gctvanas-lvm with gctvanas-fs2o INFINITY with-rsc-role=Master # pcs constraint colocation add gctvanas-tgt with gctvanas-fs2o INFINITY with-rsc-role=Master # pcs constraint colocation add gctvanas-lun1 with gctvanas-fs2o INFINITY with-rsc-role=Master # pcs constraint colocation add gctvanas-lun2 with gctvanas-fs2o INFINITY with-rsc-role=Master # pcs constraint order promote gctvanas-fs2o then start gctvanas-lvm # pcs constraint order gctvanas-vip then gctvanas-lvm # pcs constraint order gctvanas-lvm then gctvanas-tgt # pcs constraint order gctvanas-tgt then gctvanas-lun1 # pcs constraint order gctvanas-tgt then gctvanas-lun2 # pcs constraint Location Constraints: Ordering Constraints: promote gctvanas-fs2o then start gctvanas-lvm (kind:Mandatory) start gctvanas-vip then start gctvanas-lvm (kind:Mandatory) start gctvanas-lvm then start gctvanas-tgt (kind:Mandatory) start gctvanas-tgt then start gctvanas-lun1 (kind:Mandatory) start gctvanas-tgt then start gctvanas-lun2 (kind:Mandatory) Colocation Constraints: gctvanas-vip with gctvanas-fs2o (score:INFINITY) (with-rsc-role:Master) gctvanas-lvm with gctvanas-fs2o (score:INFINITY) (with-rsc-role:Master) gctvanas-tgt with gctvanas-fs2o (score:INFINITY) (with-rsc-role:Master) gctvanas-lun1 with gctvanas-fs2o (score:INFINITY) (with-rsc-role:Master) gctvanas-lun2 with gctvanas-fs2o (score:INFINITY) (with-rsc-role:Master) I think this looks about right… hopefully when I test everything doesn’t go t/u. Thanks for the input! -- [ jR ] @: ja...@eramsey.org<mailto:ja...@eramsey.org> there is no path to greatness; greatness is the path From: Greg Woods Reply-To: Cluster Labs - All topics related to open-source clustering welcomed Date: Friday, August 26, 2016 at 2:09 PM To: Cluster Labs - All topics related to open-source clustering welcomed Subject: Re: [ClusterLabs] Error When Creating LVM Resource On Fri, Aug 26, 2016 at 9:32 AM, Jason A Ramsey mailto:ja...@eramsey.org>> wrote: Failed Actions: * gctvanas-lvm_start_0 on node1 'not running' (7): call=42, status=complete, exitreason='LVM: targetfs did not activate correctly', last-rc-change='Fri Aug 26 10:57:22 2016', queued=0ms, exec=577ms * gctvanas-lvm_start_0 on node2 'unknown error' (1): call=34, status=complete, exitreason='Volume group [targetfs] does not exist or contains error! Volume group "targetfs" not found', last-rc-change='Fri Aug 26 10:57:21 2016', queued=0ms, exec=322ms I think you need a colocation constraint to prevent it from trying to start the LVM resource on the DRBD secondary node. I used to run LVM-over-DRBD clusters but don't any more (switched to NFS backend storage), so I don't remember the exact syntax, but you certainly don't want the LVM resource to start on node2 at this point because it will certainly fail. It may not be running on node1 because it failed on node2, so if you can get the proper colocation constraint in place, things may work after you do a resource cleanup. (I stand ready to be corrected by someone more knowledgeable who can spot a configuration problem that I missed). If you still get failure and the constraint is correct, then I would try running the lvcreate command manually on the DRBD primary node to make sure that works. --Greg ___ Users mailing list: Users@clusterlabs.org http://clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [ClusterLabs] Error When Creating LVM Resource
On Fri, Aug 26, 2016 at 9:32 AM, Jason A Ramsey wrote: > Failed Actions: > > * gctvanas-lvm_start_0 on node1 'not running' (7): call=42, > status=complete, exitreason='LVM: targetfs did not activate correctly', > > last-rc-change='Fri Aug 26 10:57:22 2016', queued=0ms, exec=577ms > > * gctvanas-lvm_start_0 on node2 'unknown error' (1): call=34, > status=complete, exitreason='Volume group [targetfs] does not exist or > contains error! Volume group "targetfs" not found', > > last-rc-change='Fri Aug 26 10:57:21 2016', queued=0ms, exec=322ms > > > I think you need a colocation constraint to prevent it from trying to start the LVM resource on the DRBD secondary node. I used to run LVM-over-DRBD clusters but don't any more (switched to NFS backend storage), so I don't remember the exact syntax, but you certainly don't want the LVM resource to start on node2 at this point because it will certainly fail. It may not be running on node1 because it failed on node2, so if you can get the proper colocation constraint in place, things may work after you do a resource cleanup. (I stand ready to be corrected by someone more knowledgeable who can spot a configuration problem that I missed). If you still get failure and the constraint is correct, then I would try running the lvcreate command manually on the DRBD primary node to make sure that works. --Greg ___ Users mailing list: Users@clusterlabs.org http://clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
[ClusterLabs] Error When Creating LVM Resource
So, I’m setting up a two node cluster that will eventually (hopefully) serve as
an HA iSCSI Target (Active/Passive) on RHEL 6. I’m using the [incredibly poorly
written] guide I found on Linbit’s website (“Highly available iSCSI storage
with DRBD and Pacemaker”). I have somehow gotten pretty far through it, but
I’ve hit a couple of snags.
Here’s my drbd.conf (/etc/drbd.d/global_common.conf):
global {
usage-count yes;
# minor-count dialog-refresh disable-ip-verification
}
common {
protocol C;
handlers {
# These are EXAMPLE handlers only.
# They may have severe implications,
# like hard resetting the node under certain
circumstances.
# Be careful when chosing your poison.
# pri-on-incon-degr
"/usr/lib/drbd/notify-pri-on-incon-degr.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot
-f";
# pri-lost-after-sb
"/usr/lib/drbd/notify-pri-lost-after-sb.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot
-f";
# local-io-error
"/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh;
echo o > /proc/sysrq-trigger ; halt -f";
# fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
# split-brain
"/usr/lib/drbd/notify-split-brain.sh root";
# out-of-sync
"/usr/lib/drbd/notify-out-of-sync.sh root";
# before-resync-target
"/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k";
# after-resync-target
/usr/lib/drbd/unsnapshot-resync-target-lvm.sh;
}
startup {
# wfc-timeout degr-wfc-timeout
outdated-wfc-timeout wait-after-sb
}
disk {
# on-io-error fencing use-bmbv no-disk-barrier
no-disk-flushes
# no-disk-drain no-md-flushes max-bio-bvecs
}
net {
# sndbuf-size rcvbuf-size timeout connect-int
ping-int ping-timeout max-buffers
# max-epoch-size ko-count allow-two-primaries
cram-hmac-alg shared-secret
# after-sb-0pri after-sb-1pri after-sb-2pri
data-integrity-alg no-tcp-cork
}
syncer {
# rate after al-extents use-rle cpu-mask
verify-alg csums-alg
}
}
And here’s my targetfs.res config (/etc/drbd.d/targetfs.res):
resource targetfs {
protocol C;
meta-disk internal;
device /dev/drbd1;
disk /dev/xvdf;
syncer {
verify-alg sha1;
c-plan-ahead 0;
rate 32M;
}
net {
allow-two-primaries;
}
on node1 {
address 10.130.96.120:7789;
}
on node2 {
address 10.130.97.165:7789;
}
}
These, of course, live on both nodes.
Once I create the drbd md and sync the nodes:
(node1)# drbdadm create-md targetfs
(node2)# drbdadm create-md targetfs
(node1)# drbdadm up targetfs
(node2)# drbdadm up targetfs
(node2)# drbdadm invalidate targetfs
(node1)# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,
2014-11-24 14:51:37
1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-
ns:134213632 nr:0 dw:36 dr:134215040 al:1 bm:8192 lo:0 pe:0 ua:0 ap:0 ep:1
wo:f oos:0
I run a pvcreate and an lvcreate on node1:
(node1)# pvcreate /dev/drbd/by-res/targetfs
(node1)# vgcreate targetfs /dev/drbd/by-res/targetfs
(node1)# pvs && vgs
PV VG Fmt Attr PSize PFree
/dev/drbd1 targetfs lvm2 a--u 127.99g 127.99g
VG #PV #LV #SN Attr VSize VFree
targetfs 1 0 0 wz--n- 127.99g 127.99g
pcs cluster configuration goes well enough for a bit:
# pcs cluster setup --name gctvanas node1 node2 --transport udpu
# pcs cluster start --all
# pcs property set stonith-enabled=false
# pcs property set no-quorum-policy=ignore
# pcs property set default-resource-stickiness="200"
# pcs resource create gctvanas-vip ocf:heartbeat:IPaddr2 ip=10.30.96.100
cidr_netmask=32 nic=eth0 op monitor interval=30s
# pcs cluster cib drbd_cfg
# pcs -f drbd_cfg resource create gctvanas-fs1o ocf:linbit:drbd
drbd_resource=targetfs op monitor interval=10s
# pcs -f drbd_cfg resource master gctvanas-fs2o gctvanas-fs1o master-max=1
master-node-max=1 clone-max=2 clone-node-max=1 notify=true
# pcs cluster cib-push drbd_cfg
# pcs status
Cluster name: gctvanas
Stack: cman
Current DC: node1 (version 1.1.15-1.9a34920.git.el6-9a34920) - partition with
quorum
Last updated: Fri Aug 26 11:29:11 2016
Last change: Fri Aug 26 11:29:07 2016 by root via cibadmin on node1
2 nodes
