Re: [Pacemaker] Proper way to migrate multistate resource?
On Thu, Feb 9, 2012 at 4:18 PM, Dejan Muhamedagic wrote: > Hi, > > On Tue, Feb 07, 2012 at 01:54:44PM +0100, Lars Ellenberg wrote: >> On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: >> > Greetings, >> > >> > I'm some what new to pacemaker and have been playing around with a >> > number of configurations in a lab. Most recently I've been testing a >> > multistate resource using the ofc:pacemaker:Stateful example RA. >> > >> > While I've gotten the agent to work and notice that if I shutdown or >> > kill a node the resources migrate I can't seem to figure out the >> > proper way to migrate the resource between nodes when they are both >> > up. >> > >> > For regular resources I've used "crm resource migrate " without >> > issue. However when I try this with a multistate resource it doesn't >> > seem to work. When I run the command it just puts the slave node into >> > a stopped state. If I try and tell it to migrate specifically to the >> > slave node it claims to already be running their (which I suppose in a >> > sense it is). >> >> the crm shell does not support roles for the "move" or "migrate" command >> (yet; maybe in newer versions. Dejan?). > > No, it doesn't. The crm resource commands rely on the underlying > tools (crm_resource, et al). crm_resource is probably the right > place to implement such a functionality, then we can easily add > support to the shell as well. Patches accepted :) Another place is the GUI, that already has this functionality. :) Rasto -- Dipl.-Ing. Rastislav Levrinc rasto.levr...@gmail.com Linux Cluster Management Console http://lcmc.sf.net/ ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
Hi, On Tue, Feb 07, 2012 at 01:54:44PM +0100, Lars Ellenberg wrote: > On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: > > Greetings, > > > > I'm some what new to pacemaker and have been playing around with a > > number of configurations in a lab. Most recently I've been testing a > > multistate resource using the ofc:pacemaker:Stateful example RA. > > > > While I've gotten the agent to work and notice that if I shutdown or > > kill a node the resources migrate I can't seem to figure out the > > proper way to migrate the resource between nodes when they are both > > up. > > > > For regular resources I've used "crm resource migrate " without > > issue. However when I try this with a multistate resource it doesn't > > seem to work. When I run the command it just puts the slave node into > > a stopped state. If I try and tell it to migrate specifically to the > > slave node it claims to already be running their (which I suppose in a > > sense it is). > > the crm shell does not support roles for the "move" or "migrate" command > (yet; maybe in newer versions. Dejan?). No, it doesn't. The crm resource commands rely on the underlying tools (crm_resource, et al). crm_resource is probably the right place to implement such a functionality, then we can easily add support to the shell as well. Patches accepted :) Cheers, Dejan ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
On Feb 8, 2012, at 15:34 , Andrew Beekhof wrote: > On Thu, Feb 9, 2012 at 9:58 AM, Chet Burgess wrote: >> >> >> >> On Feb 7, 2012, at 4:54 , Lars Ellenberg wrote: >> >>> On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: Greetings, I'm some what new to pacemaker and have been playing around with a number of configurations in a lab. Most recently I've been testing a multistate resource using the ofc:pacemaker:Stateful example RA. While I've gotten the agent to work and notice that if I shutdown or kill a node the resources migrate I can't seem to figure out the proper way to migrate the resource between nodes when they are both up. For regular resources I've used "crm resource migrate " without issue. However when I try this with a multistate resource it doesn't seem to work. When I run the command it just puts the slave node into a stopped state. If I try and tell it to migrate specifically to the slave node it claims to already be running their (which I suppose in a sense it is). >>> >>> the crm shell does not support roles for the "move" or "migrate" command >>> (yet; maybe in newer versions. Dejan?). >>> >>> What you need to do is set a location constraint on the role. >>> * force master role off from one node: >>> >>> location you-name-it resource-id \ >>> rule $role=Master -inf: \ >>> #uname eq node-where-it-should-be-slave >>> >>> * or force master role off from all but one node, >>> note the double negation in this one: >>> >>> location you-name-it resource-id \ >>> rule $role=Master -inf: \ >>> #uname ne node-where-it-should-be-master >>> >> >>OK I finally got around to testing this today. Both scenarios >> described above worked as described. Thank you. >> >>One minor hitch though. When I remove the constraint the resource >> goes back to the original (and preferred by the configuration) node. With my >> non-multistate resources when I remove the constraint the resource remains >> on the node it was moved to until it is forced to move again with another >> constraint (eg. what "crm resource migrate" does). >> >>I have the default resource-stickiness set to 100 > > Doesn't affect promotion I'm afraid. > One could argue that it should though. > At least I'm not crazy. So I take it that there is no way to "stick" a master role to the "last node" that was in that state? One the preferred node becoming available again the resource will be auto-magically moved back? -- Chet Burgess c...@liquidreality.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
On Thu, Feb 9, 2012 at 9:58 AM, Chet Burgess wrote: > > > > On Feb 7, 2012, at 4:54 , Lars Ellenberg wrote: > >> On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: >>> Greetings, >>> >>> I'm some what new to pacemaker and have been playing around with a >>> number of configurations in a lab. Most recently I've been testing a >>> multistate resource using the ofc:pacemaker:Stateful example RA. >>> >>> While I've gotten the agent to work and notice that if I shutdown or >>> kill a node the resources migrate I can't seem to figure out the >>> proper way to migrate the resource between nodes when they are both >>> up. >>> >>> For regular resources I've used "crm resource migrate " without >>> issue. However when I try this with a multistate resource it doesn't >>> seem to work. When I run the command it just puts the slave node into >>> a stopped state. If I try and tell it to migrate specifically to the >>> slave node it claims to already be running their (which I suppose in a >>> sense it is). >> >> the crm shell does not support roles for the "move" or "migrate" command >> (yet; maybe in newer versions. Dejan?). >> >> What you need to do is set a location constraint on the role. >> * force master role off from one node: >> >> location you-name-it resource-id \ >> rule $role=Master -inf: \ >> #uname eq node-where-it-should-be-slave >> >> * or force master role off from all but one node, >> note the double negation in this one: >> >> location you-name-it resource-id \ >> rule $role=Master -inf: \ >> #uname ne node-where-it-should-be-master >> > > OK I finally got around to testing this today. Both scenarios > described above worked as described. Thank you. > > One minor hitch though. When I remove the constraint the resource goes > back to the original (and preferred by the configuration) node. With my > non-multistate resources when I remove the constraint the resource remains on > the node it was moved to until it is forced to move again with another > constraint (eg. what "crm resource migrate" does). > > I have the default resource-stickiness set to 100 Doesn't affect promotion I'm afraid. One could argue that it should though. > and my location constraints for setting the preferred node use 10 for the > preferred node and 0 for all other nodes. I tried setting > resource-stickiness to 100 directly on the primitive for the resource and on > the ms definition of the resource but still had the same results. I event > tried setting the resource-stickiness to infinity with no result. > > So clearly my understanding of the scoring is somewhat flawed. I've > added my sample config again at the end of the message. Any pointers as to > what I'm doing wrong with my scoring and resource-stickiness? Happy to > provide any other details, or output from logs if needed. > > Thank you again for your assistance. > > > crm configure show: > > node tst3.local1.mc.metacloud.com > node tst4.local1.mc.metacloud.com > primitive stateful-test ocf:pacemaker:Stateful \ > op monitor interval="30s" role="Slave" \ > op monitor interval="31s" role="Master" > ms ms-test stateful-test \ > meta clone-node-max="1" notify="false" master-max="1" > master-node-max="1" > location ms-test_constraint_1 ms-test 10: tst3.local1.mc.metacloud.com > location ms-test_constraint_2 ms-test 0: tst4.local1.mc.metacloud.com > property $id="cib-bootstrap-options" \ > cluster-infrastructure="openais" \ > dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \ > last-lrm-refresh="1325273678" \ > expected-quorum-votes="2" \ > no-quorum-policy="ignore" \ > stonith-enabled="false" > rsc_defaults $id="rsc-options" \ > resource-stickiness="100" > > > -- > Chet Burgess > c...@liquidreality.org > > > ___ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
On Feb 7, 2012, at 4:54 , Lars Ellenberg wrote: > On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: >> Greetings, >> >> I'm some what new to pacemaker and have been playing around with a >> number of configurations in a lab. Most recently I've been testing a >> multistate resource using the ofc:pacemaker:Stateful example RA. >> >> While I've gotten the agent to work and notice that if I shutdown or >> kill a node the resources migrate I can't seem to figure out the >> proper way to migrate the resource between nodes when they are both >> up. >> >> For regular resources I've used "crm resource migrate " without >> issue. However when I try this with a multistate resource it doesn't >> seem to work. When I run the command it just puts the slave node into >> a stopped state. If I try and tell it to migrate specifically to the >> slave node it claims to already be running their (which I suppose in a >> sense it is). > > the crm shell does not support roles for the "move" or "migrate" command > (yet; maybe in newer versions. Dejan?). > > What you need to do is set a location constraint on the role. > * force master role off from one node: > > location you-name-it resource-id \ > rule $role=Master -inf: \ > #uname eq node-where-it-should-be-slave > > * or force master role off from all but one node, > note the double negation in this one: > > location you-name-it resource-id \ > rule $role=Master -inf: \ > #uname ne node-where-it-should-be-master > OK I finally got around to testing this today. Both scenarios described above worked as described. Thank you. One minor hitch though. When I remove the constraint the resource goes back to the original (and preferred by the configuration) node. With my non-multistate resources when I remove the constraint the resource remains on the node it was moved to until it is forced to move again with another constraint (eg. what "crm resource migrate" does). I have the default resource-stickiness set to 100 and my location constraints for setting the preferred node use 10 for the preferred node and 0 for all other nodes. I tried setting resource-stickiness to 100 directly on the primitive for the resource and on the ms definition of the resource but still had the same results. I event tried setting the resource-stickiness to infinity with no result. So clearly my understanding of the scoring is somewhat flawed. I've added my sample config again at the end of the message. Any pointers as to what I'm doing wrong with my scoring and resource-stickiness? Happy to provide any other details, or output from logs if needed. Thank you again for your assistance. crm configure show: node tst3.local1.mc.metacloud.com node tst4.local1.mc.metacloud.com primitive stateful-test ocf:pacemaker:Stateful \ op monitor interval="30s" role="Slave" \ op monitor interval="31s" role="Master" ms ms-test stateful-test \ meta clone-node-max="1" notify="false" master-max="1" master-node-max="1" location ms-test_constraint_1 ms-test 10: tst3.local1.mc.metacloud.com location ms-test_constraint_2 ms-test 0: tst4.local1.mc.metacloud.com property $id="cib-bootstrap-options" \ cluster-infrastructure="openais" \ dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \ last-lrm-refresh="1325273678" \ expected-quorum-votes="2" \ no-quorum-policy="ignore" \ stonith-enabled="false" rsc_defaults $id="rsc-options" \ resource-stickiness="100" -- Chet Burgess c...@liquidreality.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
OK that makes sense. I will give that a try. Thank you. -- Chet Burgess c...@liquidreality.org On Feb 7, 2012, at 5:08 , Lars Ellenberg wrote: > On Tue, Feb 07, 2012 at 02:03:32PM +0100, Michael Schwartzkopff wrote: >>> On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: Greetings, I'm some what new to pacemaker and have been playing around with a number of configurations in a lab. Most recently I've been testing a multistate resource using the ofc:pacemaker:Stateful example RA. While I've gotten the agent to work and notice that if I shutdown or kill a node the resources migrate I can't seem to figure out the proper way to migrate the resource between nodes when they are both up. For regular resources I've used "crm resource migrate " without issue. However when I try this with a multistate resource it doesn't seem to work. When I run the command it just puts the slave node into a stopped state. If I try and tell it to migrate specifically to the slave node it claims to already be running their (which I suppose in a sense it is). >>> >>> the crm shell does not support roles for the "move" or "migrate" command >>> (yet; maybe in newer versions. Dejan?). >>> >>> What you need to do is set a location constraint on the role. >>> * force master role off from one node: >>> >>> location you-name-it resource-id \ >>> rule $role=Master -inf: \ >>> #uname eq node-where-it-should-be-slave >>> >>> * or force master role off from all but one node, >>> note the double negation in this one: >>> >>> location you-name-it resource-id \ >>> rule $role=Master -inf: \ >>> #uname ne node-where-it-should-be-master >> >> These constraints would prevent the MS resource to run in Master state even >> on >> that node. Even in case the preferred node is not available any more. This >> might be not what Chet wanted. > > Well, it is just what crm resource migrate does, otherwise. > > After migration, you obviously need to "unmigrate", > i.e. delete that constraint again. > > > -- > : Lars Ellenberg > : LINBIT | Your Way to High Availability > : DRBD/HA support and consulting http://www.linbit.com > > DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. > > ___ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
On Tue, Feb 07, 2012 at 02:03:32PM +0100, Michael Schwartzkopff wrote: > > On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: > > > Greetings, > > > > > > I'm some what new to pacemaker and have been playing around with a > > > number of configurations in a lab. Most recently I've been testing a > > > multistate resource using the ofc:pacemaker:Stateful example RA. > > > > > > While I've gotten the agent to work and notice that if I shutdown or > > > kill a node the resources migrate I can't seem to figure out the > > > proper way to migrate the resource between nodes when they are both > > > up. > > > > > > For regular resources I've used "crm resource migrate " without > > > issue. However when I try this with a multistate resource it doesn't > > > seem to work. When I run the command it just puts the slave node into > > > a stopped state. If I try and tell it to migrate specifically to the > > > slave node it claims to already be running their (which I suppose in a > > > sense it is). > > > > the crm shell does not support roles for the "move" or "migrate" command > > (yet; maybe in newer versions. Dejan?). > > > > What you need to do is set a location constraint on the role. > > * force master role off from one node: > > > > location you-name-it resource-id \ > > rule $role=Master -inf: \ > > #uname eq node-where-it-should-be-slave > > > > * or force master role off from all but one node, > >note the double negation in this one: > > > > location you-name-it resource-id \ > > rule $role=Master -inf: \ > > #uname ne node-where-it-should-be-master > > These constraints would prevent the MS resource to run in Master state even > on > that node. Even in case the preferred node is not available any more. This > might be not what Chet wanted. Well, it is just what crm resource migrate does, otherwise. After migration, you obviously need to "unmigrate", i.e. delete that constraint again. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
> On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: > > Greetings, > > > > I'm some what new to pacemaker and have been playing around with a > > number of configurations in a lab. Most recently I've been testing a > > multistate resource using the ofc:pacemaker:Stateful example RA. > > > > While I've gotten the agent to work and notice that if I shutdown or > > kill a node the resources migrate I can't seem to figure out the > > proper way to migrate the resource between nodes when they are both > > up. > > > > For regular resources I've used "crm resource migrate " without > > issue. However when I try this with a multistate resource it doesn't > > seem to work. When I run the command it just puts the slave node into > > a stopped state. If I try and tell it to migrate specifically to the > > slave node it claims to already be running their (which I suppose in a > > sense it is). > > the crm shell does not support roles for the "move" or "migrate" command > (yet; maybe in newer versions. Dejan?). > > What you need to do is set a location constraint on the role. > * force master role off from one node: > > location you-name-it resource-id \ > rule $role=Master -inf: \ > #uname eq node-where-it-should-be-slave > > * or force master role off from all but one node, >note the double negation in this one: > > location you-name-it resource-id \ > rule $role=Master -inf: \ > #uname ne node-where-it-should-be-master These constraints would prevent the MS resource to run in Master state even on that node. Even in case the preferred node is not available any more. This might be not what Chet wanted. Perhaps it would be easier if you give the resource some points if it runs in Master state on the preferred node: location name-of-your-constraint resource-id \ rule $role=Master 100: \ #uname eq name-of-the-preferred-node -- Dr. Michael Schwartzkopff Guardinistr. 63 81375 München Tel: (0163) 172 50 98 Fax: (089) 620 304 13 signature.asc Description: This is a digitally signed message part. ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Proper way to migrate multistate resource?
On Mon, Feb 06, 2012 at 04:48:26PM -0800, Chet Burgess wrote: > Greetings, > > I'm some what new to pacemaker and have been playing around with a > number of configurations in a lab. Most recently I've been testing a > multistate resource using the ofc:pacemaker:Stateful example RA. > > While I've gotten the agent to work and notice that if I shutdown or > kill a node the resources migrate I can't seem to figure out the > proper way to migrate the resource between nodes when they are both > up. > > For regular resources I've used "crm resource migrate " without > issue. However when I try this with a multistate resource it doesn't > seem to work. When I run the command it just puts the slave node into > a stopped state. If I try and tell it to migrate specifically to the > slave node it claims to already be running their (which I suppose in a > sense it is). the crm shell does not support roles for the "move" or "migrate" command (yet; maybe in newer versions. Dejan?). What you need to do is set a location constraint on the role. * force master role off from one node: location you-name-it resource-id \ rule $role=Master -inf: \ #uname eq node-where-it-should-be-slave * or force master role off from all but one node, note the double negation in this one: location you-name-it resource-id \ rule $role=Master -inf: \ #uname ne node-where-it-should-be-master Cheers, Lars --- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com > The only method I've found to safely and reliable migrate a multistate > resource from one node to another is I think it has something to do > with the resource constraints I used to prefer a particular node, but > I'm not entirely sure how the constraints and the master/slave state > updating stuff works. > > Am I using the wrong tool to migrate a multistate resource or is my > configuration wrong in some way? Any input greatly appreciated. > Thank you. > > > Configuration: > r...@tst3.local1.mc:/home/cfb$ crm configure show > node tst3.local1.mc.metacloud.com > node tst4.local1.mc.metacloud.com > primitive stateful-test ocf:pacemaker:Stateful \ > op monitor interval="30s" role="Slave" \ > op monitor interval="31s" role="Master" > ms ms-test stateful-test \ > meta clone-node-max="1" notify="false" master-max="1" > master-node-max="1" target-role="Master" > location ms-test_constraint_1 ms-test 25: tst3.local1.mc.metacloud.com > location ms-test_constraint_2 ms-test 20: tst4.local1.mc.metacloud.com > property $id="cib-bootstrap-options" \ > cluster-infrastructure="openais" \ > dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \ > last-lrm-refresh="1325273678" \ > expected-quorum-votes="2" \ > no-quorum-policy="ignore" \ > stonith-enabled="false" > rsc_defaults $id="rsc-options" \ > resource-stickiness="100" > > -- > Chet Burgess > c...@liquidreality.org > > > > > ___ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
[Pacemaker] Proper way to migrate multistate resource?
Greetings, I'm some what new to pacemaker and have been playing around with a number of configurations in a lab. Most recently I've been testing a multistate resource using the ofc:pacemaker:Stateful example RA. While I've gotten the agent to work and notice that if I shutdown or kill a node the resources migrate I can't seem to figure out the proper way to migrate the resource between nodes when they are both up. For regular resources I've used "crm resource migrate " without issue. However when I try this with a multistate resource it doesn't seem to work. When I run the command it just puts the slave node into a stopped state. If I try and tell it to migrate specifically to the slave node it claims to already be running their (which I suppose in a sense it is). The only method I've found to safely and reliable migrate a multistate resource from one node to another is I think it has something to do with the resource constraints I used to prefer a particular node, but I'm not entirely sure how the constraints and the master/slave state updating stuff works. Am I using the wrong tool to migrate a multistate resource or is my configuration wrong in some way? Any input greatly appreciated. Thank you. Configuration: r...@tst3.local1.mc:/home/cfb$ crm configure show node tst3.local1.mc.metacloud.com node tst4.local1.mc.metacloud.com primitive stateful-test ocf:pacemaker:Stateful \ op monitor interval="30s" role="Slave" \ op monitor interval="31s" role="Master" ms ms-test stateful-test \ meta clone-node-max="1" notify="false" master-max="1" master-node-max="1" target-role="Master" location ms-test_constraint_1 ms-test 25: tst3.local1.mc.metacloud.com location ms-test_constraint_2 ms-test 20: tst4.local1.mc.metacloud.com property $id="cib-bootstrap-options" \ cluster-infrastructure="openais" \ dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \ last-lrm-refresh="1325273678" \ expected-quorum-votes="2" \ no-quorum-policy="ignore" \ stonith-enabled="false" rsc_defaults $id="rsc-options" \ resource-stickiness="100" -- Chet Burgess c...@liquidreality.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org