Re: [ClusterLabs] Removing a resource without stopping it
On Mon, 2022-01-31 at 10:37 +0100, Tomas Jelinek wrote: > Hi, > > The output you posted actually shows the procedure you followed > works. > Orphan resources are running resources which have no configuration > stored in CIB. Usually, they are stopped shortly after they are > removed > from CIB. If you set stop-orphan-resources to false, pacemaker won't > stop them. > > To summarize: > 1) Set stop-orphan-resources to false. This prevents pacemaker from > stopping a resource once it is removed from CIB. > 2) Run `pcs resource delete srv01-cs8 --force`. Using --force makes > pcs > not trying to stop the resource. Pcs directly removes the resource > from CIB. > 3) Confirm by running `pcs status`. If you see the resource is > orphaned, > it means the resource is still running after it has been removed > from > CIB and the only trace of it is in pacemaker state - pacemaker > remembers > it had started a resource which is now not present in CIB. And you can clear the state (to get rid of the orphan display) with "pcs resource refresh" > > In this stage, when you flip stop-orphan-resources to true, > pacemaker > stops the resource as there is no record of it in the CIB anymore > and > therefore no reason for it to be running. Once stopped, the resource > is > removed from pacemaker state as well. > > Not sure if this helps you with the actual migration, but hopefully > it > at least clarified a bit what's going on. > > > Regards, > Tomas > > > Dne 29. 01. 22 v 6:12 Digimer napsal(a): > > On 2022-01-29 00:10, Digimer wrote: > > > On 2022-01-28 16:54, Ken Gaillot wrote: > > > > On Fri, 2022-01-28 at 16:38 -0500, Digimer wrote: > > > > > Hi all, > > > > > > > > > > I'm trying to figure out how to move a running VM from > > > > > one > > > > > pacemaker > > > > > cluster to another. I've got the storage and VM live > > > > > migration > > > > > sorted, > > > > > but having trouble with pacemaker. > > > > > > > > > > I tried unmanaging the resource (the VM), then deleted > > > > > the > > > > > resource, > > > > > and the node got fenced. So I am assuming it thought it > > > > > couldn't > > > > > stop > > > > > the service so it self-fenced. In any case, can someone let > > > > > me know > > > > > what > > > > > the proper procedure is? > > > > > > > > > > Said more directly; > > > > > > > > > > How to I delete a resource from pacemaker (via pcs on > > > > > EL8) > > > > > without > > > > > stopping the resource? > > > > > > > > > Set the stop-orphan-resources cluster property to false (at > > > > least while > > > > you move it) > > > > > > > > The problem with your first approach is that once you remove > > > > the > > > > resource configuration, which includes the is-managed setting, > > > > Pacemaker no longer knows the resource is unmanaged. And even > > > > if you > > > > set it via resource defaults or something, eventually you have > > > > to set > > > > it back, at which point Pacemaker will still have the same > > > > response. > > > > > > Follow up; > > > > > > I tried to do the following sequence; > > > > > > > > > pcs property set stop-orphan-resources=false > > > pcs resource unmanage srv01-cs8 # Without this, > > > the > > > resource was stopped > > > pcs resource delete srv01-cs8 # Failed with > > > "Warning: 'srv01-cs8' is unmanaged" > > > pcs resource delete srv01-cs8 --force # Got 'Deleting > > > Resource - srv01-cs8' > > > pcs resource status > > > -- > > > * srv01-cs8 (ocf::alteeve:server): ORPHANED Started an- > > > a01n01 > > > (unmanaged) > > > -- > > > > > > > > > So it seems like this doesn't delete the resource. Can I get > > > some > > > insight on how to actually delete this resource without disabling > > > the VM? > > > > > > Thanks! > > > > > Adding; > > > > I tried 'pcs property set stop-orphan-resources=true' and it > > stopped the > > VM and then actually deleted the resource. =/ > > > > -- > > Digimer > > Papers and Projects:https://alteeve.com/w/ > > "I am, somehow, less interested in the weight and convolutions of > > Einstein’s brain than in the near certainty that people of equal > > talent have lived and died in cotton fields and sweatshops." - > > Stephen Jay Gould > > > > > > ___ > > Manage your subscription: > > https://lists.clusterlabs.org/mailman/listinfo/users > > > > ClusterLabs home: https://www.clusterlabs.org/ > > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ -- Ken Gaillot ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
Hi, On Sat, 29 Jan 2022 16:51:47 -0500 Digimer wrote: > ... > Though going back to the original question, deleting the server from > pacemaker while the VM is left running, is still something I am quite curious > about. As the real resource moved away, meaning it couldn't be stopped locally and it will be deleted shortly after, what about changing the on-fail property of the "stop" resource action to "ignore" ? https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Explained/singlehtml/index.html#operation-properties cf.: « ignore: Pretend the resource did not fail » ++ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
Hi, The output you posted actually shows the procedure you followed works. Orphan resources are running resources which have no configuration stored in CIB. Usually, they are stopped shortly after they are removed from CIB. If you set stop-orphan-resources to false, pacemaker won't stop them. To summarize: 1) Set stop-orphan-resources to false. This prevents pacemaker from stopping a resource once it is removed from CIB. 2) Run `pcs resource delete srv01-cs8 --force`. Using --force makes pcs not trying to stop the resource. Pcs directly removes the resource from CIB. 3) Confirm by running `pcs status`. If you see the resource is orphaned, it means the resource is still running after it has been removed from CIB and the only trace of it is in pacemaker state - pacemaker remembers it had started a resource which is now not present in CIB. In this stage, when you flip stop-orphan-resources to true, pacemaker stops the resource as there is no record of it in the CIB anymore and therefore no reason for it to be running. Once stopped, the resource is removed from pacemaker state as well. Not sure if this helps you with the actual migration, but hopefully it at least clarified a bit what's going on. Regards, Tomas Dne 29. 01. 22 v 6:12 Digimer napsal(a): On 2022-01-29 00:10, Digimer wrote: On 2022-01-28 16:54, Ken Gaillot wrote: On Fri, 2022-01-28 at 16:38 -0500, Digimer wrote: Hi all, I'm trying to figure out how to move a running VM from one pacemaker cluster to another. I've got the storage and VM live migration sorted, but having trouble with pacemaker. I tried unmanaging the resource (the VM), then deleted the resource, and the node got fenced. So I am assuming it thought it couldn't stop the service so it self-fenced. In any case, can someone let me know what the proper procedure is? Said more directly; How to I delete a resource from pacemaker (via pcs on EL8) without stopping the resource? Set the stop-orphan-resources cluster property to false (at least while you move it) The problem with your first approach is that once you remove the resource configuration, which includes the is-managed setting, Pacemaker no longer knows the resource is unmanaged. And even if you set it via resource defaults or something, eventually you have to set it back, at which point Pacemaker will still have the same response. Follow up; I tried to do the following sequence; pcs property set stop-orphan-resources=false pcs resource unmanage srv01-cs8 # Without this, the resource was stopped pcs resource delete srv01-cs8 # Failed with "Warning: 'srv01-cs8' is unmanaged" pcs resource delete srv01-cs8 --force # Got 'Deleting Resource - srv01-cs8' pcs resource status -- * srv01-cs8 (ocf::alteeve:server): ORPHANED Started an-a01n01 (unmanaged) -- So it seems like this doesn't delete the resource. Can I get some insight on how to actually delete this resource without disabling the VM? Thanks! Adding; I tried 'pcs property set stop-orphan-resources=true' and it stopped the VM and then actually deleted the resource. =/ -- Digimer Papers and Projects:https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
I think I have a working method, though not in the conditions first explained. Though I would love to have feedback on it's sanity. The ultimate goal is to migrate the resource (VM) to a different pacemaker cluster. Setting it to be unmanaged, migrating the VM off, setting the resource to disabled, and managing the resource again marks it as stopped, then it can be deleted. [root@an-a01n01 ~]# pcs resource unmanage srv01-cs8 # Migrate the server to another pacemaker cluster here [root@an-a01n01 ~]# pcs resource disable srv01-cs8 Warning: 'srv01-cs8' is unmanaged [root@an-a01n01 ~]# pcs resource manage srv01-cs8 [root@an-a01n01 ~]# pcs resource delete srv01-cs8 Deleting Resource - srv01-cs8 Though going back to the original question, deleting the server from pacemaker while the VM is left running, is still something I am quite curious about. Madi On 2022-01-29 13:27, Strahil Nikolov wrote: I know... and the editor stuff can be bypassed, if the approach works. Best Regards, Strahil Nikolov On Sat, Jan 29, 2022 at 15:43, Digimer wrote: On 2022-01-29 03:16, Strahil Nikolov wrote: I think there is pcs cluster edit --scope=resources (based on memory). Can you try to delete it from there ? Best Regards, Strahil Nikolov Thanks, but no that doesn't seem to work. 'pcs cluster edit' wants to open an editor, and I'm trying to find a way to make this change with a program (once I sort out the manual process). So an option that requires user input won't work in my case regardless. Thank you just the same though! -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
I know... and the editor stuff can be bypassed, if the approach works. Best Regards,Strahil Nikolov On Sat, Jan 29, 2022 at 15:43, Digimer wrote:On 2022-01-29 03:16, Strahil Nikolov wrote: I think there is pcs cluster edit --scope=resources (based on memory). Can you try to delete it from there ? Best Regards, Strahil Nikolov Thanks, but no that doesn't seem to work. 'pcs cluster edit' wants to open an editor, and I'm trying to find a way to make this change with a program (once I sort out the manual process). So an option that requires user input won't work in my case regardless. Thank you just the same though! -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
On 2022-01-29 03:16, Strahil Nikolov wrote: I think there is pcs cluster edit --scope=resources (based on memory). Can you try to delete it from there ? Best Regards, Strahil Nikolov Thanks, but no that doesn't seem to work. 'pcs cluster edit' wants to open an editor, and I'm trying to find a way to make this change with a program (once I sort out the manual process). So an option that requires user input won't work in my case regardless. Thank you just the same though! -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
I think there is pcs cluster edit --scope=resources (based on memory).Can you try to delete it from there ? Best Regards,Strahil Nikolov On Sat, Jan 29, 2022 at 7:12, Digimer wrote: ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
On 2022-01-29 00:10, Digimer wrote: On 2022-01-28 16:54, Ken Gaillot wrote: On Fri, 2022-01-28 at 16:38 -0500, Digimer wrote: Hi all, I'm trying to figure out how to move a running VM from one pacemaker cluster to another. I've got the storage and VM live migration sorted, but having trouble with pacemaker. I tried unmanaging the resource (the VM), then deleted the resource, and the node got fenced. So I am assuming it thought it couldn't stop the service so it self-fenced. In any case, can someone let me know what the proper procedure is? Said more directly; How to I delete a resource from pacemaker (via pcs on EL8) without stopping the resource? Set the stop-orphan-resources cluster property to false (at least while you move it) The problem with your first approach is that once you remove the resource configuration, which includes the is-managed setting, Pacemaker no longer knows the resource is unmanaged. And even if you set it via resource defaults or something, eventually you have to set it back, at which point Pacemaker will still have the same response. Follow up; I tried to do the following sequence; pcs property set stop-orphan-resources=false pcs resource unmanage srv01-cs8 # Without this, the resource was stopped pcs resource delete srv01-cs8 # Failed with "Warning: 'srv01-cs8' is unmanaged" pcs resource delete srv01-cs8 --force # Got 'Deleting Resource - srv01-cs8' pcs resource status -- * srv01-cs8 (ocf::alteeve:server): ORPHANED Started an-a01n01 (unmanaged) -- So it seems like this doesn't delete the resource. Can I get some insight on how to actually delete this resource without disabling the VM? Thanks! Adding; I tried 'pcs property set stop-orphan-resources=true' and it stopped the VM and then actually deleted the resource. =/ -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
On 2022-01-28 16:54, Ken Gaillot wrote: On Fri, 2022-01-28 at 16:38 -0500, Digimer wrote: Hi all, I'm trying to figure out how to move a running VM from one pacemaker cluster to another. I've got the storage and VM live migration sorted, but having trouble with pacemaker. I tried unmanaging the resource (the VM), then deleted the resource, and the node got fenced. So I am assuming it thought it couldn't stop the service so it self-fenced. In any case, can someone let me know what the proper procedure is? Said more directly; How to I delete a resource from pacemaker (via pcs on EL8) without stopping the resource? Set the stop-orphan-resources cluster property to false (at least while you move it) The problem with your first approach is that once you remove the resource configuration, which includes the is-managed setting, Pacemaker no longer knows the resource is unmanaged. And even if you set it via resource defaults or something, eventually you have to set it back, at which point Pacemaker will still have the same response. Follow up; I tried to do the following sequence; pcs property set stop-orphan-resources=false pcs resource unmanage srv01-cs8 # Without this, the resource was stopped pcs resource delete srv01-cs8 # Failed with "Warning: 'srv01-cs8' is unmanaged" pcs resource delete srv01-cs8 --force # Got 'Deleting Resource - srv01-cs8' pcs resource status -- * srv01-cs8 (ocf::alteeve:server): ORPHANED Started an-a01n01 (unmanaged) -- So it seems like this doesn't delete the resource. Can I get some insight on how to actually delete this resource without disabling the VM? Thanks! digimer -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
On 2022-01-28 16:54, Ken Gaillot wrote: On Fri, 2022-01-28 at 16:38 -0500, Digimer wrote: Hi all, I'm trying to figure out how to move a running VM from one pacemaker cluster to another. I've got the storage and VM live migration sorted, but having trouble with pacemaker. I tried unmanaging the resource (the VM), then deleted the resource, and the node got fenced. So I am assuming it thought it couldn't stop the service so it self-fenced. In any case, can someone let me know what the proper procedure is? Said more directly; How to I delete a resource from pacemaker (via pcs on EL8) without stopping the resource? Set the stop-orphan-resources cluster property to false (at least while you move it) The problem with your first approach is that once you remove the resource configuration, which includes the is-managed setting, Pacemaker no longer knows the resource is unmanaged. And even if you set it via resource defaults or something, eventually you have to set it back, at which point Pacemaker will still have the same response. Thanks for this! I'm not entirely sure I understand the implications of "stop-orphan-resources". I assume it would be a bad idea to set it to "false" and leave it that way, given it's not the default. What's the purpose of this being set to 'true'? Thanks! -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Removing a resource without stopping it
On Fri, 2022-01-28 at 16:38 -0500, Digimer wrote: > Hi all, > >I'm trying to figure out how to move a running VM from one > pacemaker > cluster to another. I've got the storage and VM live migration > sorted, > but having trouble with pacemaker. > >I tried unmanaging the resource (the VM), then deleted the > resource, > and the node got fenced. So I am assuming it thought it couldn't > stop > the service so it self-fenced. In any case, can someone let me know > what > the proper procedure is? > >Said more directly; > >How to I delete a resource from pacemaker (via pcs on EL8) > without > stopping the resource? > Set the stop-orphan-resources cluster property to false (at least while you move it) The problem with your first approach is that once you remove the resource configuration, which includes the is-managed setting, Pacemaker no longer knows the resource is unmanaged. And even if you set it via resource defaults or something, eventually you have to set it back, at which point Pacemaker will still have the same response. -- Ken Gaillot ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
