RE: Re[2]: High Availability Cloudstack
I recall in the 4.3 release that HA on KVM did not function for loss of a host. The management log would report that it was 'investigating' for days without taking action. The UI would report the host as up at this time as well. Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: Remi Bergsma [mailto:rberg...@schubergphilis.com] Sent: 10 April 2016 10:34 To: users@cloudstack.apache.org Subject: Re: Re[2]: High Availability Cloudstack Hi, I have tested this multiple times with a multi-node kvm cluster. When you kill one hypervisor (and it will not come back) then HA kicks in as expected and VMs with HA but set are started again. Quite sure all versions since 4.6 work fine, most likely older versions as well. Very simple to test, just give it a try. Maybe we should write a Marvin integration test to prove it works as people keep saying it doesn't for some reason. Regards, Remi > On 08 Apr 2016, at 04:43, Indra Pramana wrote: > > Hi Simon, > > Apologise for replying to an old thread - may I know if you have > managed to test this out on your lab? How was the result? > > Thank you. > > >> On Thu, Feb 4, 2016 at 3:05 AM, Simon Weller wrote: >> >> i'll test it out in our lab 4.7.1 lab and see what happens. >> >> >> From: Nux! >> Sent: Wednesday, February 3, 2016 12:34 PM >> To: users@cloudstack.apache.org >> Cc: Mohd Zainal Abidin Rabani >> Subject: Re: Re[2]: High Availability Cloudstack >> >> Simon, >> >> I think this is no longer correct. After a certain timeout the >> management server will bring up the VMs on another hypervisor. We >> discussed this in the somewhat recent past, but can't find the thread right >> now. >> There was some nice insight from Wido or Marcus. >> >> Lucian >> >> -- >> Sent from the Delta quadrant using Borg technology! >> >> Nux! >> www.nux.ro >> >> - Original Message - >>> From: "Simon Weller" >>> To: "users" >>> Cc: "Mohd Zainal Abidin Rabani" >>> Sent: Wednesday, 3 February, 2016 16:23:55 >>> Subject: Re: Re[2]: High Availability Cloudstack >> >>> My understanding is that HA will never actually function on KVM >>> there >> way most >>> expect, unless the original host that failed comes back up and the >>> agents report the VMs not running. This is due to Cloudstack not >>> wanting to risk starting a vm that already is locked by another >>> process, on a different >> server. >>> >>> The only way I know of to force HA to function is to remove the host >>> out >> of >>> CloudStack, to force CloudStack to reallocate the VMs. >>> >>> There has been a lot of talk about introducing a reliable fencing >> mechanism into >>> management, so that CloudStack management is 100% confident the vm >>> is >> safe to >>> restart on a new host. >>> >>> The current proposal for that is here: >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IP >> MI+Fencing >>> >>> - Si >>> >>> >>> >>> >>> From: Andrei Mikhailovsky >>> Sent: Wednesday, February 3, 2016 1:49 AM >>> To: users >>> Cc: Mohd Zainal Abidin Rabani >>> Subject: Re: Re[2]: High Availability Cloudstack >>> >>> Hi, >>> >>> from what i can can see, this issue has been resolved in version 4.4 >>> and >> Mohd is >>> running 4.6. >>> >>> From my testing with KVM, the management server performs a poll once >>> in >> a while >>> and the agent replies if a vm is up. It might take some time before >>> the >> state >>> changes, like 30 seconds or so, so give it a bit more time. My >>> testing >> show >>> that the management server does update the state if the vm is >>> manually >> powered >>> down. >>> >>> I've not tested tha HA feature for sometime, but last time i checked >>> it >> was >>> working. You should also set the HA option in the vm's system offering. >>> >>> Andrei >>> - Original Message - >>>> From: "Suresh Sadhu" >>>> To: "users&qu
Re: Re[2]: High Availability Cloudstack
Hi, I have tested this multiple times with a multi-node kvm cluster. When you kill one hypervisor (and it will not come back) then HA kicks in as expected and VMs with HA but set are started again. Quite sure all versions since 4.6 work fine, most likely older versions as well. Very simple to test, just give it a try. Maybe we should write a Marvin integration test to prove it works as people keep saying it doesn't for some reason. Regards, Remi > On 08 Apr 2016, at 04:43, Indra Pramana wrote: > > Hi Simon, > > Apologise for replying to an old thread - may I know if you have managed to > test this out on your lab? How was the result? > > Thank you. > > >> On Thu, Feb 4, 2016 at 3:05 AM, Simon Weller wrote: >> >> i'll test it out in our lab 4.7.1 lab and see what happens. >> >> >> From: Nux! >> Sent: Wednesday, February 3, 2016 12:34 PM >> To: users@cloudstack.apache.org >> Cc: Mohd Zainal Abidin Rabani >> Subject: Re: Re[2]: High Availability Cloudstack >> >> Simon, >> >> I think this is no longer correct. After a certain timeout the management >> server will bring up the VMs on another hypervisor. We discussed this in >> the somewhat recent past, but can't find the thread right now. >> There was some nice insight from Wido or Marcus. >> >> Lucian >> >> -- >> Sent from the Delta quadrant using Borg technology! >> >> Nux! >> www.nux.ro >> >> - Original Message - >>> From: "Simon Weller" >>> To: "users" >>> Cc: "Mohd Zainal Abidin Rabani" >>> Sent: Wednesday, 3 February, 2016 16:23:55 >>> Subject: Re: Re[2]: High Availability Cloudstack >> >>> My understanding is that HA will never actually function on KVM there >> way most >>> expect, unless the original host that failed comes back up and the agents >>> report the VMs not running. This is due to Cloudstack not wanting to risk >>> starting a vm that already is locked by another process, on a different >> server. >>> >>> The only way I know of to force HA to function is to remove the host out >> of >>> CloudStack, to force CloudStack to reallocate the VMs. >>> >>> There has been a lot of talk about introducing a reliable fencing >> mechanism into >>> management, so that CloudStack management is 100% confident the vm is >> safe to >>> restart on a new host. >>> >>> The current proposal for that is here: >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IPMI+Fencing >>> >>> - Si >>> >>> >>> >>> >>> From: Andrei Mikhailovsky >>> Sent: Wednesday, February 3, 2016 1:49 AM >>> To: users >>> Cc: Mohd Zainal Abidin Rabani >>> Subject: Re: Re[2]: High Availability Cloudstack >>> >>> Hi, >>> >>> from what i can can see, this issue has been resolved in version 4.4 and >> Mohd is >>> running 4.6. >>> >>> From my testing with KVM, the management server performs a poll once in >> a while >>> and the agent replies if a vm is up. It might take some time before the >> state >>> changes, like 30 seconds or so, so give it a bit more time. My testing >> show >>> that the management server does update the state if the vm is manually >> powered >>> down. >>> >>> I've not tested tha HA feature for sometime, but last time i checked it >> was >>> working. You should also set the HA option in the vm's system offering. >>> >>> Andrei >>> - Original Message - >>>> From: "Suresh Sadhu" >>>> To: "users" , "Mohd Zainal Abidin Rabani" >>>> >>>> Sent: Wednesday, 3 February, 2016 06:24:49 >>>> Subject: RE: Re[2]: High Availability Cloudstack >>> >>>> Please see below issue: >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-5582 >>>> >>>> regards >>>> sadhu >>>> >>>> -Original Message- >>>> From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] >>>> Sent: Wednesday, February 3, 2016 10:12 AM >>>> To: users@cloudstack.apache.org >>>> Subject: Re[2]: High Availability Cloudstack >>>> >>>> I'm using KVM that come with centos 7. Th
Re: Re[2]: High Availability Cloudstack
Hi Indra, Unfortunately, we haven't gotten around to this yet. We're currently working on validating PRs for 4.9. Once we're through that, I'll get some of my team to focus on testing this. - Si From: Indra Pramana Sent: Thursday, April 7, 2016 9:42 PM To: users@cloudstack.apache.org Subject: Re: Re[2]: High Availability Cloudstack Hi Simon, Apologise for replying to an old thread - may I know if you have managed to test this out on your lab? How was the result? Thank you. On Thu, Feb 4, 2016 at 3:05 AM, Simon Weller wrote: > i'll test it out in our lab 4.7.1 lab and see what happens. > > > From: Nux! > Sent: Wednesday, February 3, 2016 12:34 PM > To: users@cloudstack.apache.org > Cc: Mohd Zainal Abidin Rabani > Subject: Re: Re[2]: High Availability Cloudstack > > Simon, > > I think this is no longer correct. After a certain timeout the management > server will bring up the VMs on another hypervisor. We discussed this in > the somewhat recent past, but can't find the thread right now. > There was some nice insight from Wido or Marcus. > > Lucian > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro > > - Original Message - > > From: "Simon Weller" > > To: "users" > > Cc: "Mohd Zainal Abidin Rabani" > > Sent: Wednesday, 3 February, 2016 16:23:55 > > Subject: Re: Re[2]: High Availability Cloudstack > > > My understanding is that HA will never actually function on KVM there > way most > > expect, unless the original host that failed comes back up and the agents > > report the VMs not running. This is due to Cloudstack not wanting to risk > > starting a vm that already is locked by another process, on a different > server. > > > > The only way I know of to force HA to function is to remove the host out > of > > CloudStack, to force CloudStack to reallocate the VMs. > > > > There has been a lot of talk about introducing a reliable fencing > mechanism into > > management, so that CloudStack management is 100% confident the vm is > safe to > > restart on a new host. > > > > The current proposal for that is here: > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IPMI+Fencing > > > > - Si > > > > > > > > > > From: Andrei Mikhailovsky > > Sent: Wednesday, February 3, 2016 1:49 AM > > To: users > > Cc: Mohd Zainal Abidin Rabani > > Subject: Re: Re[2]: High Availability Cloudstack > > > > Hi, > > > > from what i can can see, this issue has been resolved in version 4.4 and > Mohd is > > running 4.6. > > > > From my testing with KVM, the management server performs a poll once in > a while > > and the agent replies if a vm is up. It might take some time before the > state > > changes, like 30 seconds or so, so give it a bit more time. My testing > show > > that the management server does update the state if the vm is manually > powered > > down. > > > > I've not tested tha HA feature for sometime, but last time i checked it > was > > working. You should also set the HA option in the vm's system offering. > > > > Andrei > > - Original Message - > >> From: "Suresh Sadhu" > >> To: "users" , "Mohd Zainal Abidin Rabani" > >> > >> Sent: Wednesday, 3 February, 2016 06:24:49 > >> Subject: RE: Re[2]: High Availability Cloudstack > > > >> Please see below issue: > >> https://issues.apache.org/jira/browse/CLOUDSTACK-5582 > >> > >> regards > >> sadhu > >> > >> -Original Message- > >> From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] > >> Sent: Wednesday, February 3, 2016 10:12 AM > >> To: users@cloudstack.apache.org > >> Subject: Re[2]: High Availability Cloudstack > >> > >> I'm using KVM that come with centos 7. That management also using > centos 7. > >> Cloudstack version is 4.6. > >> > >> - > >> Regards, > >> Mohd Zainal Abidin Rabani > >> Technical Support > >> > >> ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, > Kajang > >> Sentral, 43000 Kajang, Selangor. > >> T : 03-8737 0030 | F : 03-8737 0070 > >> E : zai...@nocser.net > >> W : www.modern.com.my | www.nocser.net > >> > >> > &g
Re: Re[2]: High Availability Cloudstack
Hi Simon, Apologise for replying to an old thread - may I know if you have managed to test this out on your lab? How was the result? Thank you. On Thu, Feb 4, 2016 at 3:05 AM, Simon Weller wrote: > i'll test it out in our lab 4.7.1 lab and see what happens. > > > From: Nux! > Sent: Wednesday, February 3, 2016 12:34 PM > To: users@cloudstack.apache.org > Cc: Mohd Zainal Abidin Rabani > Subject: Re: Re[2]: High Availability Cloudstack > > Simon, > > I think this is no longer correct. After a certain timeout the management > server will bring up the VMs on another hypervisor. We discussed this in > the somewhat recent past, but can't find the thread right now. > There was some nice insight from Wido or Marcus. > > Lucian > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro > > - Original Message - > > From: "Simon Weller" > > To: "users" > > Cc: "Mohd Zainal Abidin Rabani" > > Sent: Wednesday, 3 February, 2016 16:23:55 > > Subject: Re: Re[2]: High Availability Cloudstack > > > My understanding is that HA will never actually function on KVM there > way most > > expect, unless the original host that failed comes back up and the agents > > report the VMs not running. This is due to Cloudstack not wanting to risk > > starting a vm that already is locked by another process, on a different > server. > > > > The only way I know of to force HA to function is to remove the host out > of > > CloudStack, to force CloudStack to reallocate the VMs. > > > > There has been a lot of talk about introducing a reliable fencing > mechanism into > > management, so that CloudStack management is 100% confident the vm is > safe to > > restart on a new host. > > > > The current proposal for that is here: > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IPMI+Fencing > > > > - Si > > > > > > > > > > From: Andrei Mikhailovsky > > Sent: Wednesday, February 3, 2016 1:49 AM > > To: users > > Cc: Mohd Zainal Abidin Rabani > > Subject: Re: Re[2]: High Availability Cloudstack > > > > Hi, > > > > from what i can can see, this issue has been resolved in version 4.4 and > Mohd is > > running 4.6. > > > > From my testing with KVM, the management server performs a poll once in > a while > > and the agent replies if a vm is up. It might take some time before the > state > > changes, like 30 seconds or so, so give it a bit more time. My testing > show > > that the management server does update the state if the vm is manually > powered > > down. > > > > I've not tested tha HA feature for sometime, but last time i checked it > was > > working. You should also set the HA option in the vm's system offering. > > > > Andrei > > - Original Message - > >> From: "Suresh Sadhu" > >> To: "users" , "Mohd Zainal Abidin Rabani" > >> > >> Sent: Wednesday, 3 February, 2016 06:24:49 > >> Subject: RE: Re[2]: High Availability Cloudstack > > > >> Please see below issue: > >> https://issues.apache.org/jira/browse/CLOUDSTACK-5582 > >> > >> regards > >> sadhu > >> > >> -Original Message- > >> From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] > >> Sent: Wednesday, February 3, 2016 10:12 AM > >> To: users@cloudstack.apache.org > >> Subject: Re[2]: High Availability Cloudstack > >> > >> I'm using KVM that come with centos 7. That management also using > centos 7. > >> Cloudstack version is 4.6. > >> > >> - > >> Regards, > >> Mohd Zainal Abidin Rabani > >> Technical Support > >> > >> ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, > Kajang > >> Sentral, 43000 Kajang, Selangor. > >> T : 03-8737 0030 | F : 03-8737 0070 > >> E : zai...@nocser.net > >> W : www.modern.com.my | www.nocser.net > >> > >> > >> > >> -- Original Message -- > >> From: "Abhinandan Prateek" > >> To: "users@cloudstack.apache.org" ; "Mohd > Zainal > >> Abidin Rabani" > >> Sent: 03/02/2016 12:38:38 > >> Subject: Re: High Availability Cloudstack > >> > >>>MS version, Hypervisor ? Are there more hosts in the cl
Re: Re[2]: High Availability Cloudstack
I can confirm that in the past (probably acs versions 4.2 or 4.1) I had several cases where ACS would restart vms which were running on a host which got disconnected fro ACS. As a result, I had two instances of the same vm running, creating a number of issues with disk consistency. I had to switch HA for that reason and rely on other means. Andrei - Original Message - > From: "Simon Weller" > To: "users" > Cc: "Mohd Zainal Abidin Rabani" > Sent: Wednesday, 3 February, 2016 19:05:20 > Subject: Re: Re[2]: High Availability Cloudstack > i'll test it out in our lab 4.7.1 lab and see what happens. > > > From: Nux! > Sent: Wednesday, February 3, 2016 12:34 PM > To: users@cloudstack.apache.org > Cc: Mohd Zainal Abidin Rabani > Subject: Re: Re[2]: High Availability Cloudstack > > Simon, > > I think this is no longer correct. After a certain timeout the management > server > will bring up the VMs on another hypervisor. We discussed this in the somewhat > recent past, but can't find the thread right now. > There was some nice insight from Wido or Marcus. > > Lucian > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro > > - Original Message - >> From: "Simon Weller" >> To: "users" >> Cc: "Mohd Zainal Abidin Rabani" >> Sent: Wednesday, 3 February, 2016 16:23:55 >> Subject: Re: Re[2]: High Availability Cloudstack > >> My understanding is that HA will never actually function on KVM there way >> most >> expect, unless the original host that failed comes back up and the agents >> report the VMs not running. This is due to Cloudstack not wanting to risk >> starting a vm that already is locked by another process, on a different >> server. >> >> The only way I know of to force HA to function is to remove the host out of >> CloudStack, to force CloudStack to reallocate the VMs. >> >> There has been a lot of talk about introducing a reliable fencing mechanism >> into >> management, so that CloudStack management is 100% confident the vm is safe to >> restart on a new host. >> >> The current proposal for that is here: >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IPMI+Fencing >> >> - Si >> >> >> >> >> From: Andrei Mikhailovsky >> Sent: Wednesday, February 3, 2016 1:49 AM >> To: users >> Cc: Mohd Zainal Abidin Rabani >> Subject: Re: Re[2]: High Availability Cloudstack >> >> Hi, >> >> from what i can can see, this issue has been resolved in version 4.4 and >> Mohd is >> running 4.6. >> >> From my testing with KVM, the management server performs a poll once in a >> while >> and the agent replies if a vm is up. It might take some time before the state >> changes, like 30 seconds or so, so give it a bit more time. My testing show >> that the management server does update the state if the vm is manually >> powered >> down. >> >> I've not tested tha HA feature for sometime, but last time i checked it was >> working. You should also set the HA option in the vm's system offering. >> >> Andrei >> - Original Message ----- >>> From: "Suresh Sadhu" >>> To: "users" , "Mohd Zainal Abidin Rabani" >>> >>> Sent: Wednesday, 3 February, 2016 06:24:49 >>> Subject: RE: Re[2]: High Availability Cloudstack >> >>> Please see below issue: >>> https://issues.apache.org/jira/browse/CLOUDSTACK-5582 >>> >>> regards >>> sadhu >>> >>> -Original Message- >>> From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] >>> Sent: Wednesday, February 3, 2016 10:12 AM >>> To: users@cloudstack.apache.org >>> Subject: Re[2]: High Availability Cloudstack >>> >>> I'm using KVM that come with centos 7. That management also using centos 7. >>> Cloudstack version is 4.6. >>> >>> - >>> Regards, >>> Mohd Zainal Abidin Rabani >>> Technical Support >>> >>> ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, Kajang >>> Sentral, 43000 Kajang, Selangor. >>> T : 03-8737 0030 | F : 03-8737 0070 >>> E : zai...@nocser.net >>> W : www.modern.com.my | www.nocser.net >>> >>> >>> >>> -- Original Message -- >>> From: "Abhinan
Re: Re[2]: High Availability Cloudstack
i'll test it out in our lab 4.7.1 lab and see what happens. From: Nux! Sent: Wednesday, February 3, 2016 12:34 PM To: users@cloudstack.apache.org Cc: Mohd Zainal Abidin Rabani Subject: Re: Re[2]: High Availability Cloudstack Simon, I think this is no longer correct. After a certain timeout the management server will bring up the VMs on another hypervisor. We discussed this in the somewhat recent past, but can't find the thread right now. There was some nice insight from Wido or Marcus. Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro - Original Message - > From: "Simon Weller" > To: "users" > Cc: "Mohd Zainal Abidin Rabani" > Sent: Wednesday, 3 February, 2016 16:23:55 > Subject: Re: Re[2]: High Availability Cloudstack > My understanding is that HA will never actually function on KVM there way most > expect, unless the original host that failed comes back up and the agents > report the VMs not running. This is due to Cloudstack not wanting to risk > starting a vm that already is locked by another process, on a different > server. > > The only way I know of to force HA to function is to remove the host out of > CloudStack, to force CloudStack to reallocate the VMs. > > There has been a lot of talk about introducing a reliable fencing mechanism > into > management, so that CloudStack management is 100% confident the vm is safe to > restart on a new host. > > The current proposal for that is here: > https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IPMI+Fencing > > - Si > > > > > From: Andrei Mikhailovsky > Sent: Wednesday, February 3, 2016 1:49 AM > To: users > Cc: Mohd Zainal Abidin Rabani > Subject: Re: Re[2]: High Availability Cloudstack > > Hi, > > from what i can can see, this issue has been resolved in version 4.4 and Mohd > is > running 4.6. > > From my testing with KVM, the management server performs a poll once in a > while > and the agent replies if a vm is up. It might take some time before the state > changes, like 30 seconds or so, so give it a bit more time. My testing show > that the management server does update the state if the vm is manually powered > down. > > I've not tested tha HA feature for sometime, but last time i checked it was > working. You should also set the HA option in the vm's system offering. > > Andrei > - Original Message - >> From: "Suresh Sadhu" >> To: "users" , "Mohd Zainal Abidin Rabani" >> >> Sent: Wednesday, 3 February, 2016 06:24:49 >> Subject: RE: Re[2]: High Availability Cloudstack > >> Please see below issue: >> https://issues.apache.org/jira/browse/CLOUDSTACK-5582 >> >> regards >> sadhu >> >> -Original Message- >> From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] >> Sent: Wednesday, February 3, 2016 10:12 AM >> To: users@cloudstack.apache.org >> Subject: Re[2]: High Availability Cloudstack >> >> I'm using KVM that come with centos 7. That management also using centos 7. >> Cloudstack version is 4.6. >> >> - >> Regards, >> Mohd Zainal Abidin Rabani >> Technical Support >> >> ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, Kajang >> Sentral, 43000 Kajang, Selangor. >> T : 03-8737 0030 | F : 03-8737 0070 >> E : zai...@nocser.net >> W : www.modern.com.my | www.nocser.net >> >> >> >> -- Original Message -- >> From: "Abhinandan Prateek" >> To: "users@cloudstack.apache.org" ; "Mohd Zainal >> Abidin Rabani" >> Sent: 03/02/2016 12:38:38 >> Subject: Re: High Availability Cloudstack >> >>>MS version, Hypervisor ? Are there more hosts in the cluster ? >>> >>> >>> >>> >>>Abhinandan PrateekSoftware Architect, ShapeBlued: | s: +44 203 603 >>>0540 | m: +91 970 11 99011e: abhinandan.prat...@shapeblue.com | t: | >>>w: www.shapeblue.coma: 53 Chandos Place, Covent Garden London WC2N 4HS >>>UKShape Blue Ltd is a company incorporated in England & Wales. >>>ShapeBlue Services India LLP is a company incorporated in India and is >>>operated under license from Shape Blue Ltd. Shape Blue Brasil >>>Consultoria Ltda is a company incorporated in Brasil and is operated >>>under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company >>>registered by The Republic of South Africa and is traded under license >>>from Shape Blue
Re: Re[2]: High Availability Cloudstack
Simon, I think this is no longer correct. After a certain timeout the management server will bring up the VMs on another hypervisor. We discussed this in the somewhat recent past, but can't find the thread right now. There was some nice insight from Wido or Marcus. Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro - Original Message - > From: "Simon Weller" > To: "users" > Cc: "Mohd Zainal Abidin Rabani" > Sent: Wednesday, 3 February, 2016 16:23:55 > Subject: Re: Re[2]: High Availability Cloudstack > My understanding is that HA will never actually function on KVM there way most > expect, unless the original host that failed comes back up and the agents > report the VMs not running. This is due to Cloudstack not wanting to risk > starting a vm that already is locked by another process, on a different > server. > > The only way I know of to force HA to function is to remove the host out of > CloudStack, to force CloudStack to reallocate the VMs. > > There has been a lot of talk about introducing a reliable fencing mechanism > into > management, so that CloudStack management is 100% confident the vm is safe to > restart on a new host. > > The current proposal for that is here: > https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IPMI+Fencing > > - Si > > > > > From: Andrei Mikhailovsky > Sent: Wednesday, February 3, 2016 1:49 AM > To: users > Cc: Mohd Zainal Abidin Rabani > Subject: Re: Re[2]: High Availability Cloudstack > > Hi, > > from what i can can see, this issue has been resolved in version 4.4 and Mohd > is > running 4.6. > > From my testing with KVM, the management server performs a poll once in a > while > and the agent replies if a vm is up. It might take some time before the state > changes, like 30 seconds or so, so give it a bit more time. My testing show > that the management server does update the state if the vm is manually powered > down. > > I've not tested tha HA feature for sometime, but last time i checked it was > working. You should also set the HA option in the vm's system offering. > > Andrei > ----- Original Message - >> From: "Suresh Sadhu" >> To: "users" , "Mohd Zainal Abidin Rabani" >> >> Sent: Wednesday, 3 February, 2016 06:24:49 >> Subject: RE: Re[2]: High Availability Cloudstack > >> Please see below issue: >> https://issues.apache.org/jira/browse/CLOUDSTACK-5582 >> >> regards >> sadhu >> >> -Original Message- >> From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] >> Sent: Wednesday, February 3, 2016 10:12 AM >> To: users@cloudstack.apache.org >> Subject: Re[2]: High Availability Cloudstack >> >> I'm using KVM that come with centos 7. That management also using centos 7. >> Cloudstack version is 4.6. >> >> - >> Regards, >> Mohd Zainal Abidin Rabani >> Technical Support >> >> ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, Kajang >> Sentral, 43000 Kajang, Selangor. >> T : 03-8737 0030 | F : 03-8737 0070 >> E : zai...@nocser.net >> W : www.modern.com.my | www.nocser.net >> >> >> >> -- Original Message -- >> From: "Abhinandan Prateek" >> To: "users@cloudstack.apache.org" ; "Mohd Zainal >> Abidin Rabani" >> Sent: 03/02/2016 12:38:38 >> Subject: Re: High Availability Cloudstack >> >>>MS version, Hypervisor ? Are there more hosts in the cluster ? >>> >>> >>> >>> >>>Abhinandan PrateekSoftware Architect, ShapeBlued: | s: +44 203 603 >>>0540 | m: +91 970 11 99011e: abhinandan.prat...@shapeblue.com | t: | >>>w: www.shapeblue.coma: 53 Chandos Place, Covent Garden London WC2N 4HS >>>UKShape Blue Ltd is a company incorporated in England & Wales. >>>ShapeBlue Services India LLP is a company incorporated in India and is >>>operated under license from Shape Blue Ltd. Shape Blue Brasil >>>Consultoria Ltda is a company incorporated in Brasil and is operated >>>under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company >>>registered by The Republic of South Africa and is traded under license >>>from Shape Blue Ltd. ShapeBlue is a registered trademark. >>>This email and any attachments to it may be confidential and are >>>intended solely for the use of the individual to whom it is addressed. >>>Any views or opinions expressed are solely tho
Re: Re[2]: High Availability Cloudstack
My understanding is that HA will never actually function on KVM there way most expect, unless the original host that failed comes back up and the agents report the VMs not running. This is due to Cloudstack not wanting to risk starting a vm that already is locked by another process, on a different server. The only way I know of to force HA to function is to remove the host out of CloudStack, to force CloudStack to reallocate the VMs. There has been a lot of talk about introducing a reliable fencing mechanism into management, so that CloudStack management is 100% confident the vm is safe to restart on a new host. The current proposal for that is here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+HA+with+IPMI+Fencing - Si From: Andrei Mikhailovsky Sent: Wednesday, February 3, 2016 1:49 AM To: users Cc: Mohd Zainal Abidin Rabani Subject: Re: Re[2]: High Availability Cloudstack Hi, from what i can can see, this issue has been resolved in version 4.4 and Mohd is running 4.6. >From my testing with KVM, the management server performs a poll once in a >while and the agent replies if a vm is up. It might take some time before the >state changes, like 30 seconds or so, so give it a bit more time. My testing >show that the management server does update the state if the vm is manually >powered down. I've not tested tha HA feature for sometime, but last time i checked it was working. You should also set the HA option in the vm's system offering. Andrei - Original Message - > From: "Suresh Sadhu" > To: "users" , "Mohd Zainal Abidin Rabani" > > Sent: Wednesday, 3 February, 2016 06:24:49 > Subject: RE: Re[2]: High Availability Cloudstack > Please see below issue: > https://issues.apache.org/jira/browse/CLOUDSTACK-5582 > > regards > sadhu > > -Original Message- > From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] > Sent: Wednesday, February 3, 2016 10:12 AM > To: users@cloudstack.apache.org > Subject: Re[2]: High Availability Cloudstack > > I'm using KVM that come with centos 7. That management also using centos 7. > Cloudstack version is 4.6. > > - > Regards, > Mohd Zainal Abidin Rabani > Technical Support > > ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, Kajang > Sentral, 43000 Kajang, Selangor. > T : 03-8737 0030 | F : 03-8737 0070 > E : zai...@nocser.net > W : www.modern.com.my | www.nocser.net > > > > -- Original Message -- > From: "Abhinandan Prateek" > To: "users@cloudstack.apache.org" ; "Mohd Zainal > Abidin Rabani" > Sent: 03/02/2016 12:38:38 > Subject: Re: High Availability Cloudstack > >>MS version, Hypervisor ? Are there more hosts in the cluster ? >> >> >> >> >>Abhinandan PrateekSoftware Architect, ShapeBlued: | s: +44 203 603 >>0540 | m: +91 970 11 99011e: abhinandan.prat...@shapeblue.com | t: | >>w: www.shapeblue.coma: 53 Chandos Place, Covent Garden London WC2N 4HS >>UKShape Blue Ltd is a company incorporated in England & Wales. >>ShapeBlue Services India LLP is a company incorporated in India and is >>operated under license from Shape Blue Ltd. Shape Blue Brasil >>Consultoria Ltda is a company incorporated in Brasil and is operated >>under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company >>registered by The Republic of South Africa and is traded under license >>from Shape Blue Ltd. ShapeBlue is a registered trademark. >>This email and any attachments to it may be confidential and are >>intended solely for the use of the individual to whom it is addressed. >>Any views or opinions expressed are solely those of the author and do >>not necessarily represent those of Shape Blue Ltd or related companies. >>If you are not the intended recipient of this email, you must neither >>take any action based upon its contents, nor copy or show it to anyone. >>Please contact the sender if you believe you have received this email >>in error. >> >> >>On 03/02/16, 9:24 AM, "Mohd Zainal Abidin Rabani" >>wrote: >> >> >Hi, >> > >> >I try to do HA test for cloudstack. We are using DL380 G7. Same spec. >> >VM's have put option with HA. >> > >> >When i shutdown host A, from instance i can see all state is running. >> >But from insfrastructure host A already disconnected. Other instance >> >that not set using HA also state running. Suppose on instance that >> >use option HA is running and other than that is stopped. Is this is a >> >kind of bug? >> > >> >HA using what? Pi
Re: Re[2]: High Availability Cloudstack
Hi, from what i can can see, this issue has been resolved in version 4.4 and Mohd is running 4.6. >From my testing with KVM, the management server performs a poll once in a >while and the agent replies if a vm is up. It might take some time before the >state changes, like 30 seconds or so, so give it a bit more time. My testing >show that the management server does update the state if the vm is manually >powered down. I've not tested tha HA feature for sometime, but last time i checked it was working. You should also set the HA option in the vm's system offering. Andrei - Original Message - > From: "Suresh Sadhu" > To: "users" , "Mohd Zainal Abidin Rabani" > > Sent: Wednesday, 3 February, 2016 06:24:49 > Subject: RE: Re[2]: High Availability Cloudstack > Please see below issue: > https://issues.apache.org/jira/browse/CLOUDSTACK-5582 > > regards > sadhu > > -Original Message- > From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] > Sent: Wednesday, February 3, 2016 10:12 AM > To: users@cloudstack.apache.org > Subject: Re[2]: High Availability Cloudstack > > I'm using KVM that come with centos 7. That management also using centos 7. > Cloudstack version is 4.6. > > - > Regards, > Mohd Zainal Abidin Rabani > Technical Support > > ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, Kajang > Sentral, 43000 Kajang, Selangor. > T : 03-8737 0030 | F : 03-8737 0070 > E : zai...@nocser.net > W : www.modern.com.my | www.nocser.net > > > > -- Original Message -- > From: "Abhinandan Prateek" > To: "users@cloudstack.apache.org" ; "Mohd Zainal > Abidin Rabani" > Sent: 03/02/2016 12:38:38 > Subject: Re: High Availability Cloudstack > >>MS version, Hypervisor ? Are there more hosts in the cluster ? >> >> >> >> >>Abhinandan PrateekSoftware Architect, ShapeBlued: | s: +44 203 603 >>0540 | m: +91 970 11 99011e: abhinandan.prat...@shapeblue.com | t: | >>w: www.shapeblue.coma: 53 Chandos Place, Covent Garden London WC2N 4HS >>UKShape Blue Ltd is a company incorporated in England & Wales. >>ShapeBlue Services India LLP is a company incorporated in India and is >>operated under license from Shape Blue Ltd. Shape Blue Brasil >>Consultoria Ltda is a company incorporated in Brasil and is operated >>under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company >>registered by The Republic of South Africa and is traded under license >>from Shape Blue Ltd. ShapeBlue is a registered trademark. >>This email and any attachments to it may be confidential and are >>intended solely for the use of the individual to whom it is addressed. >>Any views or opinions expressed are solely those of the author and do >>not necessarily represent those of Shape Blue Ltd or related companies. >>If you are not the intended recipient of this email, you must neither >>take any action based upon its contents, nor copy or show it to anyone. >>Please contact the sender if you believe you have received this email >>in error. >> >> >>On 03/02/16, 9:24 AM, "Mohd Zainal Abidin Rabani" >>wrote: >> >> >Hi, >> > >> >I try to do HA test for cloudstack. We are using DL380 G7. Same spec. >> >VM's have put option with HA. >> > >> >When i shutdown host A, from instance i can see all state is running. >> >But from insfrastructure host A already disconnected. Other instance >> >that not set using HA also state running. Suppose on instance that >> >use option HA is running and other than that is stopped. Is this is a >> >kind of bug? >> > >> >HA using what? Ping? Heartbeat? Why still show running? >> > >> >- >> >Regards, >> >Mohd Zainal Abidin Rabani >> >Technical Support >> > >> >ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, >> >Kajang Sentral, 43000 Kajang, Selangor. >> >T : 03-8737 0030 | F : 03-8737 0070 >> >E : zai...@nocser.net >> >W : www.modern.com.my | www.nocser.net >>Find out more about ShapeBlue and our range of CloudStack related >>services: >>IaaS Cloud Design & Build | CSForge – rapid IaaS deployment framework >>CloudStack Consulting | CloudStack Software Engineering CloudStack > >Infrastructure Support | CloudStack Bootcamp Training Courses
RE: Re[2]: High Availability Cloudstack
Please see below issue: https://issues.apache.org/jira/browse/CLOUDSTACK-5582 regards sadhu -Original Message- From: Mohd Zainal Abidin Rabani [mailto:zai...@nocser.net] Sent: Wednesday, February 3, 2016 10:12 AM To: users@cloudstack.apache.org Subject: Re[2]: High Availability Cloudstack I'm using KVM that come with centos 7. That management also using centos 7. Cloudstack version is 4.6. - Regards, Mohd Zainal Abidin Rabani Technical Support ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, Kajang Sentral, 43000 Kajang, Selangor. T : 03-8737 0030 | F : 03-8737 0070 E : zai...@nocser.net W : www.modern.com.my | www.nocser.net -- Original Message -- From: "Abhinandan Prateek" To: "users@cloudstack.apache.org" ; "Mohd Zainal Abidin Rabani" Sent: 03/02/2016 12:38:38 Subject: Re: High Availability Cloudstack >MS version, Hypervisor ? Are there more hosts in the cluster ? > > > > >Abhinandan PrateekSoftware Architect, ShapeBlued: | s: +44 203 603 >0540 | m: +91 970 11 99011e: abhinandan.prat...@shapeblue.com | t: | >w: www.shapeblue.coma: 53 Chandos Place, Covent Garden London WC2N 4HS >UKShape Blue Ltd is a company incorporated in England & Wales. >ShapeBlue Services India LLP is a company incorporated in India and is >operated under license from Shape Blue Ltd. Shape Blue Brasil >Consultoria Ltda is a company incorporated in Brasil and is operated >under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company >registered by The Republic of South Africa and is traded under license >from Shape Blue Ltd. ShapeBlue is a registered trademark. >This email and any attachments to it may be confidential and are >intended solely for the use of the individual to whom it is addressed. >Any views or opinions expressed are solely those of the author and do >not necessarily represent those of Shape Blue Ltd or related companies. >If you are not the intended recipient of this email, you must neither >take any action based upon its contents, nor copy or show it to anyone. >Please contact the sender if you believe you have received this email >in error. > > >On 03/02/16, 9:24 AM, "Mohd Zainal Abidin Rabani" >wrote: > > >Hi, > > > >I try to do HA test for cloudstack. We are using DL380 G7. Same spec. > >VM's have put option with HA. > > > >When i shutdown host A, from instance i can see all state is running. > >But from insfrastructure host A already disconnected. Other instance > >that not set using HA also state running. Suppose on instance that > >use option HA is running and other than that is stopped. Is this is a > >kind of bug? > > > >HA using what? Ping? Heartbeat? Why still show running? > > > >- > >Regards, > >Mohd Zainal Abidin Rabani > >Technical Support > > > >ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, > >Kajang Sentral, 43000 Kajang, Selangor. > >T : 03-8737 0030 | F : 03-8737 0070 > >E : zai...@nocser.net > >W : www.modern.com.my | www.nocser.net >Find out more about ShapeBlue and our range of CloudStack related >services: >IaaS Cloud Design & Build | CSForge – rapid IaaS deployment framework >CloudStack Consulting | CloudStack Software Engineering CloudStack >Infrastructure Support | CloudStack Bootcamp Training Courses
Re[2]: High Availability Cloudstack
I'm using KVM that come with centos 7. That management also using centos 7. Cloudstack version is 4.6. - Regards, Mohd Zainal Abidin Rabani Technical Support ModernOne Data Solutions Sdn. Bhd.(1119382-D) No. 83-2, Jalan TKS 1, Kajang Sentral, 43000 Kajang, Selangor. T : 03-8737 0030 | F : 03-8737 0070 E : zai...@nocser.net W : www.modern.com.my | www.nocser.net -- Original Message -- From: "Abhinandan Prateek" To: "users@cloudstack.apache.org" ; "Mohd Zainal Abidin Rabani" Sent: 03/02/2016 12:38:38 Subject: Re: High Availability Cloudstack MS version, Hypervisor ? Are there more hosts in the cluster ? Abhinandan PrateekSoftware Architect, ShapeBlued: | s: +44 203 603 0540 | m: +91 970 11 99011e: abhinandan.prat...@shapeblue.com | t: | w: www.shapeblue.coma: 53 Chandos Place, Covent Garden London WC2N 4HS UKShape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. On 03/02/16, 9:24 AM, "Mohd Zainal Abidin Rabani" wrote: >Hi, > >I try to do HA test for cloudstack. We are using DL380 G7. Same spec. >VM's have put option with HA. > >When i shutdown host A, from instance i can see all state is running. >But from insfrastructure host A already disconnected. Other instance >that not set using HA also state running. Suppose on instance that use >option HA is running and other than that is stopped. Is this is a kind >of bug? > >HA using what? Ping? Heartbeat? Why still show running? > >- >Regards, >Mohd Zainal Abidin Rabani >Technical Support > >ModernOne Data Solutions Sdn. Bhd.(1119382-D) >No. 83-2, Jalan TKS 1, >Kajang Sentral, 43000 Kajang, Selangor. >T : 03-8737 0030 | F : 03-8737 0070 >E : zai...@nocser.net >W : www.modern.com.my | www.nocser.net Find out more about ShapeBlue and our range of CloudStack related services: IaaS Cloud Design & Build | CSForge – rapid IaaS deployment framework CloudStack Consulting | CloudStack Software Engineering CloudStack Infrastructure Support | CloudStack Bootcamp Training Courses
