Re: "Crashed" vm hypervisor
Hi, No I use local storage, but like Joan said, the problem is the vm’s are stuck, that my vm’s would stop working I expected since I use local storage that is a calculated risk for now in this test. The problem is the management in all ways still “thinks” the vm’s are working and with everything still tries to power them off which it can’t, when the host breaks it should just put them in an error state and not try powering them off as that won’t work. Now I booted the “broken” server after which everything of course started working. I do think that is a big oversight in this case and a major issue for a production rollout, since apparently even if you use NFS the same issue remains. -- Jimmy Van: Wei ZHOU Datum: donderdag, 16 november 2023 om 14:17 Aan: users@cloudstack.apache.org Onderwerp: Re: "Crashed" vm hypervisor Do you use NFS ? -Wei On Thu, 16 Nov 2023 at 14:03, Jimmy Huybrechts wrote: > The reason I put it between “” was that I turned off one of the > hypervisors to test what it would do with my vm’s and in this case system > vm’s, what I thought would happened was it would say that my vm’s are in a > state of not running / powered off / error and that it would mention the > host being offline. > > What actually happened is that it seems to be completely oblivious of the > vm’s it had in the dashboard, it still mentions them as state running (I > know they are dead as a I can’t ping them anymore). > > The only mention I can see that it has an Alert on the host and that it > says agent disconnected with the system vm’s. > > The big problem here in my mind is that I can’t do anything with the > system vm’s now, I tried destroying one but since it tries to shut it down > first as it still thinks it’s running, it runs into an error because of > course then it figures out the hypervisor is broken and it can’t. And we > run into a circle. > > What should a person do in such a situation? > -- > Jimmy >
Re: "Crashed" vm hypervisor
Similar issues are observed in our cloudstack v4.17.2 KVM with NFS. If we poweroff the hypervisor the host status will show disconnected. But all the guest VM's status still shows as running, actual case is VM's are not available anywhere -jon On Thu, Nov 16, 2023 at 6:47 PM Wei ZHOU wrote: > Do you use NFS ? > > -Wei > > On Thu, 16 Nov 2023 at 14:03, Jimmy Huybrechts > wrote: > > > The reason I put it between “” was that I turned off one of the > > hypervisors to test what it would do with my vm’s and in this case system > > vm’s, what I thought would happened was it would say that my vm’s are in > a > > state of not running / powered off / error and that it would mention the > > host being offline. > > > > What actually happened is that it seems to be completely oblivious of the > > vm’s it had in the dashboard, it still mentions them as state running (I > > know they are dead as a I can’t ping them anymore). > > > > The only mention I can see that it has an Alert on the host and that it > > says agent disconnected with the system vm’s. > > > > The big problem here in my mind is that I can’t do anything with the > > system vm’s now, I tried destroying one but since it tries to shut it > down > > first as it still thinks it’s running, it runs into an error because of > > course then it figures out the hypervisor is broken and it can’t. And we > > run into a circle. > > > > What should a person do in such a situation? > > -- > > Jimmy > > >
Re: "Crashed" vm hypervisor
Do you use NFS ? -Wei On Thu, 16 Nov 2023 at 14:03, Jimmy Huybrechts wrote: > The reason I put it between “” was that I turned off one of the > hypervisors to test what it would do with my vm’s and in this case system > vm’s, what I thought would happened was it would say that my vm’s are in a > state of not running / powered off / error and that it would mention the > host being offline. > > What actually happened is that it seems to be completely oblivious of the > vm’s it had in the dashboard, it still mentions them as state running (I > know they are dead as a I can’t ping them anymore). > > The only mention I can see that it has an Alert on the host and that it > says agent disconnected with the system vm’s. > > The big problem here in my mind is that I can’t do anything with the > system vm’s now, I tried destroying one but since it tries to shut it down > first as it still thinks it’s running, it runs into an error because of > course then it figures out the hypervisor is broken and it can’t. And we > run into a circle. > > What should a person do in such a situation? > -- > Jimmy >
