Hi Eric,
Based on your shared version details, the specific issue and why it does not work for you has to do with usage of qemu-ev 2.12, which was only recently supported: https://github.com/apache/cloudstack/pull/3278 The issue has to do with how systemvms are patched by cloudstack-agent using a patching script that no longer works well with qemu-ev 2.12. You can test latest 4.11 or wait until 4.11.3.0 is out that will have support for qemu-ev 2.12 with a new patching mechanism that uses qemu-guest-agent. Regards, Rohit Yadav Software Architect, ShapeBlue https://www.shapeblue.com ________________________________ From: Eric Lee Green <eric.lee.gr...@gmail.com> Sent: Wednesday, May 22, 2019 10:40:08 AM To: users@cloudstack.apache.org; Rohit Yadav Subject: Re: Upgrade to Cloudstack 4.11.2 fails *AGAIN* Thanks for the response, sorry if I sound frustrated, but this is supposed to be a simple easy process and it's been horrible all the way through. 4.11.1 failed so I had to downgrade back to 4.9, and now 4.11.2 has failed to upgrade too. I've thus far spent around 16 hours of my time on what should have taken an hour at most. I'm frustrated and bummed. [root@cloud2 ~]# rpm -q centos-release centos-release-7-6.1810.2.el7.centos.x86_64 [root@cloud2 ~]# rpm -q libvirt libvirt-4.5.0-10.el7_6.9.x86_64 [root@cloud1 ~]# rpm -qa | grep kvm qemu-kvm-ev-2.12.0-18.el7_6.5.1.x86_64 qemu-kvm-common-ev-2.12.0-18.el7_6.5.1.x86_64 [root@cloud2 ~]# cat /proc/version Linux version 3.10.0-862.6.3.el7.x86_64 (buil...@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Tue Jun 26 16:32:21 UTC 2018 [root@cloud2 ~]# cat /proc/cpuinfo ... processor : 23 vendor_id : GenuineIntel cpu family : 6 model : 44 model name : Intel(R) Xeon(R) CPU X5675 @ 3.07GHz stepping : 2 microcode : 0x1f cpu MHz : 3068.000 cache size : 12288 KB physical id : 1 siblings : 12 core id : 10 cpu cores : 6 apicid : 53 initial apicid : 53 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt lahf_lm epb ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid dtherm ida arat bogomips : 6133.21 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual [root@cloud1 ~]# free total used free shared buff/cache available Mem: 123596388 79795792 34047696 632116 9752900 39999332 Swap: 4194300 1956824 2237476 [root@cloud1 ~]# yum update ========================================================================================================================================================================= Package Arch Version Repository Size ========================================================================================================================================================================= Updating: cloudstack-agent x86_64 4.11.2.0-1.el7.centos cloudstack411 47 M cloudstack-common x86_64 4.11.2.0-1.el7.centos cloudstack411 58 M cloudstack-management x86_64 4.11.2.0-1.el7.centos cloudstack411 82 M Three compute servers running the above processor averaging 128gb of memory apiece, two data servers running NFS for primary and secondary storage. Running the 4.11.2 community RPM's above. Yes, I did register the 4.11.2 systemvmtemplate before updating. The routers in fact started with the template, it said 4.11.2 when I opened up the console window and looked at the login prompt. But they never got configured, when I logged in with root/password at the console prompt they had no networking set up and no configuration provided, the cloud-init output in /var/log was essentially blank. Do I recall correctly that there is an issue with a particular version of the hypervisor on the latest Centos 7? Any other information that you need? I think I provided the complete log file for the cloud-init of the router in another email... On 5/21/2019 9:38 PM, Rohit Yadav wrote: > Hi Eric, > > Can you describe your environment in detail such as management server host > distro, hypervisor version, current CloudStack version, did you register the > 4.11.2 systemvmtemplate beforw upgrading etc. > > Regards. > > Regards, > Rohit Yadav > > ________________________________ > From: Eric Lee Green <eric.lee.gr...@gmail.com> > Sent: Wednesday, May 22, 2019 6:21:16 AM > To: users@cloudstack.apache.org > Subject: Upgrade to Cloudstack 4.11.2 fails *AGAIN* > > You may remember me as the person who had to roll back to Cloudstack > 4.9.x because Cloudstack 4.11.1 wouldn't start any virtual machines once > I upgraded to it, claiming that there were inadequate resources even > though I had over 150 gigabytes of memory free in my cluster and oodles > of CPU free (and a minimum of 40gb on each node, plenty to start a 512mb > router VM). So now I'm trying to upgrade to Cloudstack 4.11.2 and > *again* it's misbehaving. > > The symptom is that my virtual routers when I log into their console > show 4.11.2 but when I look at them in the console they say 'Version: > UNKNOWN'. Also when I try to ssh into their guest IP address or link > local IP address it fails. And when I try to start up a virtual machine > that uses that virtual network, it says "Network unavailable', even > though the router for that network is showing up and running. > > Clearly something's broken in the virtual routers but I don't know what > because I can't get into the router virtual machines. How do I get the > console password to get into the router virtual machines? It's encrypted > in the database (duh), how do I decrypt it? > > > > rohit.ya...@shapeblue.com > www.shapeblue.com<http://www.shapeblue.com> > Amadeus House, Floral Street, London WC2E 9DPUK > @shapeblue > > > rohit.ya...@shapeblue.com www.shapeblue.com Amadeus House, Floral Street, London WC2E 9DPUK @shapeblue
