Hi Pearl, I just did a SSVM redeploy in the same environment, but different zone and and I see the following differences :
"isManaged":"false"}},"name":"ROOT-2055","size":"(2.44 GB)" Working zone - https://pastebin.com/raw/88N6Fg6z "isManaged":"false"}},"name":"ROOT-2055","size":"(2.44 GB) datastore-6588","managed":"false","storagePort":"0","volumeSize":"(2.44 GB) 021-09-24 05:22:23,959 INFO [c.c.h.v.r.VmwareResource] (DirectAgent-437:ctx-2f1519ba fra-de-001.test.host, job-25099/job-35371, cmd: StartCommand) (logid:d442432d) Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: fra-de-001 - 9d679070c7a6441592646bfbb82ef1b3 Not working zone - https://pastebin.com/raw/Um2uHgWM "isManaged":"false"}},"name":"ROOT-2056","size":"(0 bytes)" here is 0 bytes... datastore-6694","managed":"false","storagePort":"0","volumeSize":"(0 bytes) here is 0 bytes... 2021-09-24 05:00:55,594 INFO [c.c.h.v.r.VmwareResource] (DirectAgent-30:ctx-5e49d288 lnd-uk-002.test.host, job-25099/job-35365, cmd: StartCommand) (logid:d442432d) Mapped disk size is not the same as the cloned VM disk size: 0 - 2621440000 ( here I see this line, with disk size 0 ) 2021-09-24 05:00:55,594 INFO [c.c.h.v.r.VmwareResource] (DirectAgent-30:ctx-5e49d288 lnd-uk-002.test.host, job-25099/job-35365, cmd: StartCommand) (logid:d442432d) Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: lnd-uk-002 - fb7b551a584e4b5192b8f371af56b303 I do not understand why I have here the size "0" Regards, Cristian -----Original Message----- From: Pearl d'Silva <pearl.dsi...@shapeblue.com> Sent: Friday, September 24, 2021 9:44 AM To: users@cloudstack.apache.org Subject: Re: Failed to authentication SSH user root on host - repetitive error By any chance, are there any old systemvm.isos in the secondary store, if yes, can you please delete them. Based on the information you've provided, it seems like, the key has been injected into systemvm.iso, but during boot up, a script (cloud-early-config) that sets up the VM before bootstrapping / patching isn't copying the auth key. Can you try the steps mentioned under the VMWare section of the doc: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SystemVm.iso and see if it helps. SystemVm.iso - Apache Cloudstack - Apache Software Foundation<https://cwiki.apache.org/confluence/display/CLOUDSTACK/SystemVm.i so> Mechanism of propagating systemvm.iso. Systemvm.iso gets propagated differently for different hypervisors. It comes packaged as part of the Cloudstack rpm or is built on dev environments with the command "mvn clean install -P developer,systemvm" (More info on building cloudstack) .Once the iso is at the appropriate location depending on the hypervisor it is inserted as a cd drive and the ... cwiki.apache.org Thanks, Pearl ________________________________ From: cristian.c@istream.today <cristian.c@istream.today> Sent: Friday, September 24, 2021 11:45 AM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: RE: Failed to authentication SSH user root on host - repetitive error Hi Pearl, I already did a check via console into the secondary storage vm, I saw that there are not keys in authorized_keys. If I try to inject, I see this : /bin/bash /usr/share/cloudstack-common/scripts/vm/systemvm/injectkeys.sh /var/cloudstack/management/.ssh/id_rsa.pub /var/cloudstack/management/.ssh/id_rsa /usr/share/cloudstack-common/vms/systemvm.iso mount: /dev/loop0 is write-protected, mounting read-only New public key is the same as the one in the systemvm.iso, not injecting it, not modifying systemvm.iso [root@cloud-emea systemvm]# cd /mnt/iso/ [root@cloud-emea iso]# ls agent.zip authorized_keys cloud-scripts.tgz [root@cloud-emea iso]# cat authorized_keys ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA0UEA0mRUzKbH1cHHY2GlsaQ18q1KvqfNSV/YqAScZhcPueIl BbqwPUNznzfSiz/K/+DH8u5QkDIz+fC0Sx+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+Bd1Y2U1Tx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+lYee +zUh6vszEDwmiq5nTkuCJP1T3o8QL3gUekAFjW7CECsIzLkA41Q8lY0L3qaHBRUJntGGIMtZ +26AP IWMC7NHD0wFge3DEN5UhFODcB1f9U7oqa10XqgORjbd88JPfFv/0j92xaaerNpJKw== cl...@cloud-emea.test.host[root@cloud-emea iso]# is the same key as I have in : /var/cloudstack/management/.ssh/id_rsa.pub (For security reason I have replaced characters from ssh pub key) I'm stuck here, I do not understand the logic. Regards, Cristian -----Original Message----- From: Pearl d'Silva <pearl.dsi...@shapeblue.com> Sent: Friday, September 24, 2021 8:05 AM To: users@cloudstack.apache.org Subject: Re: Failed to authentication SSH user root on host - repetitive error Hi Cristian, The exception you are seeing is most likely to happen if the systemvm.iso hasn't been injected with the Management server's key. One way to validate it, would be to go to your secondary store - I presume you are working on a VMware setup - mount the systemvm-4.15.2.iso and verify the authorized_keys data with what's present on the MS at /var/cloudstack/management/.ssh/id_rsa.pub. Thanks, Pearl ________________________________ From: cristian.c@istream.today <cristian.c@istream.today> Sent: Thursday, September 23, 2021 9:50 PM To: users@cloudstack.apache.org <users@cloudstack.apache.org> Subject: Failed to authentication SSH user root on host - repetitive error Hello, I was not able to fix this error "Failed to authentication SSH" and looks like a bug for me, I will explain here why. My setup before adding new ZONE: 1. Cloudstack 4.15.2 + VMware 6.5 with 1 Basic Network Zone. ( This is an old setup, it was upgraded frequently, I think from 4.2 ) 2. I add new VMware zone, but this one with Advanced Networking, I end with this repetitive error and nothing else. 3. I decide to add a new zone, the same as I have at (2) with Advanced Networking, everything works perfect, no issue at systemvm deploy, routers, instances. 4. I delete the zone from point (2) 5. I add again the same zone, end with the same error. I have tried multiple things, delete the template, add again, inject, nothing works, I end with the same error, over and over, and I do not see the why. 021-09-23 11:53:54,666 ERROR [c.c.h.v.r.VmwareResource] (DirectAgent-228:ctx-083b9265 lnd-uk-002.test.host, job-35231/job-35263, cmd: SetupKeyStoreCommand) (logid:df4131be) Command failed due to Exception: java.lang.Exception Message: Failed to authentication SSH user root on host 10.15.0.204 2021-09-23 11:53:54,674 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-19:ctx-808054ab job-35231/job-35263 ctx-eb7c37bb) (logid:df4131be) Retrying after catching exception while trying to secure agent for systemvm id=2025 com.cloud.utils.exception.CloudRuntimeException: Unable to read/process CSR: Command failed due to Exception: java.lang.Exception Message: Failed to authentication SSH user root on host 10.15.0.204 Here is the full log : https://pastebin.com/raw/fFfmquVB Maybe someone have a hint for me . Regards, Cristian
