Hello,
I did this in 3 different new Setup (new Management install, NFS, etc),
no matter if I add as first zone or second zone.
" I have tested this with multiple Cloudstack versions, 4.15.0, 4.15.1,
4.15.2 and VMware, 6.5 and with 6.7 different patches and with the last
patches. I'm 100% that is related to these 2 servers, I do not understand
what is wrong with these servers, this is the problem.
We have 2 identical servers' hardware/ESXI, the only difference is the
location, the not working, are in UK, and any other location we have, DE,
NYC, FR, CA, etc. works fine. But we tested with the exact same servers
which are from DE(Germany) works perfect, with any ACS or VMware version
(also same CIDR range/size for public and private), when we test with UK, we
get same error no matter what."
I want to mention that we have these servers (UK, DE, FR, CA ) from OVH
in vRack and the UK, are not working. ( I manage these server for more
than 5 years )
Regards,
Cristian
-----Original Message-----
From: Harikrishna Patnala <harikrishna.patn...@shapeblue.com>
Sent: Monday, September 27, 2021 1:29 PM
To: users@cloudstack.apache.org
Subject: Re: Failed to authentication SSH user root on host - repetitive
error
Hi Crisitian,
To my understanding, you are trying to add a new zone and getting issues
with the system VM having some SSH failures.
May I ask you to start everything fresh?
I mean to prepare new secondary storage, seed the system VM template into
new secondary storage and then deploy the zone from the beginning of the
zone creation wizard. Make sure your management server has the latest
packages and systemvm.iso is also as expected to the CloudStack version you
are trying.
Thanks,
Harikrishna
________________________________
From: Cristian Ciobanu <cristian.c@istream.today>
Sent: Friday, September 24, 2021 5:19 PM
To: users@cloudstack.apache.org <users@cloudstack.apache.org>
Subject: Re: Failed to authentication SSH user root on host - repetitive
error
Hi Pearl,
I did upgrades multiple times, I always add the template before upgrade.
If was something related to this, i was expecting not to work for all zones,
but right now is working for 2 zones from the same setup, environment.
Regards,
Cristian
On Fri, Sep 24, 2021, 14:40 Pearl d'Silva <pearl.dsi...@shapeblue.com>
wrote:
> Hi Cristian,
>
> After going through your earlier mails again, and correct me if I am
> wrong, but I noticed in one of your mails you mentioned that the env
> that's causing issues was upgraded from a lower version, right? So,
> ideally when doing upgrades, prior to initiating the upgrade we first
> take a backup of the DB, register the template (new systemVM
> template) following this
> https://docs.cloudstack.apache.org/en/latest/upgrading/upgrade/upgrade
> -4.15.html#update-system-vm-templates
> and then upgrade the environment. However, you mentioned that you used
> the cloud-install-sys-tmplt script to seed and register the template.
> I am not familiar if using that script during upgrades works (it may,
> but I haven't ever used that approach).
> If it is possible, it would be advisable to rollback your env to the
> earlier version (disable your zone, destroy the systemVMs before that)
> and then follow the upgrade steps as mentioned in the documentation:
> https://docs.cloudstack.apache.org/en/latest/upgrading/index.html
>
> Thanks,
> Pearl
>
>
> ________________________________
> From: cristian.c@istream.today <cristian.c@istream.today>
> Sent: Friday, September 24, 2021 3:08 PM
> To: users@cloudstack.apache.org <users@cloudstack.apache.org>
> Subject: RE: Failed to authentication SSH user root on host -
> repetitive error
>
> Hi Pearl,
>
> I just did a SSVM redeploy in the same environment, but different zone
> and and I see the following differences :
>
> "isManaged":"false"}},"name":"ROOT-2055","size":"(2.44 GB)"
>
>
> Working zone - https://pastebin.com/raw/88N6Fg6z
>
> "isManaged":"false"}},"name":"ROOT-2055","size":"(2.44 GB)
>
>
> datastore-6588","managed":"false","storagePort":"0","volumeSize":"(2.4
> 4
> GB)
>
> 021-09-24 05:22:23,959 INFO [c.c.h.v.r.VmwareResource]
> (DirectAgent-437:ctx-2f1519ba fra-de-001.test.host,
> job-25099/job-35371,
> cmd: StartCommand) (logid:d442432d) Mapped disk datastore UUID is not
> the same as the cloned VM datastore UUID: fra-de-001 -
> 9d679070c7a6441592646bfbb82ef1b3
>
>
>
> Not working zone - https://pastebin.com/raw/Um2uHgWM
>
> "isManaged":"false"}},"name":"ROOT-2056","size":"(0 bytes)" here is 0
> bytes...
>
> datastore-6694","managed":"false","storagePort":"0","volumeSize":"(0
> bytes)
> here is 0 bytes...
>
> 2021-09-24 05:00:55,594 INFO [c.c.h.v.r.VmwareResource]
> (DirectAgent-30:ctx-5e49d288 lnd-uk-002.test.host,
> job-25099/job-35365,
> cmd:
> StartCommand) (logid:d442432d) Mapped disk size is not the same as the
> cloned VM disk size: 0 - 2621440000 ( here I see this line, with disk
> size 0 )
>
> 2021-09-24 05:00:55,594 INFO [c.c.h.v.r.VmwareResource]
> (DirectAgent-30:ctx-5e49d288 lnd-uk-002.test.host,
> job-25099/job-35365,
> cmd:
> StartCommand) (logid:d442432d) Mapped disk datastore UUID is not the
> same as the cloned VM datastore UUID: lnd-uk-002 -
> fb7b551a584e4b5192b8f371af56b303
>
>
>
> I do not understand why I have here the size "0"
>
>
> Regards,
> Cristian
>
>
>
>
> -----Original Message-----
> From: Pearl d'Silva <pearl.dsi...@shapeblue.com>
> Sent: Friday, September 24, 2021 9:44 AM
> To: users@cloudstack.apache.org
> Subject: Re: Failed to authentication SSH user root on host -
> repetitive error
>
> By any chance, are there any old systemvm.isos in the secondary store,
> if yes, can you please delete them. Based on the information you've
> provided, it seems like, the key has been injected into systemvm.iso,
> but during boot up, a script (cloud-early-config) that sets up the VM
> before bootstrapping / patching isn't copying the auth key.
> Can you try the steps mentioned under the VMWare section of the doc:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/SystemVm.iso
> and see if it helps.
> SystemVm.iso - Apache Cloudstack - Apache Software Foundation<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/SystemVm.i
> so>
> Mechanism of propagating systemvm.iso. Systemvm.iso gets propagated
> differently for different hypervisors. It comes packaged as part of
> the Cloudstack rpm or is built on dev environments with the command
> "mvn clean install -P developer,systemvm" (More info on building
> cloudstack) .Once the iso is at the appropriate location depending on
> the hypervisor it is inserted as a cd drive and the ...
> cwiki.apache.org
> Thanks,
> Pearl
>
> ________________________________
> From: cristian.c@istream.today <cristian.c@istream.today>
> Sent: Friday, September 24, 2021 11:45 AM
> To: users@cloudstack.apache.org <users@cloudstack.apache.org>
> Subject: RE: Failed to authentication SSH user root on host -
> repetitive error
>
> Hi Pearl,
>
> I already did a check via console into the secondary storage vm,
> I saw that there are not keys in authorized_keys.
>
> If I try to inject, I see this : /bin/bash
> /usr/share/cloudstack-common/scripts/vm/systemvm/injectkeys.sh
> /var/cloudstack/management/.ssh/id_rsa.pub
> /var/cloudstack/management/.ssh/id_rsa
> /usr/share/cloudstack-common/vms/systemvm.iso
> mount: /dev/loop0 is write-protected, mounting read-only New public
> key is the same as the one in the systemvm.iso, not injecting it, not
> modifying systemvm.iso
>
>
> [root@cloud-emea systemvm]# cd /mnt/iso/ [root@cloud-emea iso]# ls
> agent.zip authorized_keys cloud-scripts.tgz [root@cloud-emea iso]#
> cat authorized_keys ssh-rsa
>
> AAAAB3NzaC1yc2EAAAABIwAAAQEA0UEA0mRUzKbH1cHHY2GlsaQ18q1KvqfNSV/YqAScZh
> cPueIl
>
> BbqwPUNznzfSiz/K/+DH8u5QkDIz+fC0Sx+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> xxxxxx
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+Bd1Y2U1
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+Tx
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+lYee
> +zUh6vszEDwmiq5nTkuCJP1T3o8QL3gUekAFjW7CECsIzLkA41Q8lY0L3qaHBRUJntGGIM
> +tZ
> +26AP
> IWMC7NHD0wFge3DEN5UhFODcB1f9U7oqa10XqgORjbd88JPfFv/0j92xaaerNpJKw==
> cl...@cloud-emea.test.host[root@cloud-emea iso]#
>
> is the same key as I have in :
> /var/cloudstack/management/.ssh/id_rsa.pub
>
> (For security reason I have replaced characters from ssh pub key)
>
>
> I'm stuck here, I do not understand the logic.
>
>
> Regards,
> Cristian
>
>
>
>
> -----Original Message-----
> From: Pearl d'Silva <pearl.dsi...@shapeblue.com>
> Sent: Friday, September 24, 2021 8:05 AM
> To: users@cloudstack.apache.org
> Subject: Re: Failed to authentication SSH user root on host -
> repetitive error
>
> Hi Cristian,
>
> The exception you are seeing is most likely to happen if the
> systemvm.iso hasn't been injected with the Management server's key.
> One way to validate it, would be to go to your secondary store - I
> presume you are working on a VMware setup - mount the
> systemvm-4.15.2.iso and verify the authorized_keys data with what's
> present on the MS at /var/cloudstack/management/.ssh/id_rsa.pub.
>
> Thanks,
> Pearl
>
> ________________________________
> From: cristian.c@istream.today <cristian.c@istream.today>
> Sent: Thursday, September 23, 2021 9:50 PM
> To: users@cloudstack.apache.org <users@cloudstack.apache.org>
> Subject: Failed to authentication SSH user root on host - repetitive
> error
>
> Hello,
>
>
>
>
>
> I was not able to fix this error "Failed to authentication SSH"
> and looks like a bug for me, I will explain here why.
>
>
>
> My setup before adding new ZONE:
>
>
>
> 1. Cloudstack 4.15.2 + VMware 6.5 with 1 Basic Network Zone. ( This
> is
> an old setup, it was upgraded frequently, I think from 4.2 )
> 2. I add new VMware zone, but this one with Advanced Networking, I
end
> with this repetitive error and nothing else.
> 3. I decide to add a new zone, the same as I have at (2) with
Advanced
> Networking, everything works perfect, no issue at systemvm deploy,
> routers, instances.
> 4. I delete the zone from point (2)
> 5. I add again the same zone, end with the same error.
>
>
>
> I have tried multiple things, delete the template, add again,
> inject, nothing works, I end with the same error, over and over, and I
> do not see the why.
>
>
>
>
>
> 021-09-23 11:53:54,666 ERROR [c.c.h.v.r.VmwareResource]
> (DirectAgent-228:ctx-083b9265 lnd-uk-002.test.host,
> job-35231/job-35263,
> cmd: SetupKeyStoreCommand) (logid:df4131be) Command failed due to
> Exception:
> java.lang.Exception
>
> Message: Failed to authentication SSH user root on host 10.15.0.204
>
>
>
> 2021-09-23 11:53:54,674 ERROR [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-19:ctx-808054ab job-35231/job-35263 ctx-eb7c37bb)
> (logid:df4131be) Retrying after catching exception while trying to
> secure agent for systemvm id=2025
>
> com.cloud.utils.exception.CloudRuntimeException: Unable to
> read/process
> CSR:
> Command failed due to Exception: java.lang.Exception
>
> Message: Failed to authentication SSH user root on host 10.15.0.204
>
>
>
>
>
> Here is the full log : https://pastebin.com/raw/fFfmquVB
>
>
>
>
>
> Maybe someone have a hint for me .
>
>
>
> Regards,
>
> Cristian
>
>
>
>
>
>
>
>
>
>
>
