RE: juju hangs during bootstrapping
Thanks Narinder, sorry for late response, just back from holiday, I think the issue has been solved by bootstrap with a config.yaml, where the config.yaml hold the proxy that is needed for external network access, $ juju bootstrap maascloud maas-cloud-control --config=config.yaml $ cat config.yaml default-series: xenial no-proxy: localhost http-proxy: http://192.168.130.250:8000 https-proxy: http://192.168.130.250:8000 ftp-proxy: http://192.168.130.250:8000 I’m not clear why the proxy is needed since “192.168.130.250” is actually the IP address of MAAS server, anywhere, the bootstrap succeeded. Best Regards, Dave From: Narinder Gupta [mailto:narinder.gu...@canonical.com] Sent: Saturday, September 30, 2017 7:31 PM To: Chen2, Dave Cc: juju Subject: RE: juju hangs during bootstrapping Can u try nslookup for archive.ubuntu.com<http://archive.ubuntu.com> looks like upstream dns issue as well apart from network as you mentioned it was broken. Thanks and regards, Narinder Gupta 1-2817365150 On Sep 30, 2017 04:54, mailto:dave.c...@dell.com>> wrote: Hi Narinder, Here is the log from the deployed node, actually, I can deploy Operation System successfully with MAAS or “juju bootstrap” but failed at some final steps, Our external network is exactly broken for some reason yesterday and won’t be recovered in short term, but I guess the network broken is happened after I saw the below error message during bootstrapping. “Waiting for address Attempting to connect to 10.20.3.254:22” I can see apt-get update works from the log at the beginning, and the network is broken couple of hours after I saw those error message. Does the error message like below has any connection with connection refused error message? I am not quite sure. “0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Fetching Juju agent version 2.2.4 for amd64 Attempt 1 to download tools from https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz... curl: (6) Could not resolve host: streams.canonical.com<http://streams.canonical.com> “ Anyway, I will try it again when our network back to normal, it would be great if you can see any other issue beside the network, thanks again for your help! Best Regards, Dave Chen From: Narinder Gupta [mailto:narinder.gu...@canonical.com<mailto:narinder.gu...@canonical.com>] Sent: Friday, September 29, 2017 10:52 PM To: Chen2, Dave mailto:dave_ch...@dell.com>> Cc: juju mailto:juju@lists.ubuntu.com>> Subject: Re: juju hangs during bootstrapping Hi Dave, May I know which division of Dell you are working on? As i have setup Openstack deployed t Dell multiple time with MAAS and have not seen this issue so far. So please send me log /var/log/cloud-init-output.log which will let us know what is wrong. Also try sudo apt-get update on the bootstrap node to confirm you have external access. In MAAS you can always add the ssh keys to land into the installed nodes though. Thanks and Regards, Narinder Gupta (PMP) narinder.gu...@canonical.com<mailto:narinder.gu...@canonical.com> Canonical, Ltd.narindergupta [irc.freenode.net<http://irc.freenode.net>] +1.281.736.5150 narindergupta2007[skype] Ubuntu- Linux for human beings | www.ubuntu.com<http://www.ubuntu.com> | www.canonical.com<http://www.canonical.com> On Fri, Sep 29, 2017 at 9:42 AM, mailto:dave.c...@dell.com>> wrote: Hi All, I am trying to bootstrap a MAAS cloud based on juju’s official guide (https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct but after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been installed, juju hangs when attempting to connect to the MAAS node, here is what I can see from the terminal, $ juju bootstrap maas-cloud Creating Juju controller "maas-cloud" on maas-cloud Looking for packaged Juju agent version 2.2.4 for amd64 Launching controller instance(s) on maas-cloud... - cka68p (arch=amd64 mem=32G cores=12) Fetching Juju GUI 2.9.2 Waiting for address Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22> (JUJU hangs here!) And it’s pending here forever, so I tried it again with the debug mode, $ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud maas-cloud-controller I saw some detail information like below, Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22> 19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused 19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused 19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused 19:33:5
Re: juju hangs during bootstrapping
On Fri, Sep 29, 2017 at 11:55 PM wrote:
>
>
>
>
> *From:* Andrew Wilkins [mailto:andrew.wilk...@canonical.com]
> *Sent:* Saturday, September 30, 2017 12:23 AM
> *To:* Chen2, Dave ; juju@lists.ubuntu.com
> *Subject:* Re: juju hangs during bootstrapping
>
>
>
> On Fri, Sep 29, 2017 at 10:43 AM wrote:
>
> Hi All,
>
>
>
> I am trying to bootstrap a MAAS cloud based on juju’s official guide (
> https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct
> but after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been
> installed, juju hangs when attempting to connect to the MAAS node, here is
> what I can see from the terminal,
>
>
>
> $ juju bootstrap maas-cloud
>
> Creating Juju controller "maas-cloud" on maas-cloud
>
> Looking for packaged Juju agent version 2.2.4 for amd64
>
> Launching controller instance(s) on maas-cloud...
>
> - cka68p (arch=amd64 mem=32G cores=12)
>
> Fetching Juju GUI 2.9.2
>
> Waiting for address
>
> Attempting to connect to 10.20.3.254:22 (JUJU hangs here!)
>
>
>
> And it’s pending here forever, so I tried it again with the debug mode,
>
> $ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud
> maas-cloud-controller
>
>
>
> I saw some detail information like below,
>
> Attempting to connect to 10.20.3.254:22
>
> 19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent:
> curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time
> %{time_total}s; size %{size_download} bytes; speed %{speed_download}
> bytes/s ' --retry 10 -o $bin/tools.tar.gz <[
> https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz]>
>
>
>
> Is this the last thing logged? Try running that curl command on the
> machine manually. Perhaps there's an issue getting out to the internet.
>
> *[Dave] Yes, this is the last line I saw, our network topology is MAAS
> server can access internet (dual NIC with one NIC can access outer
> network), but each node that is deployed by MAAS/JUJU only get the IP from
> internal DHCP service, do you mean each deployed node also need access
> outer internet?*
>
The bootstrap machine requires access to the Juju agent repository, which
is on "streams.canonical.com". It's possible to point Juju at a private
mirror to avoid the need to go to the internet.
*And… Do you have any idea about the ssh connection refused error?*
>
That's just log spam, we should clean that up. Juju requests the MAAS
server, and then immediately starts attempting to connect via SSH. When the
server is still starting, we expect to see connection refused errors.
>
> I have no idea what’s going wrong since I can telnet to the node and ssh
> to that node is also possible, I just need type “yes” then I can login to
> the node,
>
> $ ssh ubuntu@10.20.3.254
>
> The authenticity of host ' 10.20.3.254 (10.20.3.254)' can't be established.
>
> ECDSA key fingerprint is
> SHA256:4FVm21s4dx7gc0/yDgz0+QAMGK4qWODoIqeoWtZg9RI.
>
> Are you sure you want to continue connecting (yes/no)?
>
>
>
> From the console of that node, I can find the controller’s public key has
> been injected to the node,
>
> -BEGIN SSH HOST KEY KEYS---
>
> …
>
> -END SSH KEY FINGERPRINTS
>
> …
>
> Cloud-init v. 0.7.9 finished at … Datasource DataSourceMAAS
> [http://...:5240/MAAS/metadata/].
> Up 153.77 seconds. (cloud-init hangs here!)
>
>
>
>
>
> I googled it and found someone said it is because “authorized-keys-path”
> is commented out in the “environments.yaml” [1], but the juju version I am
> using is “2.2.4-xenial-amd64”, the MAAS version is 2.2.2,
>
> Initially, I ins
Re: juju hangs during bootstrapping
On Sat, Sep 30, 2017 at 5:55 AM wrote: > Hi Narinder, > > > > Here is the log from the deployed node, actually, I can deploy Operation > System successfully with MAAS or “juju bootstrap” but failed at some final > steps, Our external network is exactly broken for some reason yesterday and > won’t be recovered in short term, but I guess the network broken is > happened after I saw the below error message during bootstrapping. > > “Waiting for address > > Attempting to connect to 10.20.3.254:22” > > > > I can see apt-get update works from the log at the beginning, and the > network is broken couple of hours after I saw those error message. > > > > Does the error message like below has any connection with connection > refused error message? I am not quite sure. > No, they're not connected, but the failing curl command is what is preventing bootstrap from proceeding. That's the server trying to download the Juju agent. > > > “0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. > > Fetching Juju agent version 2.2.4 for amd64 > > Attempt 1 to download tools from > https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz. > .. > > curl: (6) Could not resolve host: streams.canonical.com > > “ > > > > Anyway, I will try it again when our network back to normal, it would be > great if you can see any other issue beside the network, thanks again for > your help! > > > > Best Regards, > > Dave Chen > > > > *From:* Narinder Gupta [mailto:narinder.gu...@canonical.com] > *Sent:* Friday, September 29, 2017 10:52 PM > *To:* Chen2, Dave > *Cc:* juju > *Subject:* Re: juju hangs during bootstrapping > > > > Hi Dave, > > May I know which division of Dell you are working on? As i have setup > Openstack deployed t Dell multiple time with MAAS and have not seen this > issue so far. > > > > So please send me log /var/log/cloud-init-output.log which will let us > know what is wrong. Also try sudo apt-get update on the bootstrap node to > confirm you have external access. > > > > In MAAS you can always add the ssh keys to land into the installed nodes > though. > > > > > Thanks and Regards, > > Narinder Gupta (PMP) narinder.gu...@canonical.com > > Canonical, Ltd.narindergupta [irc.freenode.net] > > +1.281.736.5150 <(281)%20736-5150> > narindergupta2007[skype] > > > > Ubuntu- Linux for human beings | www.ubuntu.com | www.canonical.com > > > > On Fri, Sep 29, 2017 at 9:42 AM, wrote: > > Hi All, > > > > I am trying to bootstrap a MAAS cloud based on juju’s official guide ( > https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct > but after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been > installed, juju hangs when attempting to connect to the MAAS node, here is > what I can see from the terminal, > > > > $ juju bootstrap maas-cloud > > Creating Juju controller "maas-cloud" on maas-cloud > > Looking for packaged Juju agent version 2.2.4 for amd64 > > Launching controller instance(s) on maas-cloud... > > - cka68p (arch=amd64 mem=32G cores=12) > > Fetching Juju GUI 2.9.2 > > Waiting for address > > Attempting to connect to 10.20.3.254:22 (JUJU hangs here!) > > > > And it’s pending here forever, so I tried it again with the debug mode, > > $ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud > maas-cloud-controller > > > > I saw some detail information like below, > > Attempting to connect to 10.20.3.254:22 > > 19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: > Connection refused > > 19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: > Connection refused > > 19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: > Connection refused > > 19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist > > 19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist > > 19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist > > 19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent: &
RE: juju hangs during bootstrapping
Can u try nslookup for archive.ubuntu.com looks like upstream dns issue as well apart from network as you mentioned it was broken. Thanks and regards, Narinder Gupta 1-2817365150 On Sep 30, 2017 04:54, wrote: > Hi Narinder, > > > > Here is the log from the deployed node, actually, I can deploy Operation > System successfully with MAAS or “juju bootstrap” but failed at some final > steps, Our external network is exactly broken for some reason yesterday and > won’t be recovered in short term, but I guess the network broken is > happened after I saw the below error message during bootstrapping. > > “Waiting for address > > Attempting to connect to 10.20.3.254:22” > > > > I can see apt-get update works from the log at the beginning, and the > network is broken couple of hours after I saw those error message. > > > > Does the error message like below has any connection with connection > refused error message? I am not quite sure. > > > > “0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. > > Fetching Juju agent version 2.2.4 for amd64 > > Attempt 1 to download tools from https://streams.canonical.com/ > juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz... > > curl: (6) Could not resolve host: streams.canonical.com > > “ > > > > Anyway, I will try it again when our network back to normal, it would be > great if you can see any other issue beside the network, thanks again for > your help! > > > > Best Regards, > > Dave Chen > > > > *From:* Narinder Gupta [mailto:narinder.gu...@canonical.com] > *Sent:* Friday, September 29, 2017 10:52 PM > *To:* Chen2, Dave > *Cc:* juju > *Subject:* Re: juju hangs during bootstrapping > > > > Hi Dave, > > May I know which division of Dell you are working on? As i have setup > Openstack deployed t Dell multiple time with MAAS and have not seen this > issue so far. > > > > So please send me log /var/log/cloud-init-output.log which will let us > know what is wrong. Also try sudo apt-get update on the bootstrap node to > confirm you have external access. > > > > In MAAS you can always add the ssh keys to land into the installed nodes > though. > > > > > Thanks and Regards, > > Narinder Gupta (PMP) narinder.gu...@canonical.com > > Canonical, Ltd.narindergupta [irc.freenode.net] > > +1.281.736.5150 <(281)%20736-5150> > narindergupta2007[skype] > > > > Ubuntu- Linux for human beings | www.ubuntu.com | www.canonical.com > > > > On Fri, Sep 29, 2017 at 9:42 AM, wrote: > > Hi All, > > > > I am trying to bootstrap a MAAS cloud based on juju’s official guide ( > https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct > but after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been > installed, juju hangs when attempting to connect to the MAAS node, here is > what I can see from the terminal, > > > > $ juju bootstrap maas-cloud > > Creating Juju controller "maas-cloud" on maas-cloud > > Looking for packaged Juju agent version 2.2.4 for amd64 > > Launching controller instance(s) on maas-cloud... > > - cka68p (arch=amd64 mem=32G cores=12) > > Fetching Juju GUI 2.9.2 > > Waiting for address > > Attempting to connect to 10.20.3.254:22 (JUJU hangs here!) > > > > And it’s pending here forever, so I tried it again with the debug mode, > > $ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud > maas-cloud-controller > > > > I saw some detail information like below, > > Attempting to connect to 10.20.3.254:22 > > 19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: > Connection refused > > 19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: > Connection refused > > 19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: > Connection refused > > 19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist > > 19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist > > 19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt > for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist > > 19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent: > c
RE: juju hangs during bootstrapping
Hi Narinder,
Here is the log from the deployed node, actually, I can deploy Operation System
successfully with MAAS or “juju bootstrap” but failed at some final steps, Our
external network is exactly broken for some reason yesterday and won’t be
recovered in short term, but I guess the network broken is happened after I
saw the below error message during bootstrapping.
“Waiting for address
Attempting to connect to 10.20.3.254:22”
I can see apt-get update works from the log at the beginning, and the network
is broken couple of hours after I saw those error message.
Does the error message like below has any connection with connection refused
error message? I am not quite sure.
“0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Fetching Juju agent version 2.2.4 for amd64
Attempt 1 to download tools from
https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz...
curl: (6) Could not resolve host: streams.canonical.com
“
Anyway, I will try it again when our network back to normal, it would be great
if you can see any other issue beside the network, thanks again for your help!
Best Regards,
Dave Chen
From: Narinder Gupta [mailto:narinder.gu...@canonical.com]
Sent: Friday, September 29, 2017 10:52 PM
To: Chen2, Dave
Cc: juju
Subject: Re: juju hangs during bootstrapping
Hi Dave,
May I know which division of Dell you are working on? As i have setup Openstack
deployed t Dell multiple time with MAAS and have not seen this issue so far.
So please send me log /var/log/cloud-init-output.log which will let us know
what is wrong. Also try sudo apt-get update on the bootstrap node to confirm
you have external access.
In MAAS you can always add the ssh keys to land into the installed nodes though.
Thanks and Regards,
Narinder Gupta (PMP)
narinder.gu...@canonical.com<mailto:narinder.gu...@canonical.com>
Canonical, Ltd.narindergupta
[irc.freenode.net<http://irc.freenode.net>]
+1.281.736.5150narindergupta2007[skype]
Ubuntu- Linux for human beings | www.ubuntu.com<http://www.ubuntu.com> |
www.canonical.com<http://www.canonical.com>
On Fri, Sep 29, 2017 at 9:42 AM,
mailto:dave.c...@dell.com>> wrote:
Hi All,
I am trying to bootstrap a MAAS cloud based on juju’s official guide
(https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct but
after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been installed,
juju hangs when attempting to connect to the MAAS node, here is what I can see
from the terminal,
$ juju bootstrap maas-cloud
Creating Juju controller "maas-cloud" on maas-cloud
Looking for packaged Juju agent version 2.2.4 for amd64
Launching controller instance(s) on maas-cloud...
- cka68p (arch=amd64 mem=32G cores=12)
Fetching Juju GUI 2.9.2
Waiting for address
Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22> (JUJU hangs
here!)
And it’s pending here forever, so I tried it again with the debug mode,
$ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud
maas-cloud-controller
I saw some detail information like below,
Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22>
19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent: curl
-sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time
%{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s '
--retry 10 -o $bin/tools.tar.gz
<[https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz]><https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz%5d%3e>
I have no idea what’s going wrong since I can telnet to the node and ssh to
that node is also possible, I just need type “yes” then I can login to the node,
$ ssh ubuntu@10.20.3.254<mailto:ubuntu@10.20.3.254>
The authenticity of host ' 10.20.3.254 (10.20.3.254)' can't be established.
ECDSA key fingerprint is SHA256:4FVm21s4dx7gc0/yDgz0+QAMGK4qWO
RE: juju hangs during bootstrapping
From: Narinder Gupta [mailto:narinder.gu...@canonical.com]
Sent: Friday, September 29, 2017 10:52 PM
To: Chen2, Dave
Cc: juju
Subject: Re: juju hangs during bootstrapping
Hi Dave,
May I know which division of Dell you are working on? As i have setup Openstack
deployed t Dell multiple time with MAAS and have not seen this issue so far.
So please send me log /var/log/cloud-init-output.log which will let us know
what is wrong. Also try sudo apt-get update on the bootstrap node to confirm
you have external access.
In MAAS you can always add the ssh keys to land into the installed nodes though.
[Dave Chen] Hi Narinder, thanks for your response, I am from Dell ESI, could
you more elaborate on how to configure to make sure ssh keys can be landed into
the installed nodes? I was thinking the ssh keys has been injected into each
node correctly, but no idea why it was rejected although I can ssh to deployed
manually, see below pls.
“
$ ssh ubuntu@10.20.3.254<mailto:ubuntu@10.20.3.254>
The authenticity of host ' 10.20.3.254 (10.20.3.254)' can't be established.
ECDSA key fingerprint is SHA256:4FVm21s4dx7gc0/yDgz0+QAMGK4qWODoIqeoWtZg9RI.
Are you sure you want to continue connecting (yes/no)?
”
If I can know how user-data is written/located it might be helpful, but I
cannot find those script ☹.
Thanks and Regards,
Narinder Gupta (PMP)
narinder.gu...@canonical.com<mailto:narinder.gu...@canonical.com>
Canonical, Ltd.narindergupta
[irc.freenode.net<http://irc.freenode.net>]
+1.281.736.5150narindergupta2007[skype]
Ubuntu- Linux for human beings | www.ubuntu.com<http://www.ubuntu.com> |
www.canonical.com<http://www.canonical.com>
On Fri, Sep 29, 2017 at 9:42 AM,
mailto:dave.c...@dell.com>> wrote:
Hi All,
I am trying to bootstrap a MAAS cloud based on juju’s official guide
(https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct but
after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been installed,
juju hangs when attempting to connect to the MAAS node, here is what I can see
from the terminal,
$ juju bootstrap maas-cloud
Creating Juju controller "maas-cloud" on maas-cloud
Looking for packaged Juju agent version 2.2.4 for amd64
Launching controller instance(s) on maas-cloud...
- cka68p (arch=amd64 mem=32G cores=12)
Fetching Juju GUI 2.9.2
Waiting for address
Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22> (JUJU hangs
here!)
And it’s pending here forever, so I tried it again with the debug mode,
$ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud
maas-cloud-controller
I saw some detail information like below,
Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22>
19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent: curl
-sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time
%{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s '
--retry 10 -o $bin/tools.tar.gz
<[https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz]><https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz%5d%3e>
I have no idea what’s going wrong since I can telnet to the node and ssh to
that node is also possible, I just need type “yes” then I can login to the node,
$ ssh ubuntu@10.20.3.254<mailto:ubuntu@10.20.3.254>
The authenticity of host ' 10.20.3.254 (10.20.3.254)' can't be established.
ECDSA key fingerprint is SHA256:4FVm21s4dx7gc0/yDgz0+QAMGK4qWODoIqeoWtZg9RI.
Are you sure you want to continue connecting (yes/no)?
From the console of that node, I can find the controller’s public key has been
injected to the node,
-BEGIN SSH HOST KEY KEYS---
…
-END SSH KEY FINGERPRINTS
…
Cloud-init v. 0.7.9 finished at … Datasource DataSourceMAAS
[http://...:5240/MAAS/metadata/]. Up 153.77 seconds. (cloud-init hangs
here!)
I googled it and found someone s
RE: juju hangs during bootstrapping
From: Andrew Wilkins [mailto:andrew.wilk...@canonical.com]
Sent: Saturday, September 30, 2017 12:23 AM
To: Chen2, Dave ; juju@lists.ubuntu.com
Subject: Re: juju hangs during bootstrapping
On Fri, Sep 29, 2017 at 10:43 AM
mailto:dave.c...@dell.com>> wrote:
Hi All,
I am trying to bootstrap a MAAS cloud based on juju’s official guide
(https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct but
after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been installed,
juju hangs when attempting to connect to the MAAS node, here is what I can see
from the terminal,
$ juju bootstrap maas-cloud
Creating Juju controller "maas-cloud" on maas-cloud
Looking for packaged Juju agent version 2.2.4 for amd64
Launching controller instance(s) on maas-cloud...
- cka68p (arch=amd64 mem=32G cores=12)
Fetching Juju GUI 2.9.2
Waiting for address
Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22> (JUJU hangs
here!)
And it’s pending here forever, so I tried it again with the debug mode,
$ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud
maas-cloud-controller
I saw some detail information like below,
Attempting to connect to 10.20.3.254:22<http://10.20.3.254:22>
19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22: Connection refused
19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt for
10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent: curl
-sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time
%{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s '
--retry 10 -o $bin/tools.tar.gz
<[https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz]><https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz%5d%3e>
Is this the last thing logged? Try running that curl command on the machine
manually. Perhaps there's an issue getting out to the internet.
[Dave] Yes, this is the last line I saw, our network topology is MAAS server
can access internet (dual NIC with one NIC can access outer network), but each
node that is deployed by MAAS/JUJU only get the IP from internal DHCP service,
do you mean each deployed node also need access outer internet?
And… Do you have any idea about the ssh connection refused error?
I have no idea what’s going wrong since I can telnet to the node and ssh to
that node is also possible, I just need type “yes” then I can login to the node,
$ ssh ubuntu@10.20.3.254<mailto:ubuntu@10.20.3.254>
The authenticity of host ' 10.20.3.254 (10.20.3.254)' can't be established.
ECDSA key fingerprint is SHA256:4FVm21s4dx7gc0/yDgz0+QAMGK4qWODoIqeoWtZg9RI.
Are you sure you want to continue connecting (yes/no)?
From the console of that node, I can find the controller’s public key has been
injected to the node,
-BEGIN SSH HOST KEY KEYS---
…
-END SSH KEY FINGERPRINTS
…
Cloud-init v. 0.7.9 finished at … Datasource DataSourceMAAS
[http://...:5240/MAAS/metadata/]. Up 153.77 seconds. (cloud-init hangs
here!)
I googled it and found someone said it is because “authorized-keys-path” is
commented out in the “environments.yaml” [1], but the juju version I am using
is “2.2.4-xenial-amd64”, the MAAS version is 2.2.2,
Initially, I installed juju 1.25 and configured environments.yaml, but now I
have uninstalled juju 1.25, removed all those file in $home/.juju/ and start it
over again with juju 2.2.4.
I really cannot figure out why it always hangs at this step, is there any cache
persisted anywhere that masked the “authorized-keys-path” even after the
uninstallation of juju1.25? or there is any step I missed with juju 2.2.4?
Where is user-data of cloud-init persisted on the filesystem? Any more detail
logs I can refer to?
I feel frustration after trying several days without any progress, pls help me
out, many many thanks for any inputs!
[1]
https://serverfault.com/questions/588967/juju-bootstrap-fails-connection-refused-port-22
Best Regards,
Dave Chen
Best Regards,
Dave Chen
--
Juju mailing list
Juju@lists.ubuntu.com<mailto:Juju@lists.ubuntu.com>
Modify settings or uns
Re: juju hangs during bootstrapping
On Fri, Sep 29, 2017 at 10:43 AM wrote:
> Hi All,
>
>
>
> I am trying to bootstrap a MAAS cloud based on juju’s official guide (
> https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct
> but after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been
> installed, juju hangs when attempting to connect to the MAAS node, here is
> what I can see from the terminal,
>
>
>
> $ juju bootstrap maas-cloud
>
> Creating Juju controller "maas-cloud" on maas-cloud
>
> Looking for packaged Juju agent version 2.2.4 for amd64
>
> Launching controller instance(s) on maas-cloud...
>
> - cka68p (arch=amd64 mem=32G cores=12)
>
> Fetching Juju GUI 2.9.2
>
> Waiting for address
>
> Attempting to connect to 10.20.3.254:22 (JUJU hangs here!)
>
>
>
> And it’s pending here forever, so I tried it again with the debug mode,
>
> $ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud
> maas-cloud-controller
>
>
>
> I saw some detail information like below,
>
> Attempting to connect to 10.20.3.254:22
>
> 19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent:
> curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time
> %{time_total}s; size %{size_download} bytes; speed %{speed_download}
> bytes/s ' --retry 10 -o $bin/tools.tar.gz <[
> https://streams.canonical.com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz]>
>
Is this the last thing logged? Try running that curl command on the machine
manually. Perhaps there's an issue getting out to the internet.
>
> I have no idea what’s going wrong since I can telnet to the node and ssh
> to that node is also possible, I just need type “yes” then I can login to
> the node,
>
> $ ssh ubuntu@10.20.3.254
>
> The authenticity of host ' 10.20.3.254 (10.20.3.254)' can't be established.
>
> ECDSA key fingerprint is
> SHA256:4FVm21s4dx7gc0/yDgz0+QAMGK4qWODoIqeoWtZg9RI.
>
> Are you sure you want to continue connecting (yes/no)?
>
>
>
> From the console of that node, I can find the controller’s public key has
> been injected to the node,
>
> -BEGIN SSH HOST KEY KEYS---
>
> …
>
> -END SSH KEY FINGERPRINTS
>
> …
>
> Cloud-init v. 0.7.9 finished at … Datasource DataSourceMAAS
> [http://...:5240/MAAS/metadata/].
> Up 153.77 seconds. (cloud-init hangs here!)
>
>
>
>
>
> I googled it and found someone said it is because “authorized-keys-path”
> is commented out in the “environments.yaml” [1], but the juju version I am
> using is “2.2.4-xenial-amd64”, the MAAS version is 2.2.2,
>
> Initially, I installed juju 1.25 and configured environments.yaml, but now
> I have uninstalled juju 1.25, removed all those file in $home/.juju/ and
> start it over again with juju 2.2.4.
>
> I really cannot figure out why it always hangs at this step, is there any
> cache persisted anywhere that masked the “authorized-keys-path” even after
> the uninstallation of juju1.25? or there is any step I missed with juju
> 2.2.4?
>
>
>
> Where is user-data of cloud-init persisted on the filesystem? Any more
> detail logs I can refer to?
>
>
>
>
>
> I feel frustration after trying several days without any progress, pls
> help me out, many many thanks for any inputs!
>
>
>
>
>
> [1]
> https://serverfault.com/questions/588967/juju-bootstrap-fails-connection-refused-port-22
>
>
>
>
>
> Best Regards,
>
> Dave Chen
>
>
>
> Best Regards,
>
> Dave Chen
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: juju hangs during bootstrapping
Hi Dave,
May I know which division of Dell you are working on? As i have setup
Openstack deployed t Dell multiple time with MAAS and have not seen this
issue so far.
So please send me log /var/log/cloud-init-output.log which will let us know
what is wrong. Also try sudo apt-get update on the bootstrap node to
confirm you have external access.
In MAAS you can always add the ssh keys to land into the installed nodes
though.
Thanks and Regards,
Narinder Gupta (PMP) narinder.gu...@canonical.com
Canonical, Ltd.narindergupta [irc.freenode.net]
+1.281.736.5150narindergupta2007[skype]
Ubuntu- Linux for human beings | www.ubuntu.com | www.canonical.com
On Fri, Sep 29, 2017 at 9:42 AM, wrote:
> Hi All,
>
>
>
> I am trying to bootstrap a MAAS cloud based on juju’s official guide (
> https://jujucharms.com/docs/2.2/clouds-maas), everything seems correct
> but after the Operation System (Ubuntu 16.04 or Ubuntu14.0) has been
> installed, juju hangs when attempting to connect to the MAAS node, here is
> what I can see from the terminal,
>
>
>
> $ juju bootstrap maas-cloud
>
> Creating Juju controller "maas-cloud" on maas-cloud
>
> Looking for packaged Juju agent version 2.2.4 for amd64
>
> Launching controller instance(s) on maas-cloud...
>
> - cka68p (arch=amd64 mem=32G cores=12)
>
> Fetching Juju GUI 2.9.2
>
> Waiting for address
>
> Attempting to connect to 10.20.3.254:22 (JUJU hangs here!)
>
>
>
> And it’s pending here forever, so I tried it again with the debug mode,
>
> $ juju bootstrap --show-log --debug --bootstrap-series=trusty maas-cloud
> maas-cloud-controller
>
>
>
> I saw some detail information like below,
>
> Attempting to connect to 10.20.3.254:22
>
> 19:33:11 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:16 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:21 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: ssh: connect to host 10.20.3.254 port 22:
> Connection refused
>
> 19:33:56 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:34:32 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:35:08 DEBUG juju.provider.common bootstrap.go:497 connection attempt
> for 10.20.3.254 failed: /var/lib/juju/nonce.txt does not exist
>
> 19:35:43 INFO juju.cloudconfig userdatacfg_unix.go:410 Fetching agent:
> curl -sSfw 'tools from %{url_effective} downloaded: HTTP %{http_code}; time
> %{time_total}s; size %{size_download} bytes; speed %{speed_download}
> bytes/s ' --retry 10 -o $bin/tools.tar.gz <[https://streams.canonical.
> com/juju/tools/agent/2.2.4/juju-2.2.4-ubuntu-amd64.tgz]>
>
>
>
> I have no idea what’s going wrong since I can telnet to the node and ssh
> to that node is also possible, I just need type “yes” then I can login to
> the node,
>
> $ ssh ubuntu@10.20.3.254
>
> The authenticity of host ' 10.20.3.254 (10.20.3.254)' can't be established.
>
> ECDSA key fingerprint is SHA256:4FVm21s4dx7gc0/yDgz0+
> QAMGK4qWODoIqeoWtZg9RI.
>
> Are you sure you want to continue connecting (yes/no)?
>
>
>
> From the console of that node, I can find the controller’s public key has
> been injected to the node,
>
> -BEGIN SSH HOST KEY KEYS---
>
> …
>
> -END SSH KEY FINGERPRINTS
>
> …
>
> Cloud-init v. 0.7.9 finished at … Datasource DataSourceMAAS [http://
> ...:5240/MAAS/metadata/]. Up 153.77 seconds. (cloud-init hangs
> here!)
>
>
>
>
>
> I googled it and found someone said it is because “authorized-keys-path”
> is commented out in the “environments.yaml” [1], but the juju version I am
> using is “2.2.4-xenial-amd64”, the MAAS version is 2.2.2,
>
> Initially, I installed juju 1.25 and configured environments.yaml, but now
> I have uninstalled juju 1.25, removed all those file in $home/.juju/ and
> start it over again with juju 2.2.4.
>
> I really cannot figure out why it always hangs at this step, is there any
> cache persisted anywhere that masked the “authorized-keys-path” even after
> the uninstallation of juju1.25? or there is any step I missed with juju
> 2.2.4?
>
>
>
> Where is user-data of cloud-init persisted on the filesystem? Any more
> detail logs I can refer to?
>
>
>
>
>
> I feel frustration after trying several days without any progress, pls
> help me out, many many thanks for any inputs!
>
>
>
>
>
> [1] https://serverfault.com/questions/588967/juju-
> bootstrap-fails-connection-refused-port-22
>
>
>
>
>
> Best Regards,
>
> Dave Chen
>
>
>
> Best Regards,
>
> Dave Chen
>
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.c
