Our configs agree, our deployments may not. We're running a cluster here. Taking a closer look at the back trace. I'm missing the failure messages for parsing the the availabilityzone. There are a few conditionals in there that just return Trace (I don't know what that means) and doesn't send anything to the logger. The admin claims there's no availability zone set. Back to the logs.
17:21:08 DEBUG juju.service discovery.go:62 discovered init system "systemd" from series "xenial" 17:21:08 DEBUG juju.provider.vmware environ_broker.go:93 Vmware user data; 1856 bytes 17:21:09 INFO juju.provider.common destroy.go:20 destroying model "controller" 17:21:09 INFO juju.provider.common destroy.go:31 destroying instances 17:21:10 ERROR juju.cmd.juju.commands bootstrap.go:918 error cleaning up: destroying instances: SystemError 17:21:10 ERROR cmd supercommand.go:458 failed to bootstrap model: cannot start bootstrap instance: SystemError We definitely get here. juju.provider.vmware environ_broker.go:93 but we don't see any other messages, failure or otherwise from that routine. The back trace says I get to L121, which has me failing out of parsing availability zones, but doesn't inform the logger. 17:08:52 ERROR cmd supercommand.go:458 failed to bootstrap model: cannot start bootstrap instance: SystemError 17:08:52 DEBUG cmd supercommand.go:459 (error details: [{github.com/juju/juju/cmd/juju/commands/bootstrap.go:807: failed to bootstrap model} {github.com/juju/juju/provider/common/bootstrap.go:47: } {github.com/juju/juju/provider/common/bootstrap.go:173: cannot start bootstrap instance} {github.com/juju/juju/provider/vsphere/environ_broker.go:44: } {github.com/juju/juju/provider/vsphere/environ_broker.go:121: } {github.com/juju/juju/provider/vsphere/environ_availzones.go:116: } 114 zoneInstances, err := AvailabilityZoneAllocations(env, group) 115 if err != nil { 116 return nil, errors.Trace(err) 117 } That's the bug. It doesn't figure out what zone to place it in. If I specify the zone as Cluster1, the only zone we have, it works. juju bootstrap --to zone=Cluster1 vsphere vsphere --debug 17:37:38 INFO juju.cmd supercommand.go:63 running juju [2.0-rc3 gc go1.6.3] 17:37:38 DEBUG juju.cmd supercommand.go:64 args: []string{"juju", "bootstrap", "--to", "zone=Cluster1", "vsphere", "vsphere", "--debug"} 17:37:38 DEBUG juju.cmd.juju.commands bootstrap.go:537 provider attrs: map[] 17:37:38 INFO cmd cmd.go:141 Adding contents of "/root/.local/share/juju/ssh/juju_id_rsa.pub" to authorized-keys 17:37:38 DEBUG juju.cmd.juju.commands bootstrap.go:595 preparing controller with config: map[type:vsphere name:controller uuid:22ddb73b-4409-41a1-806d-0a8494870754 authorized-keys:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDagqFodKywSsG7itNj/J3JYYGdOuXA5QDiJ0FJuiLLag/W7cir+e3ofk2ly5J6U+um5edi7fI9sR1bEzCAFWZKs0mRI+6DcjkzxBPC0LsjmDx8rhk4b5l9er+c93UgyTYiBpsopYzekzQI2b+KSLI9t1SUSMUmS1sW7sTaw3KcSFfZYWb7jeQnXLK2EOV2NcTlcmlTEohKZZUYVSP5uth451mqViTB8BCbsYCtGtIeqvhkmdGVnj3qErn4rM0zcyeNHoTKv8rcERaNcEQUCqM5sk4z+d62NONuNUMtA/8RuzzBOMP3E89h8iy59zVP0LRPtaddl6K8fN4wB9wSgv juju-client-key ] 17:37:38 INFO cmd cmd.go:129 Creating Juju controller "vsphere" on vsphere/Datacenter 17:37:38 DEBUG juju.environs imagemetadata.go:112 obtained image datasource "default cloud images" 17:37:38 DEBUG juju.environs imagemetadata.go:112 obtained image datasource "default ubuntu cloud images" 17:37:39 DEBUG juju.environs.simplestreams simplestreams.go:454 skipping index "https://streams.canonical.com/juju/images/releases/streams/v1/index.sjson" because of missing information: "image-downloads" data not found 17:37:41 INFO juju.cmd.juju.commands bootstrap.go:738 combined bootstrap constraints: 17:37:41 DEBUG juju.environs.bootstrap bootstrap.go:185 model "controller" supports service/machine networks: true 17:37:41 DEBUG juju.environs.bootstrap bootstrap.go:187 network management by juju enabled: true 17:37:41 INFO cmd cmd.go:141 Loading image metadata 17:37:41 INFO cmd cmd.go:129 Looking for packaged Juju agent version 2.0-rc3 for amd64 17:37:41 INFO juju.environs.bootstrap tools.go:72 looking for bootstrap agent binaries: version=2.0-rc3 17:37:41 INFO juju.environs.tools tools.go:101 finding agent binaries in stream "devel" 17:37:41 INFO juju.environs.tools tools.go:103 reading agent binaries with major.minor version 2.0 17:37:41 INFO juju.environs.tools tools.go:111 filtering agent binaries by version: 2.0-rc3 17:37:41 INFO juju.environs.tools tools.go:117 filtering agent binaries by architecture: amd64 17:37:41 DEBUG juju.environs.tools urls.go:109 trying datasource "keystone catalog" 17:37:42 DEBUG juju.environs.simplestreams simplestreams.go:680 using default candidate for content id "com.ubuntu.juju:devel:tools" are {20161007 mirrors:1.0 content-download streams/v1/cpc-mirrors.sjson []} 17:37:43 INFO juju.environs.bootstrap tools.go:74 found 15 packaged agent binaries 17:37:43 INFO cmd cmd.go:141 Starting new instance for initial controller Launching controller instance(s) on vsphere/Datacenter... 17:37:43 DEBUG juju.environs imagemetadata.go:112 obtained image datasource "default cloud images" 17:37:43 DEBUG juju.environs imagemetadata.go:112 obtained image datasource "default ubuntu cloud images" 17:37:44 DEBUG juju.environs.simplestreams simplestreams.go:454 skipping index "https://streams.canonical.com/juju/images/releases/streams/v1/index.sjson" because of missing information: "image-downloads" data not found 17:37:46 DEBUG juju.cloudconfig.instancecfg instancecfg.go:782 Setting numa ctl preference to false 17:37:46 DEBUG juju.service discovery.go:62 discovered init system "systemd" from series "xenial" 17:37:46 DEBUG juju.provider.vmware environ_broker.go:93 Vmware user data; 1856 bytes 17:37:46 DEBUG juju.provider.vmware ova_import_manager.go:180 Downloading ova file from url: http://cloud-images.ubuntu.com/releases/server/releases/xenial/release-20161020/ubuntu-16.04-server-cloudimg-amd64.ova 17:37:46 DEBUG juju.provider.vmware ova_import_manager.go:207 Extracting ova to path: /tmp/558696814 17:37:46 DEBUG juju.provider.vmware ova_import_manager.go:223 Writing file ubuntu-xenial-16.04-cloudimg.ovf 17:37:46 DEBUG juju.provider.vmware ova_import_manager.go:223 Writing file ubuntu-xenial-16.04-cloudimg.mf 17:37:46 DEBUG juju.provider.vmware ova_import_manager.go:223 Writing file ubuntu-xenial-16.04-cloudimg.vmdk 17:38:20 DEBUG juju.provider.vmware ova_import_manager.go:243 Ova extracted successfully 17:38:23 DEBUG juju.provider.vmware ova_import_manager.go:249 Uploading item from path: /tmp/558696814/ubuntu-xenial-16.04-cloudimg.vmdk 17:38:23 DEBUG juju.provider.vmware ova_import_manager.go:264 Uploading image to https://cio-esx-4.cacheio.com/nfc/526fb143-5e99-9d36-c775-581f0d33736f/disk-0.vmdk 17:38:26 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 10% 17:38:29 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 20% 17:38:32 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 30% 17:38:35 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 40% 17:38:38 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 50% 17:38:41 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 60% 17:38:44 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 70% 17:38:48 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 80% 17:38:52 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 90% 17:38:56 DEBUG juju.provider.vmware ova_import_manager.go:271 Progress: 100% 17:38:57 DEBUG juju.provider.vmware ova_import_manager.go:277 Image uploaded 17:38:59 INFO juju.provider.vmware environ_broker.go:47 started instance "juju-870754-0" - juju-870754-0 17:38:59 INFO juju.environs.bootstrap bootstrap.go:587 newest version: 2.0-rc3 17:38:59 INFO juju.environs.bootstrap bootstrap.go:602 picked bootstrap tools version: 2.0-rc3 17:38:59 INFO juju.environs.bootstrap bootstrap.go:378 Installing Juju agent on bootstrap instance 17:39:00 INFO cmd cmd.go:129 Fetching Juju GUI 2.2.1 17:39:00 DEBUG juju.cloudconfig.instancecfg instancecfg.go:782 Setting numa ctl preference to false Waiting for address 17:39:01 ERROR juju.provider.vmware environ_instance.go:33 failed to get instances from vmware: SystemError 17:39:01 INFO juju.provider.common destroy.go:20 destroying model "controller" 17:39:01 INFO juju.provider.common destroy.go:31 destroying instances 17:39:02 ERROR juju.cmd.juju.commands bootstrap.go:918 error cleaning up: destroying instances: SystemError 17:39:02 ERROR cmd supercommand.go:458 failed to bootstrap model: refreshing addresses: SystemError 17:39:02 DEBUG cmd supercommand.go:459 (error details: [{github.com/juju/juju/cmd/juju/commands/bootstrap.go:807: failed to bootstrap model} {refreshing addresses: SystemError}]) It in fact created "juju-870754-0" but can't talk to it because it got on the wrong network. It got on the cluster network, 10G-Cluster, it should have gotten on DPortGroup. How do I tell it to use the right network? The docs are a little unclear on the subject. All.. most ... there.. :) Thanks! Peter On Mon, Oct 24, 2016 at 1:15 PM, Horacio Duran <horacio.du...@canonical.com> wrote: > This is how mine looks > > clouds: > vsphere: > type: vsphere > auth-types: [userpass] > endpoint: <ip redacted> > regions: > dc0: {} > > > On Mon, Oct 24, 2016 at 2:10 PM, Peter Petrakis <peter.petra...@cacheio.com> > wrote: >> >> On Mon, Oct 24, 2016 at 12:19 PM, Horacio Duran >> <horacio.du...@canonical.com> wrote: >> > Chance seems to be in the right path there. >> > In the example "dc0" is the name of our data center, to figure out yours >> > you >> > can access the vsphere web client and in there go to: >> > Home->Vcenter inventory lists->Resources->Datacenters (all this in the >> > left >> > tree like menu) >> > And you will be presented with a list of your datacenters in the right >> > pane >> > under the "Objects" tab, the "Name" column holds what you must put in >> > place >> > of "dc0" . >> > Cheers >> >> Looks like I got the datacenter name correct by accident :) >> >> From the gui: >> >> vcenter.cacheio.com [10.2.0.8] -> Datacenter -> Cluster1 -> >> [cio-esx1.cacheio.com, cio-esx2-cacheio.com ] >> >> We renamed Datacenter -> dc0 just to see what would happen, same end >> result. >> >> We're using vsphere 6.0u2. >> >> root@juju-vmware:~# cat /root/.local/share/juju/clouds.yaml >> clouds: >> myvscloud: >> type: vsphere >> auth-types: [userpass] >> endpoint: 10.2.0.8 >> regions: >> Datacenter: >> endpoint: 10.2.0.8 >> >> We're on ESXi 6 which I'm told provides HW version 11. The doc says >> the min is 8. >> >> Thanks. >> >> Peter >> >> > >> > On Mon, Oct 24, 2016 at 1:10 PM, Chance Ellis <chance_el...@yahoo.com> >> > wrote: >> >> >> >> Hi Peter, >> >> >> >> Is your vsphere data center actually named dc0? >> >> >> >> “15:38:47 ERROR cmd supercommand.go:458 failed to create new client: >> >> datacenter 'dc0' not found” >> >> >> >> >> >> The page you referenced gave an example with a data center named dc0. I >> >> imagine yours is named something different? >> >> >> >> >> >> >> >> On 10/24/16, 12:06 PM, "Peter Petrakis" <juju-boun...@lists.ubuntu.com >> >> on >> >> behalf of peter.petra...@cacheio.com> wrote: >> >> >> >> Hi All, >> >> >> >> I have 16.10 running in a container and am testing this against our >> >> datacenter using the docs found here: >> >> >> >> https://jujucharms.com/docs/2.0/help-vmware >> >> >> >> Which advises the following config: >> >> >> >> root@juju-vmware:~# cat /root/.local/share/juju/clouds.yaml >> >> clouds: >> >> myvscloud: >> >> type: vsphere >> >> auth-types: [userpass] >> >> endpoint: 10.2.0.8 >> >> regions: >> >> dc0: >> >> endpoint: 10.2.0.8 >> >> >> >> >> >> >> >> This does not work. >> >> >> >> root@juju-vmware:~# juju bootstrap vsphere myvscloud --debug >> >> >> >> 15:38:47 INFO juju.cmd supercommand.go:63 running juju [2.0-rc3 gc >> >> go1.6.3] >> >> 15:38:47 DEBUG juju.cmd supercommand.go:64 args: []string{"juju", >> >> "bootstrap", "vsphere", "myvscloud", "--debug"} >> >> 15:38:47 DEBUG juju.cmd.juju.commands bootstrap.go:537 provider >> >> attrs: >> >> map[] >> >> 15:38:47 INFO cmd cmd.go:141 Adding contents of >> >> "/root/.local/share/juju/ssh/juju_id_rsa.pub" to authorized-keys >> >> 15:38:47 DEBUG juju.cmd.juju.commands bootstrap.go:595 preparing >> >> controller with config: map[authorized-keys:ssh-rsa >> >> >> >> >> >> AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDagqFodKywSsG7itNj/J3JYYGdOuXA5QDiJ0FJuiLLag/W7cir+e3ofk2ly5J6U+um5edi7fI9sR1bEzCAFWZKs0mRI+6DcjkzxBPC0LsjmDx8rhk4b5l9er+c93UgyTYiBpsopYzekzQI2b+KSLI9t1SUSMUmS1sW7sTaw3KcSFfZYWb7jeQnXLK2EOV2NcTlcmlTEohKZZUYVSP5uth451mqViTB8BCbsYCtGtIeqvhkmdGVnj3qErn4rM0zcyeNHoTKv8rcERaNcEQUCqM5sk4z+d62NONuNUMtA/8RuzzBOMP3E89h8iy59zVP0LRPtaddl6K8fN4wB9wSgv >> >> juju-client-key >> >> type:vsphere name:controller >> >> uuid:e2f16e4a-fb8d-4a25-8686-81dc190894b9] >> >> 15:38:47 ERROR cmd supercommand.go:458 failed to create new client: >> >> datacenter 'dc0' not found >> >> 15:38:47 DEBUG cmd supercommand.go:459 (error details: >> >> [{github.com/juju/juju/cmd/juju/commands/bootstrap.go:662: } >> >> {github.com/juju/juju/environs/bootstrap/prepare.go:99: } >> >> {github.com/juju/juju/environs/bootstrap/prepare.go:160: } >> >> {github.com/juju/juju/provider/vsphere/provider.go:32: } >> >> {github.com/juju/juju/provider/vsphere/environ.go:47: failed to >> >> create >> >> new client} {github.com/juju/juju/provider/vsphere/client.go:61: } >> >> {datacenter 'dc0' not found}]) >> >> >> >> Now I look at the source and discover that "dc0" cannot be some >> >> arbitrary string, it must be named "Datacenter". >> >> >> >> # >> >> ~/juju-src/juju-core-2.0~rc3/src/github.com/juju/govmomi/find/finder.go >> >> func (f *Finder) DatacenterList(ctx context.Context, path >> >> ...string) >> >> ([]*object.Datacenter, error) { >> >> es, err := f.find(ctx, f.rootFolder, false, path...) >> >> if err != nil { >> >> return nil, err >> >> } >> >> >> >> var dcs []*object.Datacenter >> >> for _, e := range es { >> >> ref := e.Object.Reference() >> >> if ref.Type == "Datacenter" { >> >> dcs = append(dcs, >> >> object.NewDatacenter(f.client, ref)) >> >> } >> >> } >> >> >> >> return dcs, nil >> >> } >> >> >> >> Which gets me a whole lot further but not quite to the finish line. >> >> >> >> root@juju-vmware:~# juju bootstrap vsphere myvscloud --debug >> >> 16:04:15 INFO juju.cmd supercommand.go:63 running juju [2.0-rc3 gc >> >> go1.6.3] >> >> 16:04:15 DEBUG juju.cmd supercommand.go:64 args: []string{"juju", >> >> "bootstrap", "vsphere", "myvscloud", "--debug"} >> >> 16:04:15 DEBUG juju.cmd.juju.commands bootstrap.go:537 provider >> >> attrs: >> >> map[] >> >> 16:04:15 INFO cmd cmd.go:141 Adding contents of >> >> "/root/.local/share/juju/ssh/juju_id_rsa.pub" to authorized-keys >> >> 16:04:15 DEBUG juju.cmd.juju.commands bootstrap.go:595 preparing >> >> controller with config: map[name:controller >> >> uuid:cde9a49f-4a06-4d96-8197-87ddad561abf authorized-keys:ssh-rsa >> >> >> >> >> >> AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDagqFodKywSsG7itNj/J3JYYGdOuXA5QDiJ0FJuiLLag/W7cir+e3ofk2ly5J6U+um5edi7fI9sR1bEzCAFWZKs0mRI+6DcjkzxBPC0LsjmDx8rhk4b5l9er+c93UgyTYiBpsopYzekzQI2b+KSLI9t1SUSMUmS1sW7sTaw3KcSFfZYWb7jeQnXLK2EOV2NcTlcmlTEohKZZUYVSP5uth451mqViTB8BCbsYCtGtIeqvhkmdGVnj3qErn4rM0zcyeNHoTKv8rcERaNcEQUCqM5sk4z+d62NONuNUMtA/8RuzzBOMP3E89h8iy59zVP0LRPtaddl6K8fN4wB9wSgv >> >> juju-client-key >> >> type:vsphere] >> >> 16:04:15 INFO cmd cmd.go:129 Creating Juju controller "vsphere" on >> >> myvscloud/Datacenter >> >> 16:04:15 DEBUG juju.environs imagemetadata.go:112 obtained image >> >> datasource "default cloud images" >> >> 16:04:15 DEBUG juju.environs imagemetadata.go:112 obtained image >> >> datasource "default ubuntu cloud images" >> >> 16:04:16 DEBUG juju.environs.simplestreams simplestreams.go:454 >> >> skipping index >> >> >> >> "https://streams.canonical.com/juju/images/releases/streams/v1/index.sjson" >> >> because of missing information: "image-downloads" data not found >> >> 16:04:18 INFO juju.cmd.juju.commands bootstrap.go:738 combined >> >> bootstrap constraints: >> >> 16:04:18 DEBUG juju.environs.bootstrap bootstrap.go:185 model >> >> "controller" supports service/machine networks: true >> >> 16:04:18 DEBUG juju.environs.bootstrap bootstrap.go:187 network >> >> management by juju enabled: true >> >> 16:04:18 INFO cmd cmd.go:141 Loading image metadata >> >> 16:04:18 INFO cmd cmd.go:129 Looking for packaged Juju agent >> >> version >> >> 2.0-rc3 for amd64 >> >> 16:04:18 INFO juju.environs.bootstrap tools.go:72 looking for >> >> bootstrap agent binaries: version=2.0-rc3 >> >> 16:04:18 INFO juju.environs.tools tools.go:101 finding agent >> >> binaries >> >> in stream "devel" >> >> 16:04:18 INFO juju.environs.tools tools.go:103 reading agent >> >> binaries >> >> with major.minor version 2.0 >> >> 16:04:18 INFO juju.environs.tools tools.go:111 filtering agent >> >> binaries by version: 2.0-rc3 >> >> 16:04:18 INFO juju.environs.tools tools.go:117 filtering agent >> >> binaries by architecture: amd64 >> >> 16:04:18 DEBUG juju.environs.tools urls.go:109 trying datasource >> >> "keystone catalog" >> >> 16:04:18 DEBUG juju.environs.simplestreams simplestreams.go:680 >> >> using >> >> default candidate for content id "com.ubuntu.juju:devel:tools" are >> >> {20161007 mirrors:1.0 content-download streams/v1/cpc-mirrors.sjson >> >> []} >> >> 16:04:20 INFO juju.environs.bootstrap tools.go:74 found 15 >> >> packaged >> >> agent binaries >> >> 16:04:20 INFO cmd cmd.go:141 Starting new instance for initial >> >> controller >> >> Launching controller instance(s) on myvscloud/Datacenter... >> >> 16:04:20 DEBUG juju.environs imagemetadata.go:112 obtained image >> >> datasource "default cloud images" >> >> 16:04:20 DEBUG juju.environs imagemetadata.go:112 obtained image >> >> datasource "default ubuntu cloud images" >> >> 16:04:21 DEBUG juju.environs.simplestreams simplestreams.go:454 >> >> skipping index >> >> >> >> "https://streams.canonical.com/juju/images/releases/streams/v1/index.sjson" >> >> because of missing information: "image-downloads" data not found >> >> 16:04:22 DEBUG juju.cloudconfig.instancecfg instancecfg.go:782 >> >> Setting >> >> numa ctl preference to false >> >> 16:04:22 DEBUG juju.service discovery.go:62 discovered init system >> >> "systemd" from series "xenial" >> >> 16:04:22 DEBUG juju.provider.vmware environ_broker.go:93 Vmware >> >> user >> >> data; 1856 bytes >> >> 16:04:23 INFO juju.provider.common destroy.go:20 destroying model >> >> "controller" >> >> 16:04:23 INFO juju.provider.common destroy.go:31 destroying >> >> instances >> >> 16:04:24 ERROR juju.cmd.juju.commands bootstrap.go:918 error >> >> cleaning >> >> up: destroying instances: SystemError >> >> 16:04:24 ERROR cmd supercommand.go:458 failed to bootstrap model: >> >> cannot start bootstrap instance: SystemError >> >> 16:04:24 DEBUG cmd supercommand.go:459 (error details: >> >> [{github.com/juju/juju/cmd/juju/commands/bootstrap.go:807: failed >> >> to >> >> bootstrap model} >> >> {github.com/juju/juju/provider/common/bootstrap.go:47: } >> >> {github.com/juju/juju/provider/common/bootstrap.go:173: cannot >> >> start >> >> bootstrap instance} >> >> {github.com/juju/juju/provider/vsphere/environ_broker.go:44: } >> >> {github.com/juju/juju/provider/vsphere/environ_broker.go:121: } >> >> {github.com/juju/juju/provider/vsphere/environ_availzones.go:116: } >> >> {github.com/juju/juju/provider/vsphere/environ_broker.go:162: } >> >> {github.com/juju/juju/provider/vsphere/environ_instance.go:66: } >> >> {github.com/juju/juju/provider/vsphere/client.go:173: } >> >> {SystemError}]) >> >> >> >> What am I missing? Thanks. >> >> >> >> Peter >> >> >> >> -- >> >> Juju mailing list >> >> Juju@lists.ubuntu.com >> >> Modify settings or unsubscribe at: >> >> https://lists.ubuntu.com/mailman/listinfo/juju >> >> >> >> >> >> >> >> >> >> -- >> >> Juju mailing list >> >> Juju@lists.ubuntu.com >> >> Modify settings or unsubscribe at: >> >> https://lists.ubuntu.com/mailman/listinfo/juju >> > >> > > > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju