Re: [Openstack] [Fuel] node name issue

[Jim Okken]

update to an old question:
I have gotten around this issue.

I'm not sure how I got around this issue, but my theory is something I
noticed quite by accident.

The servers I was having these troubles on use 80GB hard drives, but they
also have a flash drive in them, for small OS deployments. I stumbled
across a /dev/sda6 partition on these flash drives.

On this partition I found 2 files: meta-data and user-data. In those files
was the old node name I mentioned in the original post. This partition must
have been detected, and these stale files were used by fuel-agent when the
newly provisioned node first booted, even though they were booting from the
80GB drive which had its own /dev/sda6 partition.

I suspect the stale/incorrect /dev/sda6 is probably is from Fuel 8 when we
tried to deploy on some of these flash drives...

once I deleted the stale/incorrect /dev/sda6 then theprovisioning and
deployment went perfectly

thanks

-- Jim

On Thu, Sep 28, 2017 at 5:02 PM, Jim Okken  wrote:

> I ran  "fuel2 node update -H blade13 20" just to get out of the node-*
> naming convention, as someone suggested
>
>
>
> The deploy still names the node node-11 and provisioning fails.
>
> digging a little more, i see it might have to do with the fuel-agent
> cloud-init scripts.
>
> in the cloud-init.log on the new node I see the node name being set to
> node-11!
>
>
>
> this isnt node-11. this was node-20, but I renamed it to blade13 with the
> command "fuel2 node update -H blade13 20"
>
>
>
> i also noted that after the cloud-init scripts ran at the end the first
> boot of the new provisioned OS, that in the Fuel GUI, the FQDN field became
> node-11.ourdomain.com (before it was bootstrap.ourdomain.com)
>
> (in the same window Hostname still show as blade13)
>
>
>
> But FQDN in the Fuel2 CLI output still shows  node-20.ourdomain.com!!!
>
>
>
> [fuel2 node show 20
>
> | id  | 20
>  |
>
> | name| Untitled (68:58)
>   |
>
> | status  | ready
>  |
>
> | os_platform | ubuntu
>  |
>
> | roles   | [u'compute']
>  |
>
> | kernel_params   | None
>  |
>
> | pending_roles   | []
>  |
>
> | hostname| node-20
>  |
>
> | fqdn| node-20.dialogic.com
>  |
>
> | platform_name   | ProLiant BL460c Gen9
>  |
>
>
>
>
>
>
>
>
>
>
>
> where can i find the cloud init settings which are deploy to new nodes?
>
> i guess this has something to do with this file:
> /usr/share/fuel-agent/cloud-init-templates/cloud_config_ubuntu.jinja2
>
> in that file I see
>
> hostname: {{ common.hostname }}
>
> fqdn: {{ common.fqdn }}
>
>
>
> please help me with an info you might have or let me know that populates
> those 2 parts of the template?
>
>
>
> Is there a database these values are all stored in on the fuel server?
>
>
>
> Thanks
>
>
>
> --Jim
>
> -- Jim
>
> On Tue, Sep 26, 2017 at 12:00 PM, Jim Okken  wrote:
>
>> also I should add, I dont have the original hard drives in the system so
>> it isn't because it is booting the old OS where these node names were set.
>> this is definitely the newly installed OS being given the wroing hostname
>>
>>
>>
>> is there a database this is all kept in? maybe I could look around and
>> find where these old node names are being saved?
>>
>> thanks!
>>
>> -- Jim
>>
>> On Mon, Sep 25, 2017 at 6:03 PM, Jim Okken  wrote:
>>
>>> hi all,
>>>
>>> I am using Fuel 10.
>>>
>>> i have 2 nodes I am trying to deploy as compute nodes. at one time in
>>> the past I was attempting to deploy them too. I assume back then their node
>>> names were node-11 and node-20.
>>>
>>> they were never successfully deploy and now I've worked out their
>>> hardware issues and are attempting to deploy them again. now Fuel has given
>>> them the names node-80 and node-81.
>>> (i may be at 80 in my node names but I only have 17 nodes so far)
>>>
>>> the deploy of these 2 nodes does not get past installing Ubuntu. The
>>> nodes reboot after Ubuntu is installed and come up incorrectly as node-11
>>> and node-20. After that Fuel sits for a long while and then gives an error
>>> (pasted at the end of email). I assume the nodes come up with the wrong
>>> name/ip/ssh-key and Fuel can't contact them.
>>>
>>> I'm a novice at using 

Re: [Openstack] [Fuel] node name issue

[Jim Okken]

I ran  "fuel2 node update -H blade13 20" just to get out of the node-*
naming convention, as someone suggested



The deploy still names the node node-11 and provisioning fails.

digging a little more, i see it might have to do with the fuel-agent
cloud-init scripts.

in the cloud-init.log on the new node I see the node name being set to
node-11!



this isnt node-11. this was node-20, but I renamed it to blade13 with the
command "fuel2 node update -H blade13 20"



i also noted that after the cloud-init scripts ran at the end the first
boot of the new provisioned OS, that in the Fuel GUI, the FQDN field became
node-11.ourdomain.com (before it was bootstrap.ourdomain.com)

(in the same window Hostname still show as blade13)



But FQDN in the Fuel2 CLI output still shows  node-20.ourdomain.com!!!



[fuel2 node show 20

| id  |
20
|

| name| Untitled (68:58)
  |

| status  |
ready
|

| os_platform |
ubuntu
|

| roles   |
[u'compute']
|

| kernel_params   |
None
|

| pending_roles   |
[]
  |

| hostname|
node-20
|

| fqdn| node-20.dialogic.com
|

| platform_name   | ProLiant BL460c
Gen9
|











where can i find the cloud init settings which are deploy to new nodes?

i guess this has something to do with this file:
/usr/share/fuel-agent/cloud-init-templates/cloud_config_ubuntu.jinja2

in that file I see

hostname: {{ common.hostname }}

fqdn: {{ common.fqdn }}



please help me with an info you might have or let me know that populates
those 2 parts of the template?



Is there a database these values are all stored in on the fuel server?



Thanks



--Jim

-- Jim

On Tue, Sep 26, 2017 at 12:00 PM, Jim Okken  wrote:

> also I should add, I dont have the original hard drives in the system so
> it isn't because it is booting the old OS where these node names were set.
> this is definitely the newly installed OS being given the wroing hostname
>
>
>
> is there a database this is all kept in? maybe I could look around and
> find where these old node names are being saved?
>
> thanks!
>
> -- Jim
>
> On Mon, Sep 25, 2017 at 6:03 PM, Jim Okken  wrote:
>
>> hi all,
>>
>> I am using Fuel 10.
>>
>> i have 2 nodes I am trying to deploy as compute nodes. at one time in the
>> past I was attempting to deploy them too. I assume back then their node
>> names were node-11 and node-20.
>>
>> they were never successfully deploy and now I've worked out their
>> hardware issues and are attempting to deploy them again. now Fuel has given
>> them the names node-80 and node-81.
>> (i may be at 80 in my node names but I only have 17 nodes so far)
>>
>> the deploy of these 2 nodes does not get past installing Ubuntu. The
>> nodes reboot after Ubuntu is installed and come up incorrectly as node-11
>> and node-20. After that Fuel sits for a long while and then gives an error
>> (pasted at the end of email). I assume the nodes come up with the wrong
>> name/ip/ssh-key and Fuel can't contact them.
>>
>> I'm a novice at using the fuel and fuel2 cli's but I've tried deleting
>> these nodes and removing from database. Then re-PXE boot the nodes and
>> start a fresh deploy just to have them named node11 and 20 again. Fuel cli
>> does show the correct host name for these nodes, but I've tried anyway to
>> (re)set the host name for these node with no affect.
>>
>> If I try to delete node-11 and node-20 I get this error
>> 404 Client Error: Not Found for url: http://10.20.243.1:8000/api/v1
>> /nodes/?ids=11 (NodeCollection not found)
>>
>> what can I do to get past this please?
>>
>>
>>
>> Errors from the Fuel Astute log:
>> 2017-09-25 21:06:28 ERROR [1565] Error running provisioning:
>> # ,
>> trace: ["/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:178:in
>> `rescue in initialize_mclient'", "/usr/share/gems/gems/astute-1
>> 0.0.0/lib/astute/mclient.rb:161:in `initialize_mclient'",
>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:51:in
>> `initialize'", "/usr/share/gems/gems/astute-1
>> 0.0.0/lib/astute/nailgun_hooks.rb:421:in `new'",
>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
>> `run_shell_without_check'", "/usr/share/gems/gems/astute-1
>> 0.0.0/lib/astute/nailgun_hooks.rb:449:in `update_node_status'",
>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:313:in
>> `reboot_hook'", "/usr/share/gems/gems/astute-1
>> 0.0.0/lib/astute/nailgun_hooks.rb:38:in `block in process'",
>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
>> `each'", 
>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
>> `process'", 
>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/image_provision.rb:117:in
>> `reboot'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:273:in
>> `soft_reboot'", "/usr/share/gems/gems/astute-1
>> 

Re: [Openstack] [Fuel] node name issue

[Jim Okken]

also I should add, I dont have the original hard drives in the system so it
isn't because it is booting the old OS where these node names were set.
this is definitely the newly installed OS being given the wroing hostname



is there a database this is all kept in? maybe I could look around and find
where these old node names are being saved?

thanks!

-- Jim

On Mon, Sep 25, 2017 at 6:03 PM, Jim Okken  wrote:

> hi all,
>
> I am using Fuel 10.
>
> i have 2 nodes I am trying to deploy as compute nodes. at one time in the
> past I was attempting to deploy them too. I assume back then their node
> names were node-11 and node-20.
>
> they were never successfully deploy and now I've worked out their hardware
> issues and are attempting to deploy them again. now Fuel has given them the
> names node-80 and node-81.
> (i may be at 80 in my node names but I only have 17 nodes so far)
>
> the deploy of these 2 nodes does not get past installing Ubuntu. The nodes
> reboot after Ubuntu is installed and come up incorrectly as node-11 and
> node-20. After that Fuel sits for a long while and then gives an error
> (pasted at the end of email). I assume the nodes come up with the wrong
> name/ip/ssh-key and Fuel can't contact them.
>
> I'm a novice at using the fuel and fuel2 cli's but I've tried deleting
> these nodes and removing from database. Then re-PXE boot the nodes and
> start a fresh deploy just to have them named node11 and 20 again. Fuel cli
> does show the correct host name for these nodes, but I've tried anyway to
> (re)set the host name for these node with no affect.
>
> If I try to delete node-11 and node-20 I get this error
> 404 Client Error: Not Found for url: http://10.20.243.1:8000/api/
> v1/nodes/?ids=11 (NodeCollection not found)
>
> what can I do to get past this please?
>
>
>
> Errors from the Fuel Astute log:
> 2017-09-25 21:06:28 ERROR [1565] Error running provisioning:
> # ,
> trace: ["/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:178:in
> `rescue in initialize_mclient'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/mclient.rb:161:in `initialize_mclient'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:51:in
> `initialize'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
> `new'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
> `run_shell_without_check'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/nailgun_hooks.rb:449:in `update_node_status'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:313:in
> `reboot_hook'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:38:in
> `block in process'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/nailgun_hooks.rb:26:in `each'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
> `process'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/image_provision.rb:117:in
> `reboot'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:273:in
> `soft_reboot'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:240:in
> `provision_piece'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/provision.rb:126:in `block (3 levels) in
> provision_and_watch_progress'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/provision.rb:309:in `call'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:309:in
> `sleep_not_greater_than'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/provision.rb:120:in `block (2 levels) in
> provision_and_watch_progress'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/provision.rb:119:in `loop'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:119:in `block
> in provision_and_watch_progress'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/provision.rb:118:in `catch'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:118:in
> `provision_and_watch_progress'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/provision.rb:52:in `provision'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/orchestrator.rb:109:in
> `provision'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/dispatcher.rb:46:in
> `provision'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/dispatcher.rb:37:in
> `image_provision'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:172:in
> `dispatch_message'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/server/server.rb:131:in `block in dispatch'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:64:in
> `call'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:64:in
> `block in each'", "/usr/share/gems/gems/astute-
> 10.0.0/lib/astute/server/task_queue.rb:56:in `each'",
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:56:in
> `each'", 
> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:128:in
> `each_with_index'", 
> 

[Openstack] [Fuel] node name issue

[Jim Okken]

hi all,

I am using Fuel 10.

i have 2 nodes I am trying to deploy as compute nodes. at one time in the
past I was attempting to deploy them too. I assume back then their node
names were node-11 and node-20.

they were never successfully deploy and now I've worked out their hardware
issues and are attempting to deploy them again. now Fuel has given them the
names node-80 and node-81.
(i may be at 80 in my node names but I only have 17 nodes so far)

the deploy of these 2 nodes does not get past installing Ubuntu. The nodes
reboot after Ubuntu is installed and come up incorrectly as node-11 and
node-20. After that Fuel sits for a long while and then gives an error
(pasted at the end of email). I assume the nodes come up with the wrong
name/ip/ssh-key and Fuel can't contact them.

I'm a novice at using the fuel and fuel2 cli's but I've tried deleting
these nodes and removing from database. Then re-PXE boot the nodes and
start a fresh deploy just to have them named node11 and 20 again. Fuel cli
does show the correct host name for these nodes, but I've tried anyway to
(re)set the host name for these node with no affect.

If I try to delete node-11 and node-20 I get this error
404 Client Error: Not Found for url:
http://10.20.243.1:8000/api/v1/nodes/?ids=11 (NodeCollection not found)

what can I do to get past this please?



Errors from the Fuel Astute log:
2017-09-25 21:06:28 ERROR [1565] Error running provisioning:
# ,
trace: ["/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:178:in
`rescue in initialize_mclient'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:161:in
`initialize_mclient'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:51:in
`initialize'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
`new'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
`run_shell_without_check'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:449:in
`update_node_status'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:313:in
`reboot_hook'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:38:in
`block in process'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
`each'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
`process'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/image_provision.rb:117:in
`reboot'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:273:in
`soft_reboot'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:240:in
`provision_piece'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:126:in `block
(3 levels) in provision_and_watch_progress'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:309:in `call'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:309:in
`sleep_not_greater_than'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:120:in `block
(2 levels) in provision_and_watch_progress'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:119:in `loop'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:119:in `block
in provision_and_watch_progress'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:118:in
`catch'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:118:in
`provision_and_watch_progress'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:52:in
`provision'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/orchestrator.rb:109:in
`provision'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/dispatcher.rb:46:in
`provision'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/dispatcher.rb:37:in
`image_provision'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:172:in
`dispatch_message'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:131:in
`block in dispatch'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:64:in
`call'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:64:in
`block in each'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:56:in
`each'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:56:in
`each'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:128:in
`each_with_index'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:128:in
`dispatch'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:106:in
`block in perform_main_job'"]
2017-09-25 21:06:26 ERROR [1565] Error occured while provisioning:
# >
2017-09-25 21:06:26 ERROR [1565] No more retries for MCollective client
instantiation after exception:
["/usr/share/gems/gems/mcollective-client-2.8.4/lib/mcollective/rpc/client.rb:507:in
`discover'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:167:in
`initialize_mclient'",
"/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:51:in
`initialize'",