Re: [Openstack-operators] [kolla][nova] Upgrade leaves nova_compute at earlier version]

2018-06-15 Thread Dave Williams
Sure:
Apologies for a longer post than I would like.

globals.yml with all commented out (default) values removed:

kolla_base_distro: "ubuntu"
kolla_install_type: "source"
openstack_release: "master"
kolla_internal_vip_address: "10.10.66.138"
kolla_internal_fqdn: "openstackint.uklab.local"
kolla_external_vip_address: "10.10.66.134"
kolla_external_fqdn: "openstackext.uklab.local"
kolla_enable_tls_external: "yes"
kolla_external_fqdn_cert: "{{ node_config_directory }}/certificates/haproxy.pem"
enable_central_logging: "yes"
enable_ceph: "yes"
enable_cinder: "yes"
enable_cinder_backend_netapp_iscsi: "no"
enable_neutron_provider_networks: "yes"
ceph_pool_type: "replicated"
ceph_pool_pg_num: 128
ceph_pool_pgp_num: 128
cinder_backup_driver: "ceph"
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:

inventory file again only section with changes (rest default):
[control]
mclaren   ansible_connection=local network_interface=enp5s0 
neutron_external_interface=enp3s0 api_interface=enp5s0
[network]
mclaren   ansible_connection=local network_interface=enp5s0 
neutron_external_interface=enp3s0 api_interface=enp5s0
[inner-compute]
[external-compute]
mclaren   ansible_connection=local network_interface=enp5s0 
neutron_external_interface=enp3s0 api_interface=enp5s0
aston   network_interface=enp3s0 neutron_external_interface=enp5s0
corvette2   network_interface=enp3s0 neutron_external_interface=enp5s0
lambo2  network_interface=enp3s0 neutron_external_interface=enp5s0
porsche network_interface=enp3s0 neutron_external_interface=enp5s0
ferrari network_interface=enp3s0 neutron_external_interface=enp5s0
lotus   network_interface=enp3s0 neutron_external_interface=enp5s0
maseratinetwork_interface=eth0 neutron_external_interface=eth1
chevi2  network_interface=enp4s0f1 neutron_external_interface=enp4s0f0
audinetwork_interface=em1 neutron_external_interface=em2
saabnetwork_interface=em1 neutron_external_interface=em2
radical network_interface=eth0 neutron_external_interface=eth1
[compute:children]
inner-compute
external-compute
[storage]
mclaren   ansible_connection=local network_interface=enp5s0 
neutron_external_interface=enp3s0 api_interface=enp5s0
lambo2  network_interface=enp3s0 neutron_external_interface=enp5s0
porsche network_interface=enp3s0 neutron_external_interface=enp5s0
ferrari network_interface=enp3s0 neutron_external_interface=enp5s0
lotus   network_interface=enp3s0 neutron_external_interface=enp5s0
maseratinetwork_interface=eth0 neutron_external_interface=eth1
chevi2  network_interface=enp4s0f1 neutron_external_interface=enp4s0f0
audinetwork_interface=em1 neutron_external_interface=em2
saabnetwork_interface=em1 neutron_external_interface=em2
radical network_interface=eth0 neutron_external_interface=eth1
[monitoring]
mclaren   ansible_connection=local network_interface=enp5s0 
neutron_external_interface=enp3s0 api_interface=enp5s0
radical network_interface=eth0 neutron_external_interface=eth1
aston   network_interface=enp3s0 neutron_external_interface=enp5s0
[deployment]
mclaren   ansible_connection=local network_interface=enp5s0 
neutron_external_interface=enp3s0 api_interface=enp5s0

docker ps -a (sorry for lack of line wrap)
SERVER aston
CONTAINER IDIMAGE   COMMAND CREATED 
STATUS  PORTS   NAMES
24e87677d14c0a1d3587ae5c"kolla_start"   9 days ago  
Up 9 days   neutron_openvswitch_agent
22e7e820eb96760e48f2527b"kolla_start"   9 days ago  
Up 9 days   openvswitch_vswitchd
e6cb63995d02c07ab7bd1910"kolla_start"   9 days ago  
Up 9 days   openvswitch_db
71ffd7f32eaac9af2493c9e1"kolla_start"   9 days ago  
Up 9 days   nova_libvirt
8870b6ebaf61e9d2082aa62f"kolla_start"   9 days ago  
Up 9 days   nova_ssh
042510be738972e8b021c493"kolla_start"   9 days ago  
Up 9 days   cron
83e50da14764ac0c8b1aeef2"kolla_start"   9 days ago  
Up 9 days   kolla_toolbox
5e8efb880a44afa0d8636841"kolla_start"   9 days ago  
Up 9 days   fluentd
76a7b24d339f7dc2d4695962"kolla_start"   6 weeks ago 
Up 9 days   nova_compute
SERVER chevi2
CONTAINER IDIMAGE  COMMAND  
   CREATED STATUS   PORTS   NAMES
98c58e80ed71

Re: [Openstack-operators] [kolla][nova] Upgrade leaves nova_compute at earlier version

2018-06-14 Thread Eduardo Gonzalez
Hi,

could you share a your globals file without secrets, a docker ps -a on all
compute hosts and images too. If you are able to get n upgrade log will be
helpful too.

By the way, using master is not really recommended, many changes from other
projects and kolla may break the deployment.

Regards

On Thu, Jun 14, 2018, 9:00 PM Dave Williams 
wrote:

> I am using kolla-ansible 6.0.0 with openstack_release set to master in
> globals.yml in a production environment.
>
> I am trying to fix an oslo_messaging.rpc.client.RemoteError when
> undertaking server add volume
> (https://bugs.launchpad.net/nova/+bug/1773393) and appear to have
> tracked it down to an inconsistency of container versions.
>
> kollo-ansible upgrade runs to completion without error but leaves the
> running nova_compute containers (and possibly others) at an earlier
> version.
>
> The version running is
> CONTAINER ID IMAGECOMMAND   CREATED STATUS PORTS NAMES
> 944f620a445c 7dc2d4695962 "kolla_start" 4 weeks ago Up 4 hours nova_compute
>
> whereas docker images -a shows
> REPOSITORY TAG IMAGE ID CREATED SIZE
> kolla/ubuntu-source-nova-compute master f2df8187f14e 15 hours ago 1.29GB
> kolla/ubuntu-source-nova-compute  582561ac010f 39 hours ago 1.29GB
> kolla/ubuntu-source-nova-compute  7dc2d4695962 3 months ago 1.22GB
>
> This implies f2df8187f14e is the one I should be using.
> The image 582561ac010f was after I tried to switch to queens from master
> but without success due to a bootstrap_cinder problem:
> Error during database migration:
>   "Database schema file with version 122 doesn't exist."
> I tried to investigate this but without any obvious resolution.
>
> All compute nodes show the same issue.
>
> As per the notes on
> https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html
> I have checked and virt_type is set to kvm in nova.conf and so I cannot
> see why the upgrade shouldnt have been successful.
>
> How do I get kolla to use the latest version pulled?
>
> Given I have running instances I am a little nervous of doing a deploy or
> reconfigure.
>
> Thanks for help.
>
> Dave
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


[Openstack-operators] [kolla][nova] Upgrade leaves nova_compute at earlier version

2018-06-14 Thread Dave Williams
I am using kolla-ansible 6.0.0 with openstack_release set to master in
globals.yml in a production environment.

I am trying to fix an oslo_messaging.rpc.client.RemoteError when
undertaking server add volume
(https://bugs.launchpad.net/nova/+bug/1773393) and appear to have
tracked it down to an inconsistency of container versions.

kollo-ansible upgrade runs to completion without error but leaves the
running nova_compute containers (and possibly others) at an earlier
version.

The version running is
CONTAINER ID IMAGECOMMAND   CREATED STATUS PORTS NAMES
944f620a445c 7dc2d4695962 "kolla_start" 4 weeks ago Up 4 hours nova_compute

whereas docker images -a shows
REPOSITORY TAG IMAGE ID CREATED SIZE
kolla/ubuntu-source-nova-compute master f2df8187f14e 15 hours ago 1.29GB
kolla/ubuntu-source-nova-compute  582561ac010f 39 hours ago 1.29GB
kolla/ubuntu-source-nova-compute  7dc2d4695962 3 months ago 1.22GB

This implies f2df8187f14e is the one I should be using.
The image 582561ac010f was after I tried to switch to queens from master
but without success due to a bootstrap_cinder problem:
Error during database migration: 
  "Database schema file with version 122 doesn't exist."
I tried to investigate this but without any obvious resolution.

All compute nodes show the same issue.

As per the notes on
https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html
I have checked and virt_type is set to kvm in nova.conf and so I cannot
see why the upgrade shouldnt have been successful.

How do I get kolla to use the latest version pulled?

Given I have running instances I am a little nervous of doing a deploy or
reconfigure.

Thanks for help.

Dave


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators