Re: [lxc-users] Nova-lxd plugin installation
Hi Martin
Okay, some progress which is great!
But, to help you further, a few more details:
1. Which Linux OS are you using?
2. Is the LXD snap installed? - if so, which version is it? "lxc
--version" will give the answer.
3. Have you configured a storage pool for LXD? (e.g. have you done
"sudo lxd init" and created a default storage pool). What kind of storage
pool is it? (e.g. dir, btrfs, zfs, etc.)
4. It looks like your using master version of nova-lxd (i.e. via
devstack and the plugin)? I think it's also going to use ZFS? If so,
there's a bug in nova-lxd (that I will fix in the not too distant future)
where unless the ZFS pool is the same string as the LXD pool for it, then
nova-lxd can't find the storage pool.
So, if you run the following commands:
lxc storage list
zpool list
That should show the current configuration.
Hope that this helps.
Cheers
Alex.
On Wed, Sep 19, 2018 at 2:48 PM, Martin Bobák wrote:
> Hi Alex,
>
> first off, thank you for your kind reply. I followed your advices.
> However, I still have a problem which looks like an incorrect setting up of
> a storage pool for lxd. I configured the /etc/nova-compute.conf as you
> suggested, but it didn't help.
>
> In my case those lines look like:
>
> [DEFAULT]
> compute_driver = nova_lxd.nova.virt.lxd.LXDDriver
>
> [lxd]
> allow_live_migration = True
> pool = lxd
> But the error is following (relevant parts from syslog --> it looks like
> lxd variables from the /etc/nova-compute.conf aren't delivered to the
> driver, or recognized by it, I tried to use different names for
> compute_driver e.g. lxd.LXDDriver, nova-lxd.nova.virt.lxd.(driver).LXDDriver
> -> however, it didn't help):
>
> Sep 19 09:15:08 localhost nova-conductor[1956]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone req-5cda2246-8087-4f49-b9b3-463d29fd7bf8
> #033[00;36mNone None#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=1956)
> log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:
> 2890}}#033[00m
> Sep 19 09:15:08 localhost nova-consoleauth[1984]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone req-2a9399e1-996d-42f4-899b-62db8d8e5afd
> #033[00;36mNone None#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=1984)
> log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:
> 2890}}#033[00m
> Sep 19 09:15:10 localhost nova-scheduler[2000]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone req-3d23848b-270b-4718-91dd-3b0417d27d21
> #033[00;36mNone None#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2000)
> log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:
> 2890}}#033[00m
> Sep 19 09:15:11 localhost devstack@placement-api.service[2030]:
> #033[00;32mDEBUG nova.api.openstack.placement.wsgi
> [#033[00;36m-#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2294)
> log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:
> 2890}}#033[00m
> Sep 19 09:15:12 localhost devstack@n-api.service[1999]: #033[00;32mDEBUG
> nova.api.openstack.wsgi_app [#033[01;36mNone
> req-1e20e359-2bcc-44e6-9c75-7422c5837c03
> #033[00;36mNone None#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2181)
> log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:
> 2890}}#033[00m
> Sep 19 09:15:14 localhost devstack@placement-api.service[2030]:
> #033[00;32mDEBUG nova.api.openstack.placement.wsgi
> [#033[00;36m-#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2292)
> log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:
> 2890}}#033[00m
> Sep 19 09:15:15 localhost devstack@n-api.service[1999]: #033[00;32mDEBUG
> nova.api.openstack.wsgi_app [#033[01;36mNone
> req-6d5745e7-3784-4b8e-8210-e7bf74fd9655
> #033[00;36mNone None#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2182)
> log_opt_values /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:
> 2890}}#033[00m
> Sep 19 09:15:20 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone req-1bc67100-ffbe-4d22-be1f-42133eb60611
> #033[00;36mNone None#033[00;32m] #033[01;35m#033[00;32mcompute_driver
> = lxd.LXDDriver#033[00m #033[00;33m{{(pid=1968) log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> ep 19 09:15:20 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone req-1bc67100-ffbe-4d22-be1f-42133eb60611
> #033[00;36mNone None#033[00;32m]
> #033[01;35m#033[00;32mlxd.allow_live_migration
> = False#033[00m #033[00;33m{{(pid=1968)
Re: [lxc-users] lxc container network occasional problem with bridge network on bonding device
Hi, Andrey. Thanks for reply.
It took some time to reproduce the problem. But now I found what to do.
> How do you connect containers to the bridge?
Here’s lxc info shows.
# lxc profile show default
config: {}
description: Default LXD profile
devices:
eth0:
name: eth0
nictype: bridged
parent: br0
type: nic
root:
path: /
pool: default
type: disk
name: default
> Can containers talk to each other when this happens?
Yes.
> Can host talk to the world at that same time?
Yes.
I do not attach log of results of ping commands since they are just trivial.
> And netplan did not yell at you?
I identified the time when this problem happened and inspected /var/log/syslog
at that time.
There was nothing.
I found a way to reproduce the problem quickly. The procedure is
(1) connect both of the LAN cables
(2) stop all containers (I am not sure whether “all” is necessary.)
(3) start some of the containers
(4) the problem occurs on the started container just after or serveral
minutes after
the restart
Regards,
> 2018/09/18 2:47、Andrey Repin のメール:
>
> Greetings, toshinao!
>
>> Hi.
>
>> I experienced occasional network problem of containers running on ubuntu
>> server 18.04.1. Containers
>> can communicate with host IP always and they can communicate sometimes to
>> the other hosts but they
>> are disconnected occasionally. When the problem occurs, the ping from the
>> container to external hosts
>> does not reach at all, but very rarely they recover after, for example,
>> several hours later.
>> Disconnection happens much more easily.
>
>> The host network is organized by using netplan in the following topology.
>
>> +-eno1-< <--lan_cable--> >-+
>> br0--bond0-+ +-- Cisco 3650
>> +-en02-< <--lan_cable--> >-+
>
>> The bonding mode is balance-a1b.
>
> ALB
> Adaptive Load Balancing
>
>> I also found that if one of the LAN cables is physically disconnected,
>> this problem has never happened.
>
> How do you connect containers to the bridge?
>
>> By using iptraf-ng, I watched the bridge device, the following br0, as well
>> as the slave devices.
>> Even if containers send a ping to the external hosts, no ping packet is
>> detected, when they cannot
>> communicate. Ping packets are detected by iptraf-ng on these devices when
>> the communication is working.
>
>> I guess this can be a low-level problem of virtual networking. Are there any
>> suggestions to solve
>> the problem ?
>
> Can containers talk to each other when this happens?
> Can host talk to the world at that same time?
>
>> Here's the detail of the setting.
>
>> host's netplan setting
>
>> network:
>> version: 2
>> renderer: networkd
>> ethernets:
>> eno1:
>>dhcp4: no
>> eno2:
>>dhcp4: no
>> bonds:
>> bond0:
>>interfaces: [eno1, eno2]
>>parameters:
>> mode: balanec-a1b
>
> And netplan did not yell at you?
>
>> bridges:
>> br0:
>>interfaces:
>> - bond0
>>addresses: [10.1.2.3/24]
>>gateway4: 10.1.2.254
>>dhcp4: no
>
>> host network interface status
>
>> host# ip a s
>> 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group
>> default qlen 1000
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> inet 127.0.0.1/8 scope host lo
>> valid_lft forever preferred_lft forever
>> inet6 ::1/128 scope host
>> valid_lft forever preferred_lft forever
>> 2: eno1: mtu 1500 qdisc mq master
>> bond0 state UP group default qlen 1000
>> link/ether 0b:25:b5:f2:e1:34 brd ff:ff:ff:ff:ff:ff
>> 3: eno2: mtu 1500 qdisc mq master
>> bond0 state UP group default qlen 1000
>> link/ether 0b:25:b5:f2:e1:35 brd ff:ff:ff:ff:ff:ff
>> 4: br0: mtu 1500 qdisc noqueue state UP
>> group default qlen 1000
>> link/ether 0a:1a:6c:85:ff:ed brd ff:ff:ff:ff:ff:ff
>> inet 10.1.2.3/24 brd 10.1.2.255 scope global br0
>> valid_lft forever preferred_lft forever
>> inet6 fe80::81a:6cff:fe85:ffed/64 scope link
>> valid_lft forever preferred_lft forever
>> 5: bond0: mtu 1500 qdisc noqueue
>> master br0 state UP group default qlen 1000
>> link/ether 0a:54:4b:f2:d7:10 brd ff:ff:ff:ff:ff:ff
>> 7: vethK4HOFU@if6: mtu 1500 qdisc noqueue
>> master br0 state UP group default qlen 1000
>> link/ether fe:ca:07:3e:2b:2d brd ff:ff:ff:ff:ff:ff link-netnsid 0
>> inet6 fe80::fcca:7ff:fe3e:2b2d/64 scope link
>> valid_lft forever preferred_lft forever
>> 9: veth77HJ0V@if8: mtu 1500 qdisc noqueue
>> master br0 state UP group default qlen 1000
>> link/ether fe:85:f0:ef:78:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 1
>> inet6 fe80::fc85:f0ff:feef:78b2/64 scope link
>> valid_lft forever preferred_lft forever
>
>> container's network interface status
>
>> root@bionic0:~# ip a s
>> 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group
>> default qlen 1000
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> inet 127.0.0.1/8 scope host lo
>> valid_lft forever preferred_lft forever
>> inet6
