I've added the ubuntu-fan package, as fanctl should accommodate the
underlay MTU.
There is a work-around for the Juju case, which I will look to
implement.
** Also affects: ubuntu-fan (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to ubuntu-fan in Ubuntu.
https://bugs.launchpad.net/bugs/1936842
Title:
agent cannot be up on LXD/Fan network on OpenStack OVN/geneve mtu=1442
Status in juju:
Triaged
Status in ubuntu-fan package in Ubuntu:
New
Bug description:
When one runs OpenStack with MTU=1500 underlying network, Neutron/OVN
by default will create a tenant/overlay network with
MTU=1442(1500-58). However, when deploying a workload on top, the Fan
network with Juju will use MTU=1450 which is actually bigger than
that. Then, Juju agent cannot be up inside LXD/Fan on top of
OpenStack.
OpenStack deployment is based on https://jaas.ai/openstack-base
and k8s as a workload on top of OpenStack is: https://jaas.ai/kubernetes-core
$ openstack network list
+--------------------------------------+----------+--------------------------------------+
| ID | Name | Subnets
|
+--------------------------------------+----------+--------------------------------------+
| ba9880e3-5f07-4b49-aeda-20dfa8fe66ec | internal |
45efcc4c-fb99-4fe8-98a4-77195ec4aef1 |
| e27b1cef-c53b-42e0-b307-8ba2e622c2dd | ext_net |
5315d907-ce90-4605-ad72-b78229965b40 |
+--------------------------------------+----------+--------------------------------------+
$ openstack network show internal
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
...
| mtu | 1442 |
| name | internal |
...
| provider:network_type | geneve |
| provider:physical_network | None |
| provider:segmentation_id | 1410 |
...
+---------------------------+--------------------------------------+
$ juju machines -m k8s-on-openstack
Machine State DNS Inst id
Series AZ Message
0 started 192.168.151.75 0e96a5b1-3665-44f6-bcb7-4851ab6cd22d focal
nova ACTIVE
0/lxd/0 pending juju-b9bd8b-0-lxd-0 focal
nova Container started
1 started 192.168.151.66 6bfa5d2e-24e6-42b8-b5ec-1f2a0d0e6b02 focal
nova ACTIVE
[openstack instance/VM provisioned by Juju - ens3: mtu=1442, fan-252:
mtu=1450]
juju-b9bd8b-k8s-on-openstack-0:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc fq_codel state UP
mode DEFAULT group default qlen 1000
link/ether fa:16:3e:48:85:85 brd ff:ff:ff:ff:ff:ff
3: fan-252: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP
mode DEFAULT group default qlen 1000
link/ether 06:28:c7:b4:50:eb brd ff:ff:ff:ff:ff:ff
4: ftun0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1392 qdisc noqueue master
fan-252 state UNKNOWN mode DEFAULT group default qlen 1000
link/ether 92:d7:07:6e:b4:db brd ff:ff:ff:ff:ff:ff
5: lxdbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state
DOWN mode DEFAULT group default qlen 1000
link/ether 00:16:3e:03:0e:96 brd ff:ff:ff:ff:ff:ff
7: 0lxd0-0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
master fan-252 state UP mode DEFAULT group default qlen 1000
link/ether 06:28:c7:b4:50:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0
[lxc config of LXD container as juju machine 0/lxd/0]
juju-b9bd8b-k8s-on-openstack-0:~# lxc config show juju-b9bd8b-0-lxd-0
...
user.user-data: |
#cloud-config
apt_mirror: ""
bootcmd:
- install -D -m 644 /dev/null '/etc/netplan/99-juju.yaml'
- |-
printf '%s\n' 'network:
version: 2
ethernets:
eth0:
match:
macaddress: 00:16:3e:82:f9:44
dhcp4: true
nameservers:
search: [openstack.internal]
addresses: [8.8.8.8, 8.8.4.4]
mtu: 1450
...
devices:
eth0:
host_name: 0lxd0-0
hwaddr: 00:16:3e:82:f9:44
mtu: "1450"
name: eth0
nictype: bridged
parent: fan-252
type: nic
[cloud-init-output.log - stuck at the initial apt update and also failing to
fetch the agent binary]
Cloud-init v. 21.2-3-g899bfaa9-0ubuntu2~20.04.1 running 'modules:config' at
Mon, 19 Jul 2021 03:27:32 +0000. Up 53.99 seconds.
Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease
Err:2 http://security.ubuntu.com/ubuntu focal-security InRelease
Connection failed [IP: 192.168.151.1 8000]
Err:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Connection failed [IP: 192.168.151.1 8000]
Err:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Connection failed [IP: 192.168.151.1 8000]
Ign:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages
Ign:6 http://archive.ubuntu.com/ubuntu focal/universe Translation-en
Ign:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 c-n-f Metadata
Ign:8 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages
Ign:9 http://archive.ubuntu.com/ubuntu focal/multiverse Translation-en
Ign:10 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 c-n-f Metadata
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB]
+ printf Attempt 5 to download agent binaries from %s...\n
https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64
Attempt 5 to download agent binaries from
https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64...
+ curl -sSfw agent binaries from %{url_effective} downloaded: HTTP
%{http_code}; time %{time_total}s; size %{size_download} bytes; speed
%{speed_download} bytes/s --connect-timeout 20 --noproxy * --insecure -o
/var/lib/juju/tools/2.9.8-ubuntu-amd64/tools.tar.gz
https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64
curl: (28) Operation timed out after 20000 milliseconds with 0 out of 0 bytes
received
agent binaries from
https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64
downloaded: HTTP 000; time 20.000946s; size 0 bytes; speed 0.000 bytes/s +
echo Download failed, retrying in 15s
Download failed, retrying in 15s
+ sleep 15
[manually run tracepath to get pmtu]
juju-b9bd8b-0-lxd-0:~# tracepath -n -m 5 192.168.151.1
1?: [LOCALHOST] pmtu 1450
1: 252.16.0.1 0.346ms
1: 252.16.0.1 0.051ms
2: 252.16.0.1 0.053ms pmtu 1442
2: no reply
3: 192.168.151.1 3.283ms reached
Resume: pmtu 1442 hops 3 back 3
To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1936842/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
