[Bug 1904580] Re: Permissions 0644 for '/var/lib/nova/.ssh/id_rsa' are too open
Any update here ? This is blocking critical node evacuation. Thanks ! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1904580 Title: Permissions 0644 for '/var/lib/nova/.ssh/id_rsa' are too open To manage notifications about this bug go to: https://bugs.launchpad.net/charm-nova-compute/+bug/1904580/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1952264] Re: ntpd interface listing is bugged
** Summary changed: - ntp sync checks fail when server as no IPv6 connectivity + ntpd interface listing is bugged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1952264 Title: ntpd interface listing is bugged To manage notifications about this bug go to: https://bugs.launchpad.net/ntp-charm/+bug/1952264/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1952264] Re: ntp sync checks fail when server as no IPv6 connectivity
I investigated this quite a bit, and this appears to be an ntp bug and
not a charm bug.
This host is a trusty host, running ntp version
1:4.2.6.p5+dfsg-3ubuntu2.14.04.13. We have other hosts running the same
version that don't have the problem described above.
I spent quite some time investigating this, comparing the hosts, running
strace etc, and I noticed a subtle difference in /etc/hosts : on the
working host, the ::1 entry doesn't have "localhost", but it does on the
failing host. When I removed "localhost" from the ::1 entry on the
failing host, "ntpq -pn" started working.
Investigating things a bit more, I found out that on the working host,
ntpd was listening on ::1 but on the failing host, it wasn't (by
checking "ss -anupe" output as well as ntpd starting logs).
Comparing straces of starting ntpd, I think I was able to find what's
going on. On the working host it gives (only relevant output is posted
here) :
3973 19:41:32 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 5
[...]
3973 19:41:32 ioctl(5, SIOCGIFFLAGS, {ifr_name="qvobb268af4-e9",
ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_PROMISC|IFF_MULTICAST}) = 0
3973 19:41:32 ioctl(5, SIOCGIFFLAGS, {ifr_name="qbrd5588b49-e3",
ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_MULTICAST}) = 0
3973 19:41:32 ioctl(5, SIOCGIFFLAGS, {ifr_name="qvb1693c156-5f",
ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_PROMISC|IFF_MULTICAST}) = 0
[... the same for a bunch of interfaces - this is a nova compute node so this
is expected ...]
3973 19:41:32 close(5) = 0
But on the failing host, it checks a single interface :
56717 19:37:03 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 5
[...]
56717 19:37:03 ioctl(5, SIOCGIFFLAGS, {ifr_name="qvbba244f00-69",
ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_PROMISC|IFF_MULTICAST}) = 0
56717 19:37:03 close(5) = 0
So I thought this interface was a bit special :
$ ip li sh dev qvbba244f00-69
67772: qvbba244f00-69@qvoba244f00-69:
mtu 1500 qdisc noqueue master qbrba244f00-69 state UP mode DEFAULT group
default qlen 1000
link/ether 0e:ac:86:b1:c8:24 brd ff:ff:ff:ff:ff:ff
It appears completely normal, except that it has an unusually high
ifindex (67772). Could that be the cause of the problem ? Looking at the
source code at
https://git.launchpad.net/ubuntu/+source/ntp/tree/?h=ubuntu/trusty-
updates : interfaces are parsed looking at the /proc/net/if_inet6 file
(https://git.launchpad.net/ubuntu/+source/ntp/tree/lib/isc/unix/ifiter_getifaddrs.c?h=ubuntu/trusty-
updates#n54) which strace confirms :
3973 19:41:32 open("/proc/net/if_inet6", O_RDONLY) = 6
Each line is parsed using fgets :
fgets(iter->entry, sizeof(iter->entry), iter->proc) != NULL)
https://git.launchpad.net/ubuntu/+source/ntp/tree/lib/isc/unix/interfaceiter.c?h=ubuntu/trusty-
updates#n181
What's sizeof(iter->entry) ? Well "entry" is defined like that :
charentry[ISC_IF_INET6_SZ];
https://git.launchpad.net/ubuntu/+source/ntp/tree/lib/isc/unix/ifiter_getifaddrs.c?h=ubuntu/trusty-
updates#n48
And ISC_IF_INET6_SZ is :
#define ISC_IF_INET6_SZ \
sizeof("0001 01 80 10 80 XXlo\n")
https://git.launchpad.net/ubuntu/+source/ntp/tree/lib/isc/unix/interfaceiter.c?h=ubuntu/trusty-
updates#n153
And this is where the problem is. The computation of ISC_IF_INET6_SZ
assumes that ifindex will be 2 chars (in hex), so that ifindex will be <
256. However, ifindexes higher than that are likely common, so why don't
we see this bug elsewhere ? Well because the computation of
ISC_IF_INET6_SZ also assumes that the interface name is 16 chars.
In our example, the interface name is "only" 14 chars, so we have a buffer of 2
chars for the ifindex. But that's not enough, it's off by 1 in fact !
"0001 01 80 10 80 XXlo\n" is 62 chars
long.
The first line of if_inet6 on our machine is :
fe800cac86fffeb1c824 108bc 40 20 80 qvbba244f00-69, and that's 62
chars long... but without the \n !
So what might be happening here is that the first iteration of the loop
will properly read the whole line except the \n, and the next iteration
will resume at that location, and because fgets() stops at EOF or
newline, it will just return a newline, which will make the whole
iteration stop.
The fix here is pretty simple : the computation of ISC_IF_INET6_SZ
should assume an ifindex of UINT_MAX, ie (or any 8-chars
number). If I can trust
https://git.launchpad.net/ubuntu/+source/ntp/tree/lib/isc/unix/interfaceiter.c?h=applied/ubuntu/jammy
this is still present in Jammy.
Redirecting the bug to the "ntp" package.
** Also affects: ntp (Ubuntu)
Importance: Undecided
Status: New
** Changed in: ntp-charm
Status: New => Invalid
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1952264
Title:
ntp sync checks fail when server as
[Bug 1948995] Re: Allow reverting to older revisions of a snap
OK thanks. I must say that this is not a great user experience. How would we "increase the number of revisions that are kept around on a system" ? It seems that the default of 2 is pretty low for servers. ** Changed in: snapd (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1948995 Title: Allow reverting to older revisions of a snap To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1948995/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1948995] Re: Allow reverting to older revisions of a snap
@anonymouse67 hello ! so how do you suggest we fix our production system now ? ** Changed in: snapd (Ubuntu) Status: Invalid => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1948995 Title: Allow reverting to older revisions of a snap To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1948995/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1943024] [NEW] unreliable/racy/incomplete interface state detection at boot
Public bug reported: Hi, This is somewhat related to bug 1772137. I have a "firewall" script in routable.d and here's what I get upon boot : https://pastebin.canonical.com/p/bCkQrdm3HB/ I'm interested in the "vlan3060" interface. This interface, once networkd finishes its setup, properly comes up and networkd sees it in state "State: routable (configured)". Thus, I would expect networkd-dispatcher to run routable.d/firewall for this interface. However, as you can see in the paste above, it only runs off.d scripts for this interface. In the aforementioned bug, we (Ubuntu) added "--run-startup-triggers" in /etc/default/networkd-dispatcher which explains why networkd-dispatcher is running off.d scripts for vlan3060. This is fine. However, I would expect it to also run carrier.d and routable.d script as this interface becomes fully functional. Thanks ** Affects: networkd-dispatcher (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1943024 Title: unreliable/racy/incomplete interface state detection at boot To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/networkd-dispatcher/+bug/1943024/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1922089] Re: [ovn] enable_snat cannot be disabled once enabled
Sorry I don't have access to a lab where I can patch neutron, I'm not going to be able to test the patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1922089 Title: [ovn] enable_snat cannot be disabled once enabled To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1922089/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1922089] [NEW] [ovn] enable_snat cannot be disabled once enabled
Public bug reported:
Hi,
Using Openstack focal/ussuri - ovn version 20.03.1-0ubuntu1.2 and
neutron 2:16.2.0-0ubuntu2.
If "enable_snat" is enabled on an external gateway on a router, it's not
possible to disable it without completely removing said gateway from the
router.
For example :
I have a subnet called subnet_axino_test - 10.0.100.0/24
I run the following :
$ openstack router create router_axino_test
$ openstack router set --disable-snat --external-gateway net_stg-external
router_axino_test
$ openstack router add subnet router_axino_test subnet_axino_test
And so on OVN, I get nothing :
$ sudo ovn-nbctl list NAT |grep -B5 -A4 10.131.100.0/24
Now, I enable SNAT :
$ openstack router set --enable-snat --external-gateway net_stg-external
router_axino_test
This correctly adds an OVN SNAT entry as follows :
$ sudo ovn-nbctl list NAT |grep -B5 -A4 10.131.100.0/24
_uuid : a65cc4b8-14ae-4ce4-b274-10eefdcc51dc
external_ids: {}
external_ip : "A.B.C.D"
external_mac: []
logical_ip : "10.131.100.0/24"
logical_port: []
options : {}
type: snat
Now, I remove SNAT from the router :
$ openstack router set --disable-snat --external-gateway net_stg-external
router_axino_test
I confirm this :
$ openstack router show router_axino_test | grep enable_snat
| external_gateway_info | {"network_id":
"4fb8304e-7adb-4cc3-bae5-deb968263eb0", "external_fixed_ips": [{"subnet_id":
"6d47-1e44-41af-8f64-dd802d5c3ddc", "ip_address": "A.B.C.D"}],
"enable_snat": false} |
Above, you can see that "enable_snat" is "false". So I would expect OVN to
_not_ have a NAT entry. Yet, it does :
$ sudo ovn-nbctl list NAT |grep -B5 -A4 10.131.100.0/24
_uuid : a65cc4b8-14ae-4ce4-b274-10eefdcc51dc
external_ids: {}
external_ip : "162.213.34.141"
external_mac: []
logical_ip : "10.131.100.0/24"
logical_port: []
options : {}
type: snat
The only way to remove SNAT is to completely remove the external gateway from
the router, and to re-add it with SNAT disabled :
$ openstack router unset --external-gateway router_axino_test
$ openstack router set --disable-snat --external-gateway net_stg-external
router_axino_test
Note that this requires removing all the floating IPs from VMs behind
this router, which obviously makes them unreachable - which is less than
ideal in production.
Thanks
** Affects: neutron (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1922089
Title:
[ovn] enable_snat cannot be disabled once enabled
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1922089/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1918936] [NEW] ipset does NSS lookups even if ports are numeric
Public bug reported: Hi, Do you think we could get https://git.netfilter.org/ipset/commit/?id=dbeb20a667e82e4efb8b26b24a0ec641dab5c857 SRUed to 20.04 ? This divides our ipset loading time by ~2 (from ~60s to ~25s). Thanks ** Affects: ipset (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1918936 Title: ipset does NSS lookups even if ports are numeric To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ipset/+bug/1918936/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915156] Re: sudoers file ignored
I upgraded snapd, but the file is still here : $ dpkg -l snapd Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-p> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==---==> ii snapd 2.48.3+20.04 amd64Daemon and tooling that enable> $ sudo ls -l /etc/sudoers.d/99-snapd.conf -r--r- 1 root root 91 Jul 10 2020 /etc/sudoers.d/99-snapd.conf $ dpkg -S /etc/sudoers.d/99-snapd.conf snapd: /etc/sudoers.d/99-snapd.conf $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 20.04.2 LTS Release:20.04 Codename: focal Perhaps it's not removed on upgrades ? ** Changed in: snapd (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915156 Title: sudoers file ignored To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1915156/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915156] [NEW] sudoers file ignored
Public bug reported: Hi, snapd package version 2.46.1+20.04 on focal creates a file named /etc/sudoers.d/99-snapd.conf However, man sudoers says : For example, given: #includedir /etc/sudoers.d sudo will suspend processing of the current file and read each file in /etc/sudoers.d, skipping file names that end in ‘~’ or contain a ‘.’ character to avoid causing problems with package manager or ed‐ itor temporary/backup files. Since the file has a '.' in its name, it is ignored. I haven't seen an actual problem with this, but if this file is not required then it should be removed. Thanks ! ** Affects: snapd (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915156 Title: sudoers file ignored To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1915156/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1914447] Re: geneve overlay network on vlan interface broken with offload enabled
** Tags added: ps5 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1914447 Title: geneve overlay network on vlan interface broken with offload enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-signed-hwe-5.8/+bug/1914447/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1900668] Re: MAAS PXE Boot stalls with grub 2.02
You're using the right interface. FW and BIOS versions are the same on all nodes. Leaving this bug in Incomplete state since I'm not directly impacted by this bug anymore. I guess we'll see what happens when the cloud gets redeployed from scratch (presumably by Field). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1900668 Title: MAAS PXE Boot stalls with grub 2.02 To manage notifications about this bug go to: https://bugs.launchpad.net/maas-images/+bug/1900668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1900668] Re: MAAS PXE Boot stalls with grub 2.02
@xnox I was able to hit this bug while deploying a single server, so it's definitely not a case of the rackd not keeping up. Which grub versions did you try ? How are you updating the GRUB that MAAS serves ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1900668 Title: MAAS PXE Boot stalls with grub 2.02 To manage notifications about this bug go to: https://bugs.launchpad.net/maas-images/+bug/1900668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1900668] Re: MAAS PXE Boot stalls with grub 2.02
@xnox as far as I know, the Foundations team still has access to the test hardware, if you want to test this new grub. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1900668 Title: MAAS PXE Boot stalls with grub 2.02 To manage notifications about this bug go to: https://bugs.launchpad.net/maas-images/+bug/1900668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1900668] Re: MAAS PXE Boot stalls with grub 2.02
The test environment for the Ubuntu Foundations team is now ready. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1900668 Title: MAAS PXE Boot stalls with grub 2.02 To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1900668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824372] Re: mlx5 : not able to set a high MTU on the representor port for a VF
Sorry for various reasons I won't be able to test 20.04 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824372 Title: mlx5 : not able to set a high MTU on the representor port for a VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824372/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826376] Re: cxgb4 : no VXLAN offloading with distro driver, only with OFED drivers
Sorry I won't be able to test 20.04 as I don't have a Chelsio NIC anymore -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826376 Title: cxgb4 : no VXLAN offloading with distro driver, only with OFED drivers To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1826376/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1860926] Re: Ubuntu 20.04 Systemd fails to configure bridged network
systemd-245.4-4ubuntu3.1 fixes the problem for me as well -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1860926 Title: Ubuntu 20.04 Systemd fails to configure bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1860926/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1874708] [NEW] mongodb SEGFAULT on MapReduce
Public bug reported:
Hi,
Ubuntu 18.04, mongodb version 1:3.6.3-0ubuntu1
Running this command makes mongodb crash :
> db.runCommand({ mapreduce: "", map: " function() { var k =
emitKey; emit(k+'@'+NumberInt(this.t/86400), this.c); }", reduce:
"function(key, values) { return Array.sum(values); }", out: { inline: 1
}, query: { k: { $regex: "^16m:16o:j1:1ni:.+" } }, scope: { emitKey:
"16m:16o:j1:1ni:*" } })
I can provide a dump of the collection for reproducibility.
Stacktrace :
2020-04-23T07:01:03.777+ F -[conn428] Invalid access at address: 0
2020-04-23T07:01:03.793+ F -[conn428] Got signal: 11 (Segmentation
fault).
0x55f760d5e4ca 0x55f760d5d78e 0x55f760d5dddc 0x55f75fed35e6 0x7f57d0c2d890
0x55f7603a89b0 0x55f760393fd6 0x55f7603ae07b 0x55f7603ae74d 0x55f7603af390
0x55f76018eac6 0x55f76039e47a 0x55f7601cd585 0x55f7601ce2dd 0x55f7600dfd24
0x55f7600e1cef 0x55f7600f186e 0x55f7600eae5e 0x55f7600ee5fd 0x55f7600fed39
0x3698771d926d
- BEGIN BACKTRACE -
{"backtrace":[{"b":"55F75ED23000","o":"203B4CA","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55F75ED23000","o":"203A78E"},{"b":"55F75ED23000","o":"203ADDC"},{"b":"55F75ED23000","o":"11B05E6"},{"b":"7F57D0C1B000","o":"12890"},{"b":"55F75ED23000","o":"16859B0","s":"_ZN2js14TenuringTracer8traverseI8JSObjectEEvPPT_"},{"b":"55F75ED23000","o":"1670FD6","s":"_ZN2js8frontend9ObjectBox5traceEP8JSTracer"},{"b":"55F75ED23000","o":"168B07B","s":"_ZN2JS12AutoGCRooter8traceAllEP8JSTracer"},{"b":"55F75ED23000","o":"168B74D","s":"_ZN2js2gc9GCRuntime11markRuntimeEP8JSTracerNS1_18TraceOrMarkRuntimeE"},{"b":"55F75ED23000","o":"168C390","s":"_ZN2js7Nursery7collectEP9JSRuntimeN2JS8gcreason6ReasonEPN7mozilla6VectorIPNS_11ObjectGroupELm0ENS_17SystemAllocPolicyEEE"},{"b":"55F75ED23000","o":"146BAC6","s":"_ZN2js2gc9GCRuntime7minorGCEP9JSContextN2JS8gcreason6ReasonE"},{"b":"55F75ED23000","o":"167B47A","s":"_ZN2js8AllocateI8JSObjectLNS_7AllowGCE1EEEPS1_PNS_16ExclusiveContextENS_2gc9AllocKindEmNS6_11InitialHeapEPKNS_5ClassE"},{"b":"55F75ED23000","o":"14AA585"},{"b":"55F75ED23000","o":"14AB2DD","s":"_ZN2js29NewObjectWithClassProtoCommonEPNS_16ExclusiveContextEPKNS_5ClassEN2JS6HandleIP8JSObjectEENS_2gc9AllocKindENS_13NewObjectKindE"},{"b":"55F75ED23000","o":"13BCD24"},{"b":"55F75ED23000","o":"13BECEF"},{"b":"55F75ED23000","o":"13CE86E"},{"b":"55F75ED23000","o":"13C7E5E"},{"b":"55F75ED23000","o":"13CB5FD"},{"b":"55F75ED23000","o":"13DBD39"},{"b":"0","o":"3698771D926D"}],"processInfo":{
"mongodbVersion" : "3.6.3", "gitVersion" :
"9586e557d54ef70f9ca4b43c26892cd55257e1a5", "compiledModules" : [], "uname" : {
"sysname" : "Linux", "release" : "4.15.0-88-generic", "version" : "#88-Ubuntu
SMP Tue Feb 11 20:11:34 UTC 2020", "machine" : "x86_64" }, "somap" : [ { "b" :
"55F75ED23000", "elfType" : 3, "buildId" :
"40A22A63C3F04AF7F9D3983994C20023104C5804" }, { "b" : "7FFDABFF9000", "path" :
"linux-vdso.so.1", "elfType" : 3, "buildId" :
"9478314FB21D71BAA43D3A46B2748A2F95E9C011" }, { "b" : "7F57D36B", "path" :
"/usr/lib/x86_64-linux-gnu/libstemmer.so.0d", "elfType" : 3, "buildId" :
"278CA72E21C11FF2E15A86B0B2C13A8922951702" }, { "b" : "7F57D3493000", "path" :
"/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" :
"EF3E006DFE3132A41D4D4DC0E407D6EA658E11C4" }, { "b" : "7F57D328B000", "path" :
"/usr/lib/x86_64-linux-gnu/libsnappy.so.1", "elfType" : 3, "buildId" :
"55765D88D03CC928130D788F1C7E4BF8415AC7E3" }, { "b" : "7F57D3011000", "path" :
"/usr/lib/x86_64-linux-gnu/libyaml-cpp.so.0.5", "elfType" : 3, "buildId" :
"BF65D47C8CD968E616F7D179F84A80CA71DB8249" }, { "b" : "7F57D2E08000", "path" :
"/usr/lib/x86_64-linux-gnu/libpcrecpp.so.0", "elfType" : 3, "buildId" :
"089B8438CC1394E978E56C556C9CAE768BD2F18C" }, { "b" : "7F57D2B87000", "path" :
"/usr/lib/x86_64-linux-gnu/libboost_program_options.so.1.65.1", "elfType" : 3,
"buildId" : "9F69F11220BB1FAAB0B73A2B6F4B0E81D9B901CE" }, { "b" :
"7F57D296D000", "path" :
"/usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.65.1", "elfType" : 3,
"buildId" : "32B8421A0643426D9FB008005F5A86688065008B" }, { "b" :
"7F57D2768000", "path" : "/usr/lib/x86_64-linux-gnu/libboost_system.so.1.65.1",
"elfType" : 3, "buildId" : "4BA851D242F2DB710CB1817DE860CF97AE2F9714" }, { "b"
: "7F57D24F8000", "path" : "/usr/lib/x86_64-linux-gnu/libtcmalloc.so.4",
"elfType" : 3, "buildId" : "572D5C17FBDA6B678DF653411F676819DE18CA6B" }, { "b"
: "7F57D22DD000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" :
3, "buildId" : "390E9CC4C215314B6D8ADE6D6E28F8518418039C" }, { "b" :
"7F57D205", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.1", "elfType" :
3, "buildId" : "38AE5B4499DA422AA6D86BAED1902662DED5F730" }, { "b" :
"7F57D1B85000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1",
"elfType" : 3, "buildId" : "68CECD8742C0C91CDC82709C3E52A8C9F9451FC5" }, { "b"
: "7F57D197D000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3,
"buildId" : "9826FBDF57ED7D6965131074CB3C08B1009C1CD8" }, {
[Bug 1871540] Re: Include IPs in apt-get output
@juliank : could we at least get a debug flag of some sort that would just log everything ? With timings ? So that we could map, for each file, the IP of the server it got downloaded from, time when the download started (maybe even time when the HTTP request got sent, time when we start receiving data), time at the end of transfer. And also, very important, proxy information as well, if any. And so whenever someones reports "slowness" or misbehaviour, we can give him this flag and ask for output and debug on our end. Would that be possible ? Thanks ! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1871540 Title: Include IPs in apt-get output To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1871540/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1871540] Re: Include IPs in apt-get output
** Changed in: apt (Ubuntu) Status: Won't Fix => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1871540 Title: Include IPs in apt-get output To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1871540/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1871540] Re: Include IPs in apt-get output
Can we maybe get IP and download speed displayed if a "verbose" or "debug" or something flag is specified on the command line ? Something like this maybe : | Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [972 kB] 91.189.88.142 12kB/s What do you think ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1871540 Title: Include IPs in apt-get output To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1871540/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825000] Re: Add ability for mirrors to distinguish interactive and non-interactive apt runs
I think "User-Agent: Debian APT-HTTP/1.3 (2.0.1) non-interactive" would be enough. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825000 Title: Add ability for mirrors to distinguish interactive and non-interactive apt runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1825000/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1799737] Re: l3 agent external_network_bridge broken with ovs
@hopem I think it got installed as Pike but with old charm config options used. I can't argue against "don't use external_network_bridge", and I guess no one uses that in recent openstacks, so no one is getting hit by this bug ? If anything, we should log the exception in https://github.com/openstack/ovsdbapp/blob/8275af1726cb2079afd8f5230377e064221ebcf3/ovsdbapp/backend/ovs_idl/transaction.py#L90 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1799737 Title: l3 agent external_network_bridge broken with ovs To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1799737/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825000] Re: Add ability for mirrors to distinguish interactive and non-interactive apt runs
Probably ! What is it going to be for interactive uses though ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825000 Title: Add ability for mirrors to distinguish interactive and non-interactive apt runs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1825000/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1859649] Re: networking disruption on upgrade from 14.0.0 to 14.0.3
So how does that work when we use Landscape to auto-upgrades the packages ? ** Changed in: cloud-archive Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1859649 Title: networking disruption on upgrade from 14.0.0 to 14.0.3 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1859649/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1859649] Re: neutron 2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0 not compatible
The other in neutron-l3-agent (still on neutron-gateway), but this is
probably due to the upgrade restarting the daemons :
neutron-l3-agent.log:2020-01-14 16:22:11.298 456968 ERROR pyroute2.netns.nslink
[-] forced shutdown procedure, clean up netns manually
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info [-] [Errno 9] Bad file descriptor: OSError: [Errno
9] Bad file descriptor
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info Traceback (most recent call last):
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/common/utils.py", line 158, in call
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info return func(*args, **kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 1186, in
process
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info self._process_internal_ports()
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 584, in
_process_internal_ports
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info self.internal_network_added(p)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 486, in
internal_network_added
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info mtu=port.get('mtu'))
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 461, in
_internal_network_added
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info prefix=prefix, mtu=mtu)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/agent/linux/interface.py", line 265, in
plug
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info namespace=namespace):
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 818, in
device_exists
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info return IPDevice(device_name,
namespace=namespace).exists()
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 318, in
exists
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info return privileged.interface_exists(self.name,
self.namespace)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/neutron/privileged/agent/linux/ip_lib.py", line
50, in sync_inner
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info return input_func(*args, **kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 241, in
_wrap
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info return self.channel.remote_call(name, args,
kwargs)
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info File
"/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 203, in
remote_call
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info raise exc_type(*result[2])
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info OSError: [Errno 9] Bad file descriptor
neutron-l3-agent.log:2020-01-14 16:22:11.322 456743 ERROR
neutron.agent.l3.router_info
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR
neutron.agent.l3.agent [-] Failed to process compatible router:
209673dc-a96b-4367-8b48-08bdf8f5ad5e: OSError: [Errno 9] Bad file descriptor
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR
neutron.agent.l3.agent Traceback (most recent call last):
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR
neutron.agent.l3.agent File
"/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 723, in
_process_routers_if_compatible
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR
neutron.agent.l3.agent self._process_router_if_compatible(router)
neutron-l3-agent.log:2020-01-14 16:22:11.327 456743 ERROR
neutron.agent.l3.agent File
[Bug 1859649] Re: neutron 2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0 not compatible
All errors appear to be on the neutron gateway. One kind in neutron-ovs-agent : neutron-openvswitch-agent.log:2020-01-14 16:21:31.563 456292 ERROR neutron.agent.rpc neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc [req-c5918d0d-f11b-4de7-b77c-f4628e33774d - - - - -] Failed to get details for device ab683ddb-a1f3-4404-b2b4-f912efe3530a: oslo_messaging.rpc.client.RemoteError: Remote error: InvalidTargetVersion Invalid target version 1.5 neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc Traceback (most recent call last): neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/neutron/agent/rpc.py", line 303, in get_devices_details_list_and_failed_devices neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc agent_restarted)) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/neutron/agent/rpc.py", line 312, in get_device_details neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc resources.PORT, device, agent_restarted) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/neutron/agent/resource_cache.py", line 61, in get_resource_by_id neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc agent_restarted=agent_restarted) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/neutron/agent/resource_cache.py", line 79, in _flood_cache_for_query neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc filter_kwargs=filter_kwargs) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/oslo_log/helpers.py", line 67, in wrapper neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc return method(*args, **kwargs) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/neutron/api/rpc/handlers/resources_rpc.py", line 114, in bulk_pull neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc version=resource_type_cls.VERSION, filter_kwargs=filter_kwargs) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/neutron_lib/rpc.py", line 157, in call neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc return self._original_context.call(ctxt, method, **kwargs) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 178, in call neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc retry=self.retry) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 128, in _send neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc retry=retry) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc call_monitor_timeout, retry=retry) neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc raise result neutron-openvswitch-agent.log:2020-01-14 16:21:31.674 456292 ERROR neutron.agent.rpc oslo_messaging.rpc.client.RemoteError: Remote error: InvalidTargetVersion Invalid target version 1.5 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1859649 Title: neutron 2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0 not compatible To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1859649/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1859649] [NEW] neutron 2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0 not compatible
Public bug reported: Hi, Not entirely sure why, but if a cloud has services running these 2 versions of neutron (2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0), networking is basically broken until everything is running 2:14.0.3-0ubuntu1~cloud0. Which causes networking disruption when not all nodes are upgraded at the same time. Thanks ** Affects: neutron (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1859649 Title: neutron 2:14.0.3-0ubuntu1~cloud0 and 2:14.0.0-0ubuntu1.1~cloud0 not compatible To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1859649/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826857] Re: offloading disabled when using linuxbrige or bonding
Correction : I tested today and this bug cannot be reproduced without OVS, at least on 4.15.0-55-generic. Offloading works properly even when the interface is in a linuxbridge or in a bond. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826857 Title: offloading disabled when using linuxbrige or bonding To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1826857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826857] Re: offloading disabled when using linuxbrige or bonding
I don't think I tested with OVS out of the picture. However, none of the physical interface was enslaved to an OVS bridge, so I'd say this is not an ovs bug and can likely be repro'ed outside of OVS. I don't have time to do this in the short term though. ** Changed in: openvswitch (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826857 Title: offloading disabled when using linuxbrige or bonding To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1826857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1836475] [NEW] update-notifier-common weekly cron job runs at the same time for all computers across the globe
Public bug reported: Hi, /etc/cron.weekly/update-notifier-common is run at the same time on all machines across the globe. It's putting unnecessary stress on Canonical's infrastructure (changelogs.ubuntu.com, specifically). Could we please have this job spread out over 24h ? And could we please SRU this change ? Thanks ** Affects: update-notifier (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1836475 Title: update-notifier-common weekly cron job runs at the same time for all computers across the globe To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1836475/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824372] Re: mlx5 : not able to set a high MTU on the representor port for a VF
Actually the cma_alloc error is present much before VF creation. dmesg is attached. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824372 Title: mlx5 : not able to set a high MTU on the representor port for a VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824372/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824372] Re: mlx5 : not able to set a high MTU on the representor port for a VF
** Attachment added: "kern.log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824372/+attachment/5259994/+files/kern.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824372 Title: mlx5 : not able to set a high MTU on the representor port for a VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824372/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824372] Re: mlx5 : not able to set a high MTU on the representor port for a VF
I am unable to test with latest mainline (5.1.0-050100rc6-generic) as the creation of a single VF starts spamming dmesg with "cma: cma_alloc: alloc failed, req-size: 1 pages, ret: -12" ** Tags added: kernel-bug-exists-upstream -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824372 Title: mlx5 : not able to set a high MTU on the representor port for a VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824372/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1821305] Re: csiostor and cxgb4 fight for the NIC
working kern.log ** Attachment added: "kern.log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821305/+attachment/5259992/+files/kern.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1821305 Title: csiostor and cxgb4 fight for the NIC To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821305/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1821305] Re: csiostor and cxgb4 fight for the NIC
Hi, This is indeed fixed in 5.1.0-rc6. ** Tags added: kernel-fixed-upstream -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1821305 Title: csiostor and cxgb4 fight for the NIC To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821305/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826857] [NEW] offloading disabled when using linuxbrige or bonding
Public bug reported:
Hi,
I'm running Ubuntu 18.04 kernel 4.15.0-47-generic, openvswitch
2.10.0-0ubuntu2~cloud0 set up by neutron (from openstack rocky) - VXLAN
overlay.
I've found out that as soon as a linuxbrige or bond is involved the
network path of openvswitch, the offloading features of the NIC
(checksumming, segmentation, VXLAN encap) get disabled, which reduces
the throughput a lot.
You can find the flows below. Examples of setup used :
"Normal" setup :
br-bond0 (192.168.11.11)
bond0
enp50s0f0
enp113s0f0 (handling traffic)
Only interface in the bond :
br-bond0 (192.168.11.11)
bond0
enp50s0f0 (handling traffic)
Interface directly in the bridge :
br-bond0 (192.168.11.11)
enp50s0f0
With the setups above, there is NO offloading happening (tcpdump reports
packet sizes of the MTU (~9k)). If the IP address is directly on the
enp50s0f0 interface, then offloading works.
For what it's worth, the NICs are Mellanox MT27800.
Thanks !
OVS flows :
$ sudo ovs-vsctl show
6f7e2040-e13b-4029-86b7-4f360aad2efe
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge "br-physnet1"
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "phy-br-physnet1"
Interface "phy-br-physnet1"
type: patch
options: {peer="int-br-physnet1"}
Port "br-physnet1"
Interface "br-physnet1"
type: internal
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "sg-eecf32f7-da"
tag: 2
Interface "sg-eecf32f7-da"
type: internal
Port "qvoc654e76e-de"
tag: 2
Interface "qvoc654e76e-de"
Port "qr-84debef4-b7"
tag: 2
Interface "qr-84debef4-b7"
type: internal
Port "qr-3e9ad9a7-02"
tag: 3
Interface "qr-3e9ad9a7-02"
type: internal
Port "fg-997d43cb-27"
tag: 5
Interface "fg-997d43cb-27"
type: internal
Port "qvoba76f8a7-50"
tag: 2
Interface "qvoba76f8a7-50"
Port "qvod01a61c2-59"
tag: 2
Interface "qvod01a61c2-59"
Port "enp50s0f0_60"
tag: 2
Interface "enp50s0f0_60"
Port "tap15879e40-d9"
tag: 2
Interface "tap15879e40-d9"
Port "int-br-physnet1"
Interface "int-br-physnet1"
type: patch
options: {peer="phy-br-physnet1"}
Port "qg-3f2f0aa4-fa"
tag: 5
Interface "qg-3f2f0aa4-fa"
type: internal
Port "ha-a481b72f-6a"
tag: 4
Interface "ha-a481b72f-6a"
type: internal
Port "enp50s0f0_62"
tag: 2
Interface "enp50s0f0_62"
Port "sg-5e3c2880-79"
tag: 3
Interface "sg-5e3c2880-79"
type: internal
Port "tap62452ada-90"
tag: 3
Interface "tap62452ada-90"
Port br-int
Interface br-int
type: internal
Port "qvo75f1a5f8-e6"
tag: 2
Interface "qvo75f1a5f8-e6"
Port "tapbc9a8481-c2"
tag: 2
Interface "tapbc9a8481-c2"
Port "tapd7155195-25"
tag: 1
Interface "tapd7155195-25"
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "qvo1e23c34e-5f"
tag: 2
Interface "qvo1e23c34e-5f"
Port "enp50s0f0_61"
tag: 2
Interface "enp50s0f0_61"
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Port "vxlan-c0a80a0b"
Interface "vxlan-c0a80a0b"
type: vxlan
options: {df_default="true", in_key=flow,
local_ip="192.168.11.11", out_key=flow, remote_ip="192.168.10.11"}
ovs_version: "2.10.0"
Features for all interfaces :
$ for i in br-bond0 bond0 enp50s0f0 enp113s0f0; do sudo ethtool -k $i; echo
"=="; done
Features for br-bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: on
tcp-segmentation-offload: on
[Bug 1826376] [NEW] cxgb4 : no VXLAN offloading with distro driver, only with OFED drivers
Public bug reported:
Hi,
Using ubuntu 18.04 with kernel 4.15.0-47-generic, my Chelsio NIC can't
offload VXLAN with the distro drivers (features "tx-udp_tnl-
segmentation" and "tx-udp_tnl-csum-segmentation"), but with the OFED
driver ("ChelsioUwire-3.10.0.0"), it can.
Can we please get this in the distro driver ?
Thanks
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: linux-image-4.15.0-47-generic 4.15.0-47.50
ProcVersionSignature: Ubuntu 4.15.0-47.50-generic 4.15.18
Uname: Linux 4.15.0-47-generic x86_64
AlsaDevices:
total 0
crw-rw 1 root audio 116, 1 Apr 19 14:48 seq
crw-rw 1 root audio 116, 33 Apr 19 14:48 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.6
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
Date: Thu Apr 25 08:53:20 2019
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
MachineType: Supermicro Super Server
PciMultimedia:
ProcEnviron:
TERM=screen-256color
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
ProcFB: 0 astdrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-47-generic
root=UUID=7b06790c-9447-446a-b1cf-b25104638b26 ro console=ttyS1,115200 nosplash
iommu=pt amd_iommu=on l1tf=full module_blacklist=csiostor
RelatedPackageVersions:
linux-restricted-modules-4.15.0-47-generic N/A
linux-backports-modules-4.15.0-47-generic N/A
linux-firmware 1.173.3
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 02/21/2019
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 1.2
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: H11DSU-iN
dmi.board.vendor: Supermicro
dmi.board.version: 1.02A
dmi.chassis.asset.tag: To be filled by O.E.M.
dmi.chassis.type: 17
dmi.chassis.vendor: Supermicro
dmi.chassis.version: 0123456789
dmi.modalias:
dmi:bvnAmericanMegatrendsInc.:bvr1.2:bd02/21/2019:svnSupermicro:pnSuperServer:pvr0123456789:rvnSupermicro:rnH11DSU-iN:rvr1.02A:cvnSupermicro:ct17:cvr0123456789:
dmi.product.family: To be filled by O.E.M.
dmi.product.name: Super Server
dmi.product.version: 0123456789
dmi.sys.vendor: Supermicro
** Affects: linux (Ubuntu)
Importance: Undecided
Status: Confirmed
** Tags: amd64 apport-bug bionic uec-images
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1826376
Title:
cxgb4 : no VXLAN offloading with distro driver, only with OFED drivers
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1826376/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 871907] Re: vim should have "set bg=dark" by default
** Information type changed from Public Security to Public -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/871907 Title: vim should have "set bg=dark" by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/vim/+bug/871907/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826237] [NEW] Not possible to offload ct_state flows (OVS firewalling in openstack)
Public bug reported: Hi, Running Ubuntu 18.04, kernel 4.15.0-47-generic, openvswitch 2.10.0-0ubuntu2~cloud0. I enabled OVS offloading in OpenStack by following https://docs.openstack.org/neutron/rocky/admin/config-ovs-offload.html It would appear that if you use OVS firewalling in neutron, neutron will create flows with the "ct_state" property, and these flows cannot be offloaded : (from /var/log/openvswitch/ovs-vswitchd.log) : 2019-04-24T13:01:20.634Z|00968|dpif(handler550)|DBG|system@ovs-system: miss upcall: recirc_id(0),dp_hash(0),skb_priority(0),in_port(2),skb_mark(0),ct_state(0),ct_zone(0),ct_mark(0),ct_label(0),eth(src=1a:b2:bd:d6:df:8f,dst=01:80:c2:00:00:0e),eth_type(0x 88cc) vlan_tci=0x,dl_src=1a:b2:bd:d6:df:8f,dl_dst=01:80:c2:00:00:0e,dl_type=0x88cc 2019-04-24T13:01:20.634Z|00969|netdev_tc_offloads(handler550)|DBG|offloading attribute ct_state isn't supported 2019-04-24T13:01:20.634Z|00970|dpif_netlink(handler550)|ERR|failed to offload flow: Operation not supported Thanks ** Affects: openvswitch (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826237 Title: Not possible to offload ct_state flows (OVS firewalling in openstack) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1826237/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825147] Re: ovs flooding packets, not learning MAC addresses
*** This bug is a duplicate of bug 1732067 *** https://bugs.launchpad.net/bugs/1732067 Hi - it is indeed a duplicate. Marking as such - thanks ! ** This bug has been marked a duplicate of bug 1732067 openvswitch firewall flows cause flooding on integration bridge -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825147 Title: ovs flooding packets, not learning MAC addresses To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1825147/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825147] Re: ovs flooding packets, not learning MAC addresses
An additional datapoint : MAC learning appears to be working fine for subnets not attached to a router. As soon as I attach the subnet to a router, the bad behaviour starts. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825147 Title: ovs flooding packets, not learning MAC addresses To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1825147/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825162] [NEW] neutron CLI not displaying anything even though an error is encountered
Public bug reported:
Hi,
I'm using the openstack rocky client ("openstack 3.16.2" apparently -
from the "openstackclients" snap, rocky v37).
Trying to add a router interface for a subnet that is out of IP, the
neutron client get an error, but nothing is displayed back to the user.
The neutron client should make it pretty clear that the request failed,
and why :
$ openstack router add subnet MyRouter uselesssubnet --debug
[...]
REQ: curl -g -i -X GET http://192.168.0.22:9696/v2.0/routers -H "Accept:
application/json" -H "User-Agent: openstacksdk/0.17.2 keystoneauth1/3.10.0
python-requests/2.19.1 CPython/3.5.2" -H "X-Auth-Token:
{SHA1}0e210f0581f07ec7bf245b9ac391e5aa552af3bd"
http://192.168.0.22:9696 "GET /v2.0/routers HTTP/1.1" 200 695
RESP: [200] Connection: keep-alive Content-Length: 695 Content-Type:
application/json Date: Wed, 17 Apr 2019 11:52:08 GMT X-Openstack-Request-Id:
req-ec539fce-526f-4d2b-99f7-d190a503bd62
RESP BODY: {"routers": [{"id": "2e78c120-5e25-4ca5-b8ca-91c98c4a386a", "name":
"MyRouter", "tenant_id": "8d9dfa0fb0924053bb39d19a2f436282", "admin_state_up":
true, "status": "ACTIVE", "external_gateway_info": {"network_id":
"aee1250b-20c9-4697-8a75-223af7353ecf", "enable_snat": true,
"external_fixed_ips": [{"subnet_id": "cad3a8de-b1ec-4acc-baae-dcef67e6ece0",
"ip_address": "192.168.0.163"}]}, "description": "", "availability_zones":
["nova"], "distributed": true, "ha": true, "availability_zone_hints": [],
"routes": [], "flavor_id": null, "tags": [], "created_at":
"2019-04-12T07:14:18Z", "updated_at": "2019-04-17T10:29:15Z",
"revision_number": 16, "project_id": "8d9dfa0fb0924053bb39d19a2f436282"}]}
GET call to network for http://192.168.0.22:9696/v2.0/routers used request id
req-ec539fce-526f-4d2b-99f7-d190a503bd62
Manager unknown ran task network.GET.routers in 0.12100529670715332s
Manager unknown running task network.PUT.routers.add_router_interface
REQ: curl -g -i -X PUT
http://192.168.0.22:9696/v2.0/routers/2e78c120-5e25-4ca5-b8ca-91c98c4a386a/add_router_interface
-H "Content-Type: application/json" -H "User-Agent: openstacksdk/0.17.2
keystoneauth1/3.10.0 python-requests/2.19.1 CPython/3.5.2" -H "X-Auth-Token:
{SHA1}0e210f0581f07ec7bf245b9ac391e5aa552af3bd" -d '{"subnet_id":
"890666f3-dd3d-4596-a477-2369b1c73996"}'
http://192.168.0.22:9696 "PUT
/v2.0/routers/2e78c120-5e25-4ca5-b8ca-91c98c4a386a/add_router_interface
HTTP/1.1" 409 408
RESP: [409] Connection: keep-alive Content-Length: 408 Content-Type:
application/json Date: Wed, 17 Apr 2019 11:52:10 GMT X-Openstack-Request-Id:
req-3e364ad1-331f-4673-89a4-b51e454169e0
RESP BODY: {"NeutronError": {"type": "RouterInterfaceAttachmentConflict",
"message": "Error cannot perform router interface attachment due to Callback
neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create--9223372036851126307
failed with \"No more IP addresses available on network
f135dbc4-f712-4987-9149-0bd0b809e5bb.\" while attempting the operation.",
"detail": ""}}
PUT call to network for
http://192.168.0.22:9696/v2.0/routers/2e78c120-5e25-4ca5-b8ca-91c98c4a386a/add_router_interface
used request id req-3e364ad1-331f-4673-89a4-b51e454169e0
Manager unknown ran task network.PUT.routers.add_router_interface in
1.849557876586914s
clean_up AddSubnetToRouter:
END return value: 0
$
$ echo $?
130
** Affects: neutron (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1825162
Title:
neutron CLI not displaying anything even though an error is
encountered
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1825162/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825147] [NEW] ovs flooding packets, not learning MAC addresses
Public bug reported: Hi, Using OpenStack rocky on Ubuntu 18.04, with dvr_snat and L3HA, and using the openvswitch firewall driver. openvswitch version 2.10.0-0ubuntu2~cloud0. Deployed with juju. I was doing load testing by creating a bunch of instances, and noticed that the network throughput available to instances dropped dramatically as I was creating VMs. In other words, with 2 VMs on my cloud, I had pretty good bandwith, but with 100 (idle) VMs, bandwidth became ridiculously slow. Investigating the problem, I noticed that ovs was flooding traffic : all instances of an hypervisor were getting all the traffic destined to any VM on another hypervisor. In other words, I had vmA1 and vmA2 on hypervisor A, and vmB1 on hypervisor B, then TCP traffic between vmA1 and vmB1 could be seen on vmA2. Digging more into this, I think I located the problem in the ovs MAC learning process, more specifically on br-int (using "sudo ovs-appctl fdb/show br-int"). Traffic flow from vmA1 to vmB1, on hypervisor A, looks like : tap (on br-int), patch-tun (on br-int), patch-int (on br-tun), vxlan to hypervisor B. So whenever traffic comes back (the other way around), the MAC address of vmB1 should be learned, on br-int, on the patch-tun port - and that is not the case. So whenever vmA1 sends traffic to vmB1, at some point it reaches the "NORMAL" action, and since the destination MAC is not learned, traffic is getting flooded : see ofproto/trace https://pastebin.ubuntu.com/p/mbrrj4wPxY/ (see "no learned MAC for destination, flooding") Digging more into this, it would appear that ovs learns a MAC address only from broadcast ARP requests, and not from ARP requests with a unicast MAC address (which is what Linux uses after a successful broadcast ARP request) : https://pastebin.ubuntu.com/p/Sfq775cX6V/. Once the MAC is learned, there's no more flooding : https://pastebin.ubuntu.com/p/bBNHrRKndg/ (see "forwarding to learned port" instead of "no learned MAC for destination, flooding"). Flooding has security consequences (VMs can see traffic not destined to them - although only traffic for VMs in the same neutron network), and performance consequences, so it should be avoided. Thanks ** Affects: neutron Importance: Undecided Status: New ** Affects: neutron (Ubuntu) Importance: Undecided Status: New ** Also affects: neutron (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825147 Title: ovs flooding packets, not learning MAC addresses To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1825147/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824475] [NEW] neutron-rootwrap not authorized to kill neutron-keepalived-state-change
Public bug reported: Hi, Using neutron version 2:13.0.2-0ubuntu1~cloud0, l3ha, dvr-snat. The L3 agent is not authorized to kill neutron-keepalived-state-change (see traceback below), because it's a python3.6 process and the filters only allow python3.5 : $ grep python3 /etc/neutron/rootwrap.d/l3.filters kill_metadata35: KillFilter, root, python3.5, -15, -9 So I guess we need to add python3.6 there : $ ps wwp 20689 PID TTY STAT TIME COMMAND 20689 ?S 0:00 /usr/bin/python3.6 /usr/bin/neutron-keepalived-state-change --router_id=73355faf-6060-4635-a15c-a9900b8cf100 --namespace=snat-73355faf-6060-4635-a15c-a9900b8cf100 --conf_dir=/var/lib/neutron/ha_confs/73355faf-6060-4635-a15c-a9900b8cf100 --monitor_interface=ha-fa344937-01 --monitor_cidr=169.254.0.1/24 --pid_file=/var/lib/neutron/external/pids/73355faf-6060-4635-a15c-a9900b8cf100.monitor.pid --state_path=/var/lib/neutron --user=116 --group=122 Thanks Traceback (the process above is the one mentioned below) : 2019-04-12 06:38:03.974 6656 ERROR neutron.agent.linux.utils [-] Exit code: 99; Stdin: ; Stdout: ; Stderr: /usr/bin/neutron-rootwrap: Unauthorized command: kill -15 20689 (no filter matched) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent [-] Error while deleting router 73355faf-6060-4635-a15c-a9900b8cf100: neutron.common.exceptions.ProcessExecutionError: Exit code: 99; Stdin: ; Stdout: ; Stderr: /usr/bin/neutron-rootwrap: Unauthorized command: kill -15 20689 (no filter matched) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 381, in _safe_router_removed 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent self._router_removed(router_id) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 402, in _router_removed 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent ri.delete() 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent File "/usr/lib/python3/dist-packages/neutron/agent/l3/dvr_edge_router.py", line 220, in delete 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent super(DvrEdgeRouter, self).delete() 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent File "/usr/lib/python3/dist-packages/neutron/agent/l3/ha_router.py", line 451, in delete 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent self.destroy_state_change_monitor(self.process_monitor) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent File "/usr/lib/python3/dist-packages/neutron/agent/l3/ha_router.py", line 388, in destroy_state_change_monitor 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent pm.disable(sig=str(int(signal.SIGTERM))) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent File "/usr/lib/python3/dist-packages/neutron/agent/linux/external_process.py", line 109, in disable 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent utils.execute(cmd, run_as_root=self.run_as_root) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 147, in execute 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent returncode=returncode) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent neutron.common.exceptions.ProcessExecutionError: Exit code: 99; Stdin: ; Stdout: ; Stderr: /usr/bin/neutron-rootwrap: Unauthorized command: kill -15 20689 (no filter matched) 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent 2019-04-12 06:38:03.975 6656 ERROR neutron.agent.l3.agent ** Affects: neutron (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824475 Title: neutron-rootwrap not authorized to kill neutron-keepalived-state- change To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1824475/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824367] Re: mlx5 : not able to set a high MTU on the representor port for a VF
*** This bug is a duplicate of bug 1824372 *** https://bugs.launchpad.net/bugs/1824372 ** This bug has been marked a duplicate of bug 1824372 mlx5 : not able to set a high MTU on the representor port for a VF -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824367 Title: mlx5 : not able to set a high MTU on the representor port for a VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824367/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824372] [NEW] mlx5 : not able to set a high MTU on the representor port for a VF
Public bug reported: Hi, Using Ubuntu 18.04, kernel 4.15.0-47, with a Mellanox MCX556M-ECAT-S25 [1]. With the distro driver, it's impossible to increase the MTU of a representor port : $ sudo ip li set enp50s0f0_0 mtu 1501 RTNETLINK answers: Invalid argument Whereas with the OFED driver [2], I can set it up to 9978 : $ sudo ip li set mtu 9978 enp50s0f0_0 $ This is blocking to use OVS offloading with jumbo frames. Could we please get the feature in the distro driver ? Thanks ! Steps to create the representor : echo 1 | sudo tee /sys/class/net/enp50s0f0/device/sriov_numvfs echo :32:00.3 | sudo tee /sys/bus/pci/drivers/mlx5_core/unbind sudo devlink dev eswitch set pci/:32:00.0 mode switchdev echo :32:00.2 | sudo tee /sys/bus/pci/drivers/mlx5_core/bind [1] http://www.mellanox.com/related-docs/prod_adapter_cards/PB_ConnectX- 5_VPI_Card_SocketDirect.pdf [2] http://www.mellanox.com/page/mlnx_ofed_eula?mtag=linux_sw_drivers=downloads=ofed=MLNX_OFED-4.5-1.0.1.0=MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu18.04-x86_64.tgz ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-image-4.15.0-47-generic 4.15.0-47.50 ProcVersionSignature: Ubuntu 4.15.0-47.50-generic 4.15.18 Uname: Linux 4.15.0-47-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Apr 11 14:18 seq crw-rw 1 root audio 116, 33 Apr 11 14:18 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu7.6 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Date: Thu Apr 11 14:42:57 2019 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' MachineType: Supermicro AS -2023US-TR4 PciMultimedia: ProcEnviron: TERM=screen-256color PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash ProcFB: 0 astdrmfb ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-47-generic root=UUID=89c8c952-93fa-499c-bebe-fd9f4c4fbedb ro console=ttyS1,115200 nosplash iommu=pt amd_iommu=on l1tf=full module_blacklist=csiostor RelatedPackageVersions: linux-restricted-modules-4.15.0-47-generic N/A linux-backports-modules-4.15.0-47-generic N/A linux-firmware 1.173.3 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 02/21/2019 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 1.2 dmi.board.asset.tag: To be filled by O.E.M. dmi.board.name: H11DSU-iN dmi.board.vendor: Supermicro dmi.board.version: 1.02A dmi.chassis.asset.tag: To be filled by O.E.M. dmi.chassis.type: 1 dmi.chassis.vendor: Supermicro dmi.chassis.version: 0123456789 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1.2:bd02/21/2019:svnSupermicro:pnAS-2023US-TR4:pvr0123456789:rvnSupermicro:rnH11DSU-iN:rvr1.02A:cvnSupermicro:ct1:cvr0123456789: dmi.product.family: To be filled by O.E.M. dmi.product.name: AS -2023US-TR4 dmi.product.version: 0123456789 dmi.sys.vendor: Supermicro ** Affects: linux (Ubuntu) Importance: Undecided Status: Confirmed ** Tags: amd64 apport-bug bionic uec-images -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824372 Title: mlx5 : not able to set a high MTU on the representor port for a VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824372/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1824367] [NEW] mlx5 : not able to set a high MTU on the representor port for a VF
Public bug reported: Hi, Using Ubuntu 18.04, kernel 4.15.0-47, with a Mellanox MCX556M-ECAT-S25 [1]. With the distro driver, it's impossible to increase the MTU of a representor port : $ sudo ip li set enp50s0f0_0 mtu 1501 RTNETLINK answers: Invalid argument Whereas with the OFED driver [2], I can set it up to 9978 : $ sudo ip li set mtu 9978 enp50s0f0_0 $ This is blocking to use OVS offloading with jumbo frames. Could we please get the feature in the distro driver ? Thanks ! Steps to create the representor : echo 1 | sudo tee /sys/class/net/enp50s0f0/device/sriov_numvfs echo :32:00.3 | sudo tee /sys/bus/pci/drivers/mlx5_core/unbind sudo devlink dev eswitch set pci/:32:00.0 mode switchdev echo :32:00.2 | sudo tee /sys/bus/pci/drivers/mlx5_core/bind [1] http://www.mellanox.com/related-docs/prod_adapter_cards/PB_ConnectX- 5_VPI_Card_SocketDirect.pdf [2] http://www.mellanox.com/page/mlnx_ofed_eula?mtag=linux_sw_drivers=downloads=ofed=MLNX_OFED-4.5-1.0.1.0=MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu18.04-x86_64.tgz ** Affects: linux (Ubuntu) Importance: Undecided Status: Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1824367 Title: mlx5 : not able to set a high MTU on the representor port for a VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824367/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1821345] Re: Mellanox MT27800 / mlx5_core : cannot bring up VFs if the total number of VFs is >= 64 : alloc irq vectors failed
Here is the reply from Mellanox support : == • There is a limitations on the firmware/driver for ConnectX-5 Socket direct adapter • There is a limitation on the number of total PFs+VFs that wecan allocate on the adapter which is 256. • The max number of VF's of each port/PF is 63 VF's => 63*4=252 . Currently on your setup NUM_PF_MSIX==63. for all PFs it is 252. In order to bring up the max supported number of VF'S=63 , please reduce the NUM_PF_MSIX to 32 - mlxconfig -d /dev/mst/mt4119_pciconf0 set NUM_PF_MSIX=32 - mlxconfig -d /dev/mst/mt4119_pciconf0.1 set NUM_PF_MSIX=32 - mlxconfig -d /dev/mst/mt4119_pciconf1 set NUM_PF_MSIX=32 - mlxconfig -d /dev/mst/mt4119_pciconf1.1 set NUM_PF_MSIX=32 == And indeed, setting NUM_PF_MSIX to 32 allowed me to create 4*63 VFs. ** Changed in: linux (Ubuntu) Status: Confirmed => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1821345 Title: Mellanox MT27800 / mlx5_core : cannot bring up VFs if the total number of VFs is >= 64 : alloc irq vectors failed To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1821345] Re: Mellanox MT27800 / mlx5_core : cannot bring up VFs if the total number of VFs is >= 64 : alloc irq vectors failed
Note : if NUM_OF_VFS is set to 32, VF creation works fine. If it's set to 64 or above, VF creation fails. I haven't tested in between. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1821345 Title: Mellanox MT27800 / mlx5_core : cannot bring up VFs if the total number of VFs is >= 64 : alloc irq vectors failed To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1821345] [NEW] Mellanox MT27800 / mlx5_core : cannot bring up VFs if the total number of VFs is >= 64 : alloc irq vectors failed
Public bug reported: Hi, For the Mellanox NIC MT27800 (MCX556M-ECAT-S25 to be precise), the maximum number of VF per PF can be configured in the firmware, by running e.g. : $ sudo mstconfig -d 32:00.1 set NUM_OF_VFS=127 If I set this NUM_OF_VFS to 64 or more, the creation of even a single VF fails mid-way : $ echo 1 | sudo tee /sys/class/net/enp50s0f0/device/sriov_numvfs 1 dmesg shows : [101418.090970] (:32:00.0): E-Switch: E-Switch enable SRIOV: nvfs(1) mode (1) [101418.182736] (:32:00.0): E-Switch: SRIOV enabled: active vports(2) [101418.289642] pci :32:00.2: [15b3:1018] type 00 class 0x02 [101418.290235] pci :32:00.2: Max Payload Size set to 512 (was 128, max 512) [101418.290261] pci :32:00.2: enabling Extended Tags [101418.292432] iommu: Adding device :32:00.2 to group 139 [101418.292580] iommu: Using direct mapping for device :32:00.2 [101418.292979] mlx5_core :32:00.2: enabling device ( -> 0002) [101418.293719] mlx5_core :32:00.2: firmware version: 16.24.1000 [101418.550589] mlx5_core :32:00.2: alloc irq vectors failed [101418.974082] mlx5_core :32:00.2: mlx5_load_one failed with error code -28 [101418.981656] mlx5_core: probe of :32:00.2 failed with error -28 This is a problem with both the Ubuntu kernel driver and the OFED driver. Thanks ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-image-4.15.0-46-generic 4.15.0-46.49 ProcVersionSignature: User Name 4.15.0-46.49-generic 4.15.18 Uname: Linux 4.15.0-46-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Mar 21 08:33 seq crw-rw 1 root audio 116, 33 Mar 21 08:33 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu7.6 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Date: Fri Mar 22 12:43:26 2019 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' MachineType: Supermicro AS -2023US-TR4 PciMultimedia: ProcEnviron: TERM=screen-256color PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=C.UTF-8 SHELL=/bin/bash ProcFB: 0 astdrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.15.0-46-generic root=UUID=52d72a05-517b-4c16-8dcb-486bdebcc695 ro console=ttyS1,115200 nosplash iommu=pt amd_iommu=on processor.max_cstate=0 l1tf=full cpuidle.off=1 RelatedPackageVersions: linux-restricted-modules-4.15.0-46-generic N/A linux-backports-modules-4.15.0-46-generic N/A linux-firmware 1.173.3 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 10/04/2018 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 1.1c dmi.board.asset.tag: To be filled by O.E.M. dmi.board.name: H11DSU-iN dmi.board.vendor: Supermicro dmi.board.version: 1.02A dmi.chassis.asset.tag: To be filled by O.E.M. dmi.chassis.type: 1 dmi.chassis.vendor: Supermicro dmi.chassis.version: 0123456789 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1.1c:bd10/04/2018:svnSupermicro:pnAS-2023US-TR4:pvr0123456789:rvnSupermicro:rnH11DSU-iN:rvr1.02A:cvnSupermicro:ct1:cvr0123456789: dmi.product.family: To be filled by O.E.M. dmi.product.name: AS -2023US-TR4 dmi.product.version: 0123456789 dmi.sys.vendor: Supermicro ** Affects: linux (Ubuntu) Importance: Undecided Status: Confirmed ** Tags: amd64 apport-bug bionic uec-images -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1821345 Title: Mellanox MT27800 / mlx5_core : cannot bring up VFs if the total number of VFs is >= 64 : alloc irq vectors failed To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1821305] [NEW] csiostor and cxgb4 fight for the NIC
Public bug reported: Hi, I have a server with 2 * Chelsio T62100-LP-CR. Whenever the server is booted, the csiostor and cxgb4 modules will fight for the card. If csiostor wins, the following is logged : [9.331803] csiostor :21:00.6: PF: 6, Coming up as MASTER, HW state: Initializing ... [ 15.048308] cxgb4 :21:00.4: Coming up as SLAVE: Adapter already initialized and then the interface comes up a 40 Gbps. If cxgb4 wins, however, the following is logged : [8.960020] cxgb4 :21:00.4: Coming up as MASTER: Initializing adapter ... [ 12.141538] csiostor :21:00.6: PF: 6, Coming up as SLAVE, Master PF: 4, HW state: Initialized and the interface comes up at 100 Gbps as expected. Since I'm not using FCoE, I blacklisted the csiostor module, but these 2 modules should play nice with each other. Thanks ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-image-4.15.0-46-generic 4.15.0-46.49 ProcVersionSignature: User Name 4.15.0-46.49-generic 4.15.18 Uname: Linux 4.15.0-46-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Mar 21 15:42 seq crw-rw 1 root audio 116, 33 Mar 21 15:42 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu7.6 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Date: Fri Mar 22 09:05:38 2019 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' MachineType: Supermicro AS -2023US-TR4 PciMultimedia: ProcEnviron: TERM=screen-256color PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=C.UTF-8 SHELL=/bin/bash ProcFB: 0 astdrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.15.0-46-generic root=UUID=7a2aa06e-2ea8-451c-9b77-e0c5cdc445ce ro console=ttyS1,115200 nosplash iommu=pt amd_iommu=on processor.max_cstate=0 l1tf=full cpuidle.off=1 RelatedPackageVersions: linux-restricted-modules-4.15.0-46-generic N/A linux-backports-modules-4.15.0-46-generic N/A linux-firmware 1.173.3 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 10/04/2018 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 1.1c dmi.board.asset.tag: To be filled by O.E.M. dmi.board.name: H11DSU-iN dmi.board.vendor: Supermicro dmi.board.version: 1.02A dmi.chassis.asset.tag: To be filled by O.E.M. dmi.chassis.type: 1 dmi.chassis.vendor: Supermicro dmi.chassis.version: 0123456789 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1.1c:bd10/04/2018:svnSupermicro:pnAS-2023US-TR4:pvr0123456789:rvnSupermicro:rnH11DSU-iN:rvr1.02A:cvnSupermicro:ct1:cvr0123456789: dmi.product.family: To be filled by O.E.M. dmi.product.name: AS -2023US-TR4 dmi.product.version: 0123456789 dmi.sys.vendor: Supermicro ** Affects: linux (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1821305 Title: csiostor and cxgb4 fight for the NIC To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821305/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1821205] [NEW] "couldn't find IFLA_VF_INFO for VF 1 in netlink response" when trying to start libvirt VM with Chelsio VF
Public bug reported: Hi, I have a Chelsio T62100-LP-CR and I'm trying to create a libvirt VM that gets a VF from the NIC. The following XML is used : When starting the VM, libvirt fails with the following : ubuntu@lubbock:~$ sudo virsh start vmaxino error: Failed to start domain vmaxino error: internal error: couldn't find IFLA_VF_INFO for VF 1 in netlink response This is similar to bug 1496942. I can create the VM with the VF if I pass it as a like this : More info : ubuntu@lubbock:~$ sudo virsh nodedev-dumpxml pci__21_01_4 pci__21_01_4 /sys/devices/pci:20/:20:03.1/:21:01.4 pci__20_03_1 vfio-pci 0 33 1 4 T62100-LP-CR Unified Wire Ethernet Controller [VF] Chelsio Communications Inc pcap of netlink exchange is attached. ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-image-4.15.0-46-generic 4.15.0-46.49 ProcVersionSignature: User Name 4.15.0-46.49-generic 4.15.18 Uname: Linux 4.15.0-46-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Mar 21 15:42 seq crw-rw 1 root audio 116, 33 Mar 21 15:42 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu7.6 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Date: Thu Mar 21 16:16:41 2019 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' MachineType: Supermicro AS -2023US-TR4 PciMultimedia: ProcEnviron: TERM=screen-256color PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=C.UTF-8 SHELL=/bin/bash ProcFB: 0 astdrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.15.0-46-generic root=UUID=7a2aa06e-2ea8-451c-9b77-e0c5cdc445ce ro console=ttyS1,115200 nosplash iommu=pt amd_iommu=on processor.max_cstate=0 l1tf=full cpuidle.off=1 RelatedPackageVersions: linux-restricted-modules-4.15.0-46-generic N/A linux-backports-modules-4.15.0-46-generic N/A linux-firmware 1.173.3 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 10/04/2018 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 1.1c dmi.board.asset.tag: To be filled by O.E.M. dmi.board.name: H11DSU-iN dmi.board.vendor: Supermicro dmi.board.version: 1.02A dmi.chassis.asset.tag: To be filled by O.E.M. dmi.chassis.type: 1 dmi.chassis.vendor: Supermicro dmi.chassis.version: 0123456789 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1.1c:bd10/04/2018:svnSupermicro:pnAS-2023US-TR4:pvr0123456789:rvnSupermicro:rnH11DSU-iN:rvr1.02A:cvnSupermicro:ct1:cvr0123456789: dmi.product.family: To be filled by O.E.M. dmi.product.name: AS -2023US-TR4 dmi.product.version: 0123456789 dmi.sys.vendor: Supermicro ** Affects: linux (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug bionic ** Attachment added: "pcap of netlink exchange" https://bugs.launchpad.net/bugs/1821205/+attachment/5248235/+files/pcap.bz2 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1821205 Title: "couldn't find IFLA_VF_INFO for VF 1 in netlink response" when trying to start libvirt VM with Chelsio VF To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1821205/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812672] Re: impossible to create nova instances after upgrading to rocky
Oh wow ok pretty big changes. Thanks ! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812672 Title: impossible to create nova instances after upgrading to rocky To manage notifications about this bug go to: https://bugs.launchpad.net/charm-nova-cloud-controller/+bug/1812672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812672] Re: impossible to create nova instances after upgrading to rocky
Where's the commit ? I'm curious :) Thanks -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812672 Title: impossible to create nova instances after upgrading to rocky To manage notifications about this bug go to: https://bugs.launchpad.net/charm-nova-cloud-controller/+bug/1812672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1511840] Re: cannot apport-bug from IPv6-only network
Noted, I'll see what I can do - login.launchpad.net currently lives in an IPv4-only cloud. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1511840 Title: cannot apport-bug from IPv6-only network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1511840/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1686437] Re: [SRU] glance sync: need keystone v3 auth support
Version 0.1.0~bzr460-0ubuntu1 from bionic on fixes the problem, so I'm marking B/C/D as "Fix Released". ** Changed in: simplestreams (Ubuntu Bionic) Status: New => Fix Released ** Changed in: simplestreams (Ubuntu Cosmic) Status: New => Fix Released ** Changed in: simplestreams (Ubuntu Disco) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1686437 Title: [SRU] glance sync: need keystone v3 auth support To manage notifications about this bug go to: https://bugs.launchpad.net/simplestreams/+bug/1686437/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1771467] Re: Reboot/shutdown kernel panic on HP DL360/DL380 Gen9 w/ bionic 4.15.0
What's the status of the SRU for this bug ? Thanks ! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1771467 Title: Reboot/shutdown kernel panic on HP DL360/DL380 Gen9 w/ bionic 4.15.0 To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1771467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1686437] Re: [SRU] glance sync: need keystone v3 auth support
** Also affects: simplestreams (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: simplestreams (Ubuntu Cosmic) Importance: Undecided Status: New ** Also affects: simplestreams (Ubuntu Disco) Importance: Medium Status: Fix Released ** Changed in: simplestreams (Ubuntu Disco) Status: Fix Released => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1686437 Title: [SRU] glance sync: need keystone v3 auth support To manage notifications about this bug go to: https://bugs.launchpad.net/simplestreams/+bug/1686437/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1511840] Re: cannot apport-bug from IPv6-only network
This should now work. If it doesn't please let me know :) ** Changed in: apport (Ubuntu) Assignee: Canonical IS (canonical-is-public) => Junien Fridrick (axino) ** Changed in: apport (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1511840 Title: cannot apport-bug from IPv6-only network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1511840/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1260855] Re: http://start.ubuntu.com/connectivity-check.html and other ubiquity services are not accessible over IPv6 (no internet connectivity in ubiquity)
** Changed in: apport (Ubuntu) Status: Confirmed => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1260855 Title: http://start.ubuntu.com/connectivity-check.html and other ubiquity services are not accessible over IPv6 (no internet connectivity in ubiquity) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1260855/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1790904] Re: Glance v2 required by newer versions of OpenStack
Thanks for the correction :) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1790904 Title: Glance v2 required by newer versions of OpenStack To manage notifications about this bug go to: https://bugs.launchpad.net/simplestreams/+bug/1790904/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1790904] Re: Glance v2 required by newer versions of OpenStack
Hi, When will this be SRUed ? We'll also likely need it for xenial, since xenial-rocky is a thing as far as I know. Thanks -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1790904 Title: Glance v2 required by newer versions of OpenStack To manage notifications about this bug go to: https://bugs.launchpad.net/simplestreams/+bug/1790904/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1775732] Re: arm64 soft lock crashes on nova-compute charm running
4.19 appears to fix this problem -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1775732 Title: arm64 soft lock crashes on nova-compute charm running To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1775732/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1799737] Re: l3 agent external_network_bridge broken with ovs
The charm appears to do the right thing, it marks the feature as "deprecated" but properly configures it whenever asked. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1799737 Title: l3 agent external_network_bridge broken with ovs To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1799737/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1799737] Re: l3 agent external_network_bridge broken with ovs
This cloud is indeed upgraded from pike, and using juju charms - said charms still offer the option to set external_network_bridge, although the option is clearly marked as "deprecated". In my book, "deprecated" doesn't mean "completely broken" though... If I use "external_network_bridge = ", networking is likely to break since it's not configured elsewhere. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1799737 Title: l3 agent external_network_bridge broken with ovs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1799737/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1799737] [NEW] l3 agent external_network_bridge broken with ovs
Public bug reported: Hi, I'm running queens on xenial. The following commit introduced a regression : https://git.openstack.org/cgit/openstack/neutron/commit/?id=2b1d413ee90dfe2e9ae41c35ab37253df53fc6cd (fixing bug 1767422) The following call is wrong (in router_info.py) : self.driver.remove_vlan_tag(self.agent_conf.external_network_bridge, interface_name) In this call, interface_name is something like "qg-abcdefg-12", but remove_vlan_tag() expects a tap interface. This results in the following log : 2018-10-24 00:28:40.880 23623 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbClearCommand(column=tag, table=Port, record=qg-0410dbf1-51) do_commit /usr/lib/python2.7/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:84 2018-10-24 00:28:40.881 23623 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction aborted do_commit /usr/lib/python2.7/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:112 Sadly, the cause of the "Transaction aborted" is hidden (in https://github.com/openstack/ovsdbapp/blob/master/ovsdbapp/backend/ovs_idl/transaction.py#L87). If I print the exception, I get the following : 2018-10-24 00:28:40.881 23623 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] EXCEPTION Cannot find Port with name=qg-0410dbf1-51 do_commit /usr/lib/python2.7/dist- packages/ovsdbapp/backend/ovs_idl/transaction.py:88 Checking the ovs database reveals that Port "names" are the tap interfaces, not the qg- interfaces. The ports staying the VLAN 4095 will basically make the network unusable. Using the following call fixes my problem : self.driver.remove_vlan_tag(self.agent_conf.external_network_bridge, self.driver._get_tap_name(interface_name,prefix=EXTERNAL_DEV_PREFIX)) It would be nice to print the exception caught in https://github.com/openstack/ovsdbapp/blob/master/ovsdbapp/backend/ovs_idl/transaction.py#L87 by the way... Thanks ! For reference : $ dpkg -l|grep neutron ii neutron-common 2:12.0.3-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - common ii neutron-dhcp-agent 2:12.0.3-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - DHCP agent ii neutron-l3-agent 2:12.0.3-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - l3 agent ii neutron-lbaas-common 2:12.0.0-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - common ii neutron-lbaasv2-agent2:12.0.0-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - LBaaSv2 agent ii neutron-metadata-agent 2:12.0.3-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - metadata agent ii neutron-metering-agent 2:12.0.3-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - metering agent ii neutron-openvswitch-agent2:12.0.3-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - Open vSwitch plugin agent ii python-neutron 2:12.0.3-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - Python library ii python-neutron-fwaas 1:12.0.0-0ubuntu1~cloud0 all Firewall-as-a-Service driver for OpenStack Neutron ii python-neutron-lbaas 2:12.0.0-0ubuntu1~cloud0 all Loadbalancer-as-a-Service driver for OpenStack Neutron ii python-neutron-lib 1.13.0-0ubuntu1~cloud0 all Neutron shared routines and utilities - Python 2.7 ii python-neutronclient 1:6.7.0-0ubuntu1~cloud0 all client API library for Neutron - Python 2.7 ** Affects: neutron (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1799737 Title: l3 agent external_network_bridge broken with ovs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1799737/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] ProcModules.txt
apport information ** Attachment added: "ProcModules.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168126/+files/ProcModules.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] UdevDb.txt
apport information ** Attachment added: "UdevDb.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168128/+files/UdevDb.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] nvram.gz
apport information ** Attachment added: "nvram.gz" https://bugs.launchpad.net/bugs/1766543/+attachment/5168130/+files/nvram.gz ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] WifiSyslog.txt
apport information ** Attachment added: "WifiSyslog.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168129/+files/WifiSyslog.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] ProcPpc64.tar.gz
apport information ** Attachment added: "ProcPpc64.tar.gz" https://bugs.launchpad.net/bugs/1766543/+attachment/5168127/+files/ProcPpc64.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] ProcLocks.txt
apport information ** Attachment added: "ProcLocks.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168124/+files/ProcLocks.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] ProcMisc.txt
apport information ** Attachment added: "ProcMisc.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168125/+files/ProcMisc.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] OpalElog.tar.gz
apport information ** Attachment added: "OpalElog.tar.gz" https://bugs.launchpad.net/bugs/1766543/+attachment/5168121/+files/OpalElog.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] ProcCpuinfoMinimal.txt
apport information ** Attachment added: "ProcCpuinfoMinimal.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168122/+files/ProcCpuinfoMinimal.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] ProcInterrupts.txt
apport information ** Attachment added: "ProcInterrupts.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168123/+files/ProcInterrupts.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] Lspci.txt
apport information ** Attachment added: "Lspci.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168120/+files/Lspci.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] HookError_powerpc.txt
apport information ** Attachment added: "HookError_powerpc.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168119/+files/HookError_powerpc.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] CurrentDmesg.txt
apport information ** Attachment added: "CurrentDmesg.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168116/+files/CurrentDmesg.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] HookError_generic.txt
apport information ** Attachment added: "HookError_generic.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168118/+files/HookError_generic.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] DeviceTree.tar.gz
apport information ** Attachment added: "DeviceTree.tar.gz" https://bugs.launchpad.net/bugs/1766543/+attachment/5168117/+files/DeviceTree.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] CRDA.txt
apport information ** Attachment added: "CRDA.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168115/+files/CRDA.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] Re: instance deletion takes a while and blocks nova-compute
apport information ** Tags added: apport-collected uec-images xenial ** Description changed: Hi, I have a cloud running xenial/mitaka (with 18.02 charms). Sometimes, an instance will take minutes to delete. I tracked down the time taken to be file deletion : Apr 23 07:23:00 hostname nova-compute[54255]: 2018-04-23 07:23:00.920 54255 INFO nova.virt.libvirt.driver [req- 35ccfe64-9280-4de6-ae88-045ca91bf90f bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] [instance: 97731f51-63be-4056 -869f-084b38580b9a] Deleting instance files /srv/nova/instances/97731f51-63be-4056-869f-084b38580b9a_del Apr 23 07:27:33 hostname nova-compute[54255]: 2018-04-23 07:27:33.767 54255 INFO nova.virt.libvirt.driver [req- 35ccfe64-9280-4de6-ae88-045ca91bf90f bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] [instance: 97731f51-63be-4056 -869f-084b38580b9a] Deletion of /srv/nova/instances/97731f51-63be-4056 -869f-084b38580b9a_del complete As you can see, 4 minutes and 33 seconds have elapsed between the 2 lines. nova-compute logs absolutely _nothing_ during this time. Periodic tasks are not run, etc... Generally, a deletion takes a few seconds top. The logs above are generally immediately followed by : Apr 23 07:27:33 hostname nova-compute[54255]: 2018-04-23 07:27:33.771 54255 DEBUG oslo.messaging._drivers.impl_rabbit [req- 35ccfe64-9280-4de6-ae88-045ca91bf90f bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] Received recoverable error from kombu: on_error /usr/lib/python2.7/dist- packages/oslo_messaging/_drivers/impl_rabbit.py:683 (which is error: [Errno 104] Connection reset by peer) because nova-compute doesn't even maintain the rabbitmq connection (on the rabbitmq server I can see errors about "Missed heartbeats from client, timeout: 60s"). So nova-compute appears to be "frozen" during several minutes. This can cause problems because events can be missed, etc... We have telegraf on this host, and there's little to no CPU, disk, network or memory activity at that time. Nothing relevant in kern.log either. And this is happening on 3 different architectures, so this is all very puzzling. Is nova-compute supposed to be totally stuck while deleting instance files ? Have you ever seen something similar ? I'm going to try to repro on queens. Thanks + --- + AlsaDevices: + total 0 + crw-rw 1 root audio 116, 1 Jun 23 14:23 seq + crw-rw 1 root audio 116, 33 Jun 23 14:23 timer + AplayDevices: Error: [Errno 2] No such file or directory + ApportVersion: 2.20.1-0ubuntu2.18 + Architecture: ppc64el + ArecordDevices: Error: [Errno 2] No such file or directory + AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: + DistroRelease: Ubuntu 16.04 + IwConfig: Error: [Errno 2] No such file or directory + Lsusb: + Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub + Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Package: nova (not installed) + PciMultimedia: + + ProcEnviron: + TERM=screen-256color + PATH=(custom, no user) + LANG=en_US.UTF-8 + SHELL=/bin/bash + ProcFB: + + ProcKernelCmdLine: root=UUID=bf574d52-09d7-4c9d-beb8-ccc542d50654 ro console=tty0 console=ttyS1,115200 panic=30 raid=noautodetect + ProcLoadAvg: 10.43 10.79 9.76 11/2237 15123 + ProcSwaps: + Filename TypeSizeUsedPriority + /swap.img file 8388544 0 -1 + ProcVersion: Linux version 4.4.0-128-generic (buildd@bos02-ppc64el-001) (gcc version 5.4.0 20160609 (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.9) ) #154-Ubuntu SMP Fri May 25 14:13:59 UTC 2018 + ProcVersionSignature: Ubuntu 4.4.0-128.154-generic 4.4.131 + RelatedPackageVersions: + linux-restricted-modules-4.4.0-128-generic N/A + linux-backports-modules-4.4.0-128-generic N/A + linux-firmware 1.157.20 + RfKill: Error: [Errno 2] No such file or directory + Tags: xenial uec-images xenial uec-images xenial uec-images + Uname: Linux 4.4.0-128-generic ppc64le + UnreportableReason: The report belongs to a package that is not installed. + UpgradeStatus: No upgrade log present (probably fresh install) + UserGroups: + + _MarkForUpload: False + cpu_cores: Number of cores present = 20 + cpu_coreson: Number of cores online = 20 + cpu_dscr: DSCR is 0 + cpu_freq: + min: 1.201 GHz (cpu 48) + max: 3.710 GHz (cpu 112) + avg: 2.615 GHz + cpu_runmode: + Could not retrieve current diagnostics mode, + No kernel interface to firmware + cpu_smt: SMT is off ** Attachment added: ".sys.firmware.opal.msglog.txt" https://bugs.launchpad.net/bugs/1766543/+attachment/5168114/+files/.sys.firmware.opal.msglog.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.
[Bug 1766543] Re: instance deletion takes a while and blocks nova-compute
By the way the task blocks on the following : ==> /proc/54255/task/54255/stack <== [] wakeup_preempt_entity.isra.6+0x7c/0x90 [] __switch_to+0x1f8/0x350 [] get_request+0x29c/0x910 [] blk_queue_bio+0x164/0x500 [] generic_make_request+0x154/0x310 [] submit_bio+0xd4/0x1f0 [] ext4_io_submit+0x7c/0xb0 [] ext4_writepages+0x4a8/0xdd0 [] do_writepages+0x60/0xc0 [] __filemap_fdatawrite_range+0xf8/0x170 [] jbd2_journal_begin_ordered_truncate+0xe8/0x130 [] ext4_evict_inode+0x530/0x5e0 [] evict+0xf8/0x2a0 [] do_unlinkat+0x1a8/0x3a0 [] system_call+0x38/0xe4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1555562] Re: lastpass-cli changed bundled CA certificates
-proposed package works for me and resolves the CA issue. ** Tags removed: verification-needed verification-needed-bionic ** Tags added: verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/162 Title: lastpass-cli changed bundled CA certificates To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/lastpass-cli/+bug/162/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] Re: instance deletion takes a while and blocks nova-compute
OK, the bug happened again with strace attached to nova-compute. Once again, there's little to no IO/network while it happens. memory is stable. CPU is at least 50% idle (and the rest of it largely user mode). Nothing in dmesg. nova-compute logs are as follow : 2018-04-25 14:48:04.587 54255 INFO nova.virt.libvirt.driver [req-85551d96-713d-499d-b7ff-9f911fb0842d bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] [instance: bd17aeef-240b-489c-8bb6-b37167155174] Deleting instance files /srv/nova/instances/bd17aeef-240b-489c-8bb6-b37167155174_del 2018-04-25 14:52:06.350 54255 INFO nova.virt.libvirt.driver [req- 85551d96-713d-499d-b7ff-9f911fb0842d bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] [instance: bd17aeef-240b-489c- 8bb6-b37167155174] Deletion of /srv/nova/instances/bd17aeef-240b-489c- 8bb6-b37167155174_del complete So it took 4 minutes to remove the files. Looking at strace, it's mostly a single thread doing (unrelated) stuff in the meantime : https://pastebin.canonical.com/p/w63Z62r4zN/ (sorry, Canonical-only link). The first line is the unlink, and you can see the syscall finishing at the bottom. Looking at this, I'm fairly convinced that it's the unlink() itself that took 4 minutes. So perhaps a kernel bug ? I'm now running "perf record sleep 60" in a loop to try and diagnose what the hell the system is doing during that time. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] Re: instance deletion takes a while and blocks nova-compute
Hi Corey, * I assume nova-compute is up the whole time because its PID doesn't change * I'm running with debug on * I'm going to try getting more details with strace, but the problem is I can't repro the problem - it just sometimes happen... No luck on repro-ing on queens so far, but the cloud is much, much less used. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] Re: instance deletion takes a while and blocks nova-compute
Also, nova-scheduler or nova-api-os-compute will log the following lines (a few times per minute) while this is happening : Apr 23 07:24:47 juju-8c74e6-4-lxd-7 nova-scheduler[15786]: 2018-04-23 07:24:47.785 15786 DEBUG nova.servicegroup.drivers.db [req- 1573c400-116c-4825-b108-3291a014b0e9 bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] Seems service nova-compute on host hostname is down. Last heartbeat was 2018-04-23 07:22:56. Elapsed time is 111.785844 is_up /usr/lib/python2.7/dist- packages/nova/servicegroup/drivers/db.py:82 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1766543] [NEW] instance deletion takes a while and blocks nova-compute
Public bug reported: Hi, I have a cloud running xenial/mitaka (with 18.02 charms). Sometimes, an instance will take minutes to delete. I tracked down the time taken to be file deletion : Apr 23 07:23:00 hostname nova-compute[54255]: 2018-04-23 07:23:00.920 54255 INFO nova.virt.libvirt.driver [req- 35ccfe64-9280-4de6-ae88-045ca91bf90f bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] [instance: 97731f51-63be-4056 -869f-084b38580b9a] Deleting instance files /srv/nova/instances/97731f51-63be-4056-869f-084b38580b9a_del Apr 23 07:27:33 hostname nova-compute[54255]: 2018-04-23 07:27:33.767 54255 INFO nova.virt.libvirt.driver [req- 35ccfe64-9280-4de6-ae88-045ca91bf90f bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] [instance: 97731f51-63be-4056 -869f-084b38580b9a] Deletion of /srv/nova/instances/97731f51-63be-4056 -869f-084b38580b9a_del complete As you can see, 4 minutes and 33 seconds have elapsed between the 2 lines. nova-compute logs absolutely _nothing_ during this time. Periodic tasks are not run, etc... Generally, a deletion takes a few seconds top. The logs above are generally immediately followed by : Apr 23 07:27:33 hostname nova-compute[54255]: 2018-04-23 07:27:33.771 54255 DEBUG oslo.messaging._drivers.impl_rabbit [req- 35ccfe64-9280-4de6-ae88-045ca91bf90f bc0ab055427645aca4ed09266e85b1db 1cb457a8302543fea067e5f14b5241e7 - - -] Received recoverable error from kombu: on_error /usr/lib/python2.7/dist- packages/oslo_messaging/_drivers/impl_rabbit.py:683 (which is error: [Errno 104] Connection reset by peer) because nova-compute doesn't even maintain the rabbitmq connection (on the rabbitmq server I can see errors about "Missed heartbeats from client, timeout: 60s"). So nova-compute appears to be "frozen" during several minutes. This can cause problems because events can be missed, etc... We have telegraf on this host, and there's little to no CPU, disk, network or memory activity at that time. Nothing relevant in kern.log either. And this is happening on 3 different architectures, so this is all very puzzling. Is nova-compute supposed to be totally stuck while deleting instance files ? Have you ever seen something similar ? I'm going to try to repro on queens. Thanks ** Affects: nova (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1766543 Title: instance deletion takes a while and blocks nova-compute To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1766543/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1611987] Re: glance-simplestreams-sync charm doesn't support keystone v3
I can confirm that the workaround proposed in #13 works. Small twist though : updating the "source" of an already installed glance- simplestream-sync application won't do anything, since the "source" option is only used at install time. You'll want to manually add the PPA and upgrade the python-simplestreams package. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1611987 Title: glance-simplestreams-sync charm doesn't support keystone v3 To manage notifications about this bug go to: https://bugs.launchpad.net/charm-glance-simplestreams-sync/+bug/1611987/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1754015] Re: [SRU] nova-compute-kvm does not pull ipxe-qemu on non-amd64 archs
Testing done : 1. create new xenial LXD 2. sudo add-apt-repository cloud-archive:pike-proposed ; sudo apt-get update 3. echo 'APT::Install-Recommends "false";' | sudo tee /etc/apt/apt.conf.d/99no-recommends 4. sudo apt-get install nova-compute-kvm 5. verify ipxe-qemu is installed Testing successful ! ** Tags removed: verification-pike-needed ** Tags added: verification-pike-done -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1754015 Title: [SRU] nova-compute-kvm does not pull ipxe-qemu on non-amd64 archs To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1754015/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1763442] Re: instance creation fails with "Failed to allocate the network(s), not rescheduling." because neutron-ovs-agent rpc_loop took too long
** Also affects: neutron (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1763442 Title: instance creation fails with "Failed to allocate the network(s), not rescheduling." because neutron-ovs-agent rpc_loop took too long To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1763442/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1715609] Re: kernel warning: skb_warn_bad_offload
Hi, I got this on 4.4.0-116-generic : [266447.178906] [ cut here ] [266447.178919] WARNING: CPU: 0 PID: 3343 at /build/linux-fQ94TU/linux-4.4.0/net/core/dev.c:2448 skb_warn_bad_offload+0xd1/0x120() [266447.178924] qr-8a4bd2f3-04: caps=(0x0184075b59e9, 0x0184075b59e9) len=2058 data_len=582 gso_size=1406 gso_type=1 ip_summed=3 [266447.178926] Modules linked in: kpatch_livepatch_Ubuntu_4_4_0_116_140_generic_36(OE) xt_REDIRECT nf_nat_redirect ip6table_mangle xt_nat xt_mark xt_connmark ip6table_raw iptable_raw ip_gre ip_tunnel gre xt_recent ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 xt_hl ip6t_rt nf_conntrack_ipv6 ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_addrtype xt_conntrack dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag veth netlink_diag nbd ip6table_filter ip6_tables xt_CHECKSUM iptable_mangle xt_tcpudp ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_filter ip_tables x_tables openvswitch nf_defrag_ipv6 nf_conntrack binfmt_misc ipmi_ssif intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel bridge stp kvm llc [266447.178988] irqbypass sb_edac edac_core ipmi_si hpilo ipmi_msghandler serio_raw 8250_fintek lpc_ich shpchp acpi_power_meter ioatdma mac_hid dca ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse tg3 ptp pps_core hpsa scsi_transport_sas wmi fjes [266447.179045] CPU: 0 PID: 3343 Comm: haproxy Tainted: GW OE K 4.4.0-116-generic #140-Ubuntu [266447.179048] Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 01/22/2018 [266447.179050] 0286 6f5737dc927d16dd 880a3646b818 813ffc13 [266447.179055] 880a3646b860 81d715e8 880a3646b850 810830f2 [266447.179059] 880064507ce8 880b4ff12000 0001 8808723268fc [266447.179064] Call Trace: [266447.179072] [] dump_stack+0x63/0x90 [266447.179079] [] warn_slowpath_common+0x82/0xc0 [266447.179084] [] warn_slowpath_fmt+0x5c/0x80 [266447.179090] [] ? ___ratelimit+0xa2/0xe0 [266447.179094] [] skb_warn_bad_offload+0xd1/0x120 [266447.179100] [] skb_checksum_help+0x185/0x1a0 [266447.179106] [] checksum_tg+0x22/0x29 [xt_CHECKSUM] [266447.179112] [] ipt_do_table+0x301/0x720 [ip_tables] [266447.179117] [] ? ipt_do_table+0x349/0x720 [ip_tables] [266447.179122] [] ? enqueue_entity+0x3a7/0xd20 [266447.179129] [] iptable_mangle_hook+0x39/0x107 [iptable_mangle] [266447.179134] [] nf_iterate+0x68/0x80 [266447.179137] [] nf_hook_slow+0x73/0xd0 [266447.179143] [] ip_output+0xcf/0xe0 [266447.179148] [] ? __ip_flush_pending_frames.isra.42+0x90/0x90 [266447.179153] [] ip_local_out+0x3b/0x50 [266447.179158] [] ip_queue_xmit+0x154/0x380 [266447.179164] [] tcp_transmit_skb+0x51b/0x980 [266447.179169] [] tcp_write_xmit+0x1d6/0xf40 [266447.179174] [] __tcp_push_pending_frames+0x31/0xd0 [266447.179178] [] tcp_push+0xec/0x110 [266447.179181] [] tcp_sendmsg+0x739/0xb80 [266447.179186] [] inet_sendmsg+0x6b/0xa0 [266447.179192] [] sock_sendmsg+0x3e/0x50 [266447.179196] [] SYSC_sendto+0x101/0x190 [266447.179203] [] ? syscall_trace_enter_phase1+0xc8/0x140 [266447.179208] [] SyS_sendto+0xe/0x10 [266447.179215] [] entry_SYSCALL_64_fastpath+0x1c/0xbb [266447.179218] ---[ end trace 4c054fa8d617289c ]--- Joseph, can we maybe get a new kernel to test ? The one in your last comment is gone. Note that this isn't appearing a lot (~10 occurrences on a server with 3 days of uptime) ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed ** Changed in: linux (Ubuntu Xenial) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1715609 Title: kernel warning: skb_warn_bad_offload To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1715609/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1758868] Re: ovs restart can lead to critical ovs flows missing
I'm fairly confident this is not the ovs-cleanup bug. "grep neutron-ovs- cleanup syslog" doesn't have anything, I'm using neutron-plugin- openvswitch-agent 2:8.4.0-0ubuntu6, and neutron-ovs-cleanup.service is "Active: active (exited) since Mon 2018-02-19 05:09:12 UTC; 1 months 4 days ago" so it didn't run in March -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1758868 Title: ovs restart can lead to critical ovs flows missing To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1758868/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1758868] [NEW] ovs restart can lead to critical ovs flows missing
Public bug reported:
Hi,
Running mitaka on xenial (neutron 2:8.4.0-0ubuntu6). We have l2pop and
no l3ha. Using ovs with GRE tunnels.
The cloud has around 30 compute nodes (mostly arm64). Last week, ovs got
restarted during a package upgrade :
2018-03-21 17:17:25 upgrade openvswitch-common:arm64
2.5.2-0ubuntu0.16.04.3 2.5.4-0ubuntu0.16.04.1
This led to instances on 2 arm64 compute nodes lose networking
completely. Upon closer inspection, I realized that a flow was missing
in br-tun table 3 : https://pastebin.ubuntu.com/p/VXRJJX8J3k/
I believe this is due to a race in ovs_neutron_agent.py. These flows in
table 3 are set up in provision_local_vlan() :
https://github.com/openstack/neutron/blob/mitaka-
eol/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L675
which is called by port_bound() :
https://github.com/openstack/neutron/blob/mitaka-eol/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L789-L791
which is called by treat_vif_port() :
https://github.com/openstack/neutron/blob/mitaka-
eol/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1405-L1410
which is called by treat_devices_added_or_updated() :
https://github.com/openstack/neutron/blob/mitaka-
eol/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1517-L1525
which is called by process_network_ports() :
https://github.com/openstack/neutron/blob/mitaka-
eol/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1618-L1623
which is called by the big rpc_loop() :
https://github.com/openstack/neutron/blob/mitaka-
eol/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L2023-L2029
So how does the agent knows when to create these table 3 flows ? Well, in
rpc_loop(), it checks for OVS restarts
(https://github.com/openstack/neutron/blob/mitaka-eol/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1947-L1948),
and if OVS did restart, it does some basic ovs setup (default flows, etc), and
(very important for later), it restarts the OVS polling manager.
Later (still in rpc_loop()), it sets "ovs_restarted" to True, and
process the ports as usual. The expected behaviour here is that since
the polling manager got restarted, any port up will be marked as "added"
and processed as such, in port_bound() (see call stack above). If this
function is called on a port when ovs_restarted is True, then
provision_local_vlan() will get called and will able the table 3 flows.
This is all working great under the assumption that the polling manager
(which is an async process) will raise the "I got new port !" event
before the rpc_loop() checks it (in process_port_events(), called by
process_port_info()). However, if for example the node is under load,
this may not always be the case.
What happens then is that the rpc_loop in which OVS is detected as
restarted doesn't see any change on the ports, and so does nothing. The
next run of the rpc_loop will process the "I got new port !" events, but
that loop will not be running with ovs_restarted set to True, so the
ports won't be brought up properly - more specifically, the table 3
flows in br-tun will be missing. This is shown in the debug logs :
https://pastebin.ubuntu.com/p/M8yYn3YnQ6/ - you can see the loop in
which "OVS is restarted" is detected (loop iteration 320773) doesn't
process any port ("iteration:320773 completed. Processed ports
statistics: {'regular': {'updated': 0, 'added': 0, 'removed': 0}}.), but
the next iteration does process 3 "added" ports. You can see that the
"output received" is logged in the first loop, 49ms after "starting
polling" is logged, which is presumably the problem. On all the non-
failing nodes, the output is received before "starting polling".
I believe the proper thing to do is to set "sync" to True (in
rpc_loop()) if an ovs restart is detected, forcing process_port_info()
to not use async events and scan the ports itself using scan_ports().
Thanks
** Affects: neutron (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1758868
Title:
ovs restart can lead to critical ovs flows missing
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1758868/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1758868] Re: ovs restart can lead to critical ovs flows missing
This problem can probably be easily replicated if you replace /usr/bin /ovsdb-client (used by the polling manager) by a shell script that sleeps 0.5s and then exec the actual ovsdb-client. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1758868 Title: ovs restart can lead to critical ovs flows missing To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1758868/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1754015] Re: [SRU] nova-compute-kvm does not pull ipxe-qemu on non-amd64 archs
I don't have an artful around to test, FWIW. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1754015 Title: [SRU] nova-compute-kvm does not pull ipxe-qemu on non-amd64 archs To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1754015/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
