I've switched this bug to Public and marked it as security affecting. It sounds like a serious defect with security implications, and might warrant a security note if it turns out to be unfixable in supported branches for some reason, but I don't think we would issue an advisory about it (class D in our report taxonomy). https://security.openstack.org/vmt-process.html#incident-report-taxonomy
** Description changed: - This issue is being treated as a potential security risk under - embargo. Please do not make any public mention of embargoed - (private) security vulnerabilities before their coordinated - publication by the OpenStack Vulnerability Management Team in the - form of an official OpenStack Security Advisory. This includes - discussion of the bug or associated fixes in public forums such as - mailing lists, code review systems and bug trackers. Please also - avoid private disclosure to other individuals not already approved - for access to this information, and provide this same reminder to - those who are made aware of the issue prior to publication. All - discussion should remain confined to this private bug report, and - any proposed fixes should be added to the bug as attachments. This - embargo shall not extend past 2021-02-07 and will be made - public by or on that date even if no fix is identified. - During the update of Neutron from 15.1 to 15.3 (Train) server and agent are not compatible (RPC format) anymore. I was able to narrow down the issue to commit https://opendev.org/openstack/neutron/commit/00298fe6e84cd7610b39af50e9517885a182f47c or patchset https://review.opendev.org/#/c/744133/ With this commit, fixing the security issue in bug https://bugs.launchpad.net/neutron/+bug/1867119 was addressed and a change to the RPC data structure now containing also the MAC address was required consequently back-ported to Train (https://bugs.launchpad.net/neutron/+bug/1867119/comments/6). This change breaks compatibility as the loop in method "_sanitize_addresses" (neutron/agent/linux/ipset_manager.py", line 47) does not handle this new format properly. --- 15.1 --- for ip in addresses: ip = netaddr.IPNetwork(ip) if ip.prefixlen == 0: --- /15.1 --- --- 15.3 --- for ip, _mac in addresses: ip = netaddr.IPNetwork(ip) if ip.prefixlen == 0: --- /15.3 --- As neutron server in version 15.1 delivers just a list of addresses, i,e. "['192.168.100.57', '192.168.100.160']", while after the update to 15.3 it sends a list of lists with each list element containing an IP as well a MAC address, i.e. "[['192.168.100.57', None], ['192.168.100.160', None]])" the code on the agent side has no way of handling both cases. According to https://docs.openstack.org/neutron/latest/contributor/internals/upgrade.html .... which states: " Those requirements are achieved in Neutron by: If the Neutron backend makes use of Neutron agents, the Neutron server have backwards compatibility code to deal with older messaging payloads. isolating a single service that accesses database (neutron-server). To simplify the matter, it’s always assumed that the order of service upgrades is as following: first, all neutron-servers are upgraded. then, if applicable, neutron agents are upgraded. This approach allows us to avoid backwards compatibility code on agent side and is in line with other OpenStack projects that support rolling upgrades (specifically, nova). " an upgraded neutron-server should still work with the previous neutron- agent. I took the liberty to flag this as "security vulnerability" as security groups will likely not be applied / managed properly when running mixed between 15.1 and 15.3 which might be a common case in larger clusters. ** Information type changed from Private Security to Public Security ** Information type changed from Public Security to Public ** Changed in: ossa Status: Incomplete => Won't Fix ** Tags added: security -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1903531 Title: Update of neutron-server breaks compatibility to previous neutron- agent version Status in neutron: Confirmed Status in OpenStack Security Advisory: Won't Fix Bug description: During the update of Neutron from 15.1 to 15.3 (Train) server and agent are not compatible (RPC format) anymore. I was able to narrow down the issue to commit https://opendev.org/openstack/neutron/commit/00298fe6e84cd7610b39af50e9517885a182f47c or patchset https://review.opendev.org/#/c/744133/ With this commit, fixing the security issue in bug https://bugs.launchpad.net/neutron/+bug/1867119 was addressed and a change to the RPC data structure now containing also the MAC address was required consequently back-ported to Train (https://bugs.launchpad.net/neutron/+bug/1867119/comments/6). This change breaks compatibility as the loop in method "_sanitize_addresses" (neutron/agent/linux/ipset_manager.py", line 47) does not handle this new format properly. --- 15.1 --- for ip in addresses: ip = netaddr.IPNetwork(ip) if ip.prefixlen == 0: --- /15.1 --- --- 15.3 --- for ip, _mac in addresses: ip = netaddr.IPNetwork(ip) if ip.prefixlen == 0: --- /15.3 --- As neutron server in version 15.1 delivers just a list of addresses, i,e. "['192.168.100.57', '192.168.100.160']", while after the update to 15.3 it sends a list of lists with each list element containing an IP as well a MAC address, i.e. "[['192.168.100.57', None], ['192.168.100.160', None]])" the code on the agent side has no way of handling both cases. According to https://docs.openstack.org/neutron/latest/contributor/internals/upgrade.html .... which states: " Those requirements are achieved in Neutron by: If the Neutron backend makes use of Neutron agents, the Neutron server have backwards compatibility code to deal with older messaging payloads. isolating a single service that accesses database (neutron- server). To simplify the matter, it’s always assumed that the order of service upgrades is as following: first, all neutron-servers are upgraded. then, if applicable, neutron agents are upgraded. This approach allows us to avoid backwards compatibility code on agent side and is in line with other OpenStack projects that support rolling upgrades (specifically, nova). " an upgraded neutron-server should still work with the previous neutron-agent. I took the liberty to flag this as "security vulnerability" as security groups will likely not be applied / managed properly when running mixed between 15.1 and 15.3 which might be a common case in larger clusters. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1903531/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
