Reviewed: https://review.openstack.org/623066 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f2111e035424bf714099966ad724e9a4bd604c18 Submitter: Zuul Branch: master
commit f2111e035424bf714099966ad724e9a4bd604c18 Author: Arjun Baindur <xag...@gmail.com> Date: Wed Dec 5 12:43:05 2018 -0800 Do not release DHCP lease when no client ID is set on port The DHCP agent has a really strict enforcement of client ID, which is part of the DHCP extra options. If a VM advertises a client ID, DHCP agent will automatically release it's lease whenever *any* other port is updated/deleted, even if no client ID is set on the port, because it thinks the client ID has changed. When reload_allocations() is called, the DHCP agent parses the leases and hosts files, and gets the list of all the ports in the network from the DB, computing 3 different sets. The set from the leases file (v4_leases) could have a client ID, but the set from the port DB and hosts file will have None. As a result, the set subtraction does not filter out the entry, and all ports that have an active lease with a client ID are released. The Client ID should only be enforced and leases released if it's actually set in the port DB's DHCP extra Opts. In that case it means someone knows what they are doing, and we want to check for a mismatch. If the client ID on a port is empty, it should not be treated like an unused lease. We can't expect end users that just create VMs with auto created ports to know/care about DHCP client IDs, then manually update ports or change app templates. In some cases, like Windows VMs, the client ID is advertised as the MAC by default. In fact, there is a Windows bug which prevents you from even turning this off: https://support.microsoft.com/en-us/help/3004537/dhcp-client-always-includes-option-61-in-the-dhcp-request-in-windows-8 Linux VMs don't have this on by default, but it may be enabled in some templates unknown to users. Change-Id: I8021f740bd78e654915337bd3287b45b2c422e95 Closes-Bug: #1806770 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1806770 Title: DHCP Agent should not release DHCP lease when client ID is not set on port Status in neutron: Fix Released Bug description: DHCP agent has a really strict enforcement of client ID, which is part of the DHCP extra options. If a VM advertises a client ID, DHCP agent will automatically release it's lease whenever *any* other port is updated/deleted. This happens even if no client ID is set on the port. When reload_allocations() is called, DHCP agent parses the current leases file, the hosts file, and gets the list all the ports in the network from DB, computing 3 different sets. The set from leases file (v4_leases) will have some client ID. The set from port DB will have None. As a result the set subtraction does not filter out the entry, and the port's DHCP lease is constantly released, whenever the VM renews its lease and any other port in the network is deleted: https://github.com/openstack/neutron/blob/stable/pike/neutron/agent/linux/dhcp.py#L850 v4_leases = set() for (k, v) in cur_leases.items(): # IPv4 leases have a MAC, IPv6 ones do not, so we must ignore if netaddr.IPAddress(k).version == constants.IP_VERSION_4: # treat '*' as None, see note in _read_leases_file_leases() client_id = v['client_id'] if client_id is '*': client_id = None v4_leases.add((k, v['iaid'], client_id)) new_leases = set() for port in self.network.ports: client_id = self._get_client_id(port) for alloc in port.fixed_ips: new_leases.add((alloc.ip_address, port.mac_address, client_id)) # If an entry is in the leases or host file(s), but doesn't have # a fixed IP on a corresponding neutron port, consider it stale. entries_to_release = (v4_leases | old_leases) - new_leases if not entries_to_release: return It was observed in one example of a released lease, its entries looked like: new_leases (from port DB) (u'10.81.96.186', u'fa:16:3e:eb:a1:13', None) old_leases (from hosts file) ('10.81.96.186', 'fa:16:3e:eb:a1:13', None) v4_leases (from leases file - updated by dnsmasq when VM requests) ('10.81.96.186', 'fa:16:3e:eb:a1:13', '01:fa:16:3e:eb:a1:13') Therefore the entries_to_release did not have that IP, MAC filtered out. The client_id in v4_leases entry was coming from a Windows VM, which faces a bug that prevents it from disabling client ID. entries_to_release in fact had some 50+ entries like that, causing a storm of DHCPRELEASE. This can cause issues where when the VM later asks to renew its lease when the expiry period is coming up (I think about halfway thru), dnsmasq sends an DHCP NAK and the lease is re-negotiated and existing networking connections can get disrupted. It also just causes DHCP agent to do unneccessary work, releasing a ton of leases when it technically shouldn't. Setting the client ID in the port's DHCP extra opts is not an good solution: 1. In some cases, like Windows VMs, the client ID is advertised as the MAC by default. In fact, there is a Windows bug which prevents you from even turning this off: https://support.microsoft.com/en- us/help/3004537/dhcp-client-always-includes-option-61-in-the-dhcp- request-in-windows-8 Linux VMs dont have this on by default, when I checked, but they may be enabled in some templates unknown to users 2. End users will usually just be deploying a VM, with the port being auto created by Nova. They don't know or need to know about advanced networking concepts like DHCP client IDs. 3. We can't expect everyone to modify their existing app templates, or end users to make API calls, to update ports everytime they deploy a VM So, client ID should only be enforced, and leases released, if it's actually set on the port DB's DHCP extra Opts. In that case it means someone knows what they are doing, and we want to check for a mismatch. If its None, I suspect in 99.9999% of cases the operator does not know or care about client ID field. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1806770/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp