Public bug reported: Enable bringup of subports via exposing trunk/subport details over the metadata API
With the completion of the trunk port feature in Newton (Neutron bp/vlan-aware-vms [1]), trunk and subports are now available. But the bringup of the subports' VLAN interfaces inside an instance is not automatic. In Newton there's no easy way to pass information about the subports to the guest operating system. But using the metadata API we can change this. Problem Description ------------------- To bring up (and/or tear down) a subport the guest OS (a) must know the segmentation-type and segmentation-id of a subport as set in 'openstack network trunk create/set --subport' (b) must know the MAC address of a subport as set in 'openstack port create' (c) must know which vNIC the subport belongs to (d) may need to know when were subports added or removed (if they are added or removed during the lifetime of an instance) Since subports do not have a corresponding vNIC, the approach used for regular ports (with a vNIC) cannot work. This write-up addresses problems (a), (b) and (c), but not (d). Proposed Change --------------- Here we propose a change involving both Nova and Neutron to expose the information needed via the metadata API. Information covering (a) and (b) is already available (read-only) in the 'trunk_details' attribute of the trunk parent port (ie. the port which the instance was booted with). [2] We propose to use the MAC address of the trunk parent port to cover (c). We recognize this may occasionally be problematic, because MAC addresses (of ports belonging to different neutron networks) are not guaranteed to be unique, therefore collision may happen. But this seems to be a small price for avoiding the complexity of other solutions. The mechanism would be the following. Let's suppose we have port0 which is a trunk parent port and instance0 was booted with '--nic port-id=port0'. On every update of port0's trunk_details Neutron constructs the following JSON structure: PORT0-DETAILS = { "mac_address": PORT0-MAC-ADDRESS, "trunk_details": PORT0-TRUNK-DETAILS } Then Neutron sets a metadata key-value pair of instance0, equivalent to the following nova command: nova meta set instance0 trunk_details::PORT0-MAC-ADDRESS=PORT0-DETAILS Nova in Newton limits meta values to <= 255 characters, this limit must be raised. Assuming the current format of trunk_details roughly 150 characters/subport are needed. Alternatively meta values could have unlimited length - at least for the service tenant used by Neutron. (Though tenant-specific API validators may not be a good idea.) The 'values' column of the the 'instance_metadata' table should be altered from VARCHAR(255) to TEXT() in a Nova DB migration. (A slightly related bug report: [3]) A program could read http://169.254.169.254/openstack/2016-06-30/meta_data.json and bring up the subport VLAN interfaces accordingly. This program is not covered here, however it is worth pointing out that it could be called by cloud-init. Alternatives ------------ (1) The MAC address of a parent port can be reused for all its child ports (when creating the child ports). Then VLAN subinterfaces of a network interface will have the correct MAC address by default. Segmentation type and ID can be shared in other ways, for example as a VLAN plan embedded into a golden image. This approach could even partially solve problem (d), however it cannot solve problem (a) in the dynamic case. Use of this approach is currently blocked by an openvswitch firewall driver bug. [4][5] (2) Generate and inject a subport bringup script into the instance as user data. Cannot handle subports added or removed after VM boot. (3) An alternative solution to problem (c) could rely on the preservation of ordering between NICs passed to nova boot and NICs inside an instance. However this would turn the update of trunk_details into an instance-level operation instead of the port-level operation proposed here. Plus it would fail if this ordering is ever lost. References ---------- [1] https://blueprints.launchpad.net/neutron/+spec/vlan-aware-vms [2] https://review.openstack.org/#q,Id23ce8fc16c6ea6a405cb8febf8470a5bf3bcb43,n,z [3] https://bugs.launchpad.net/nova/+bug/1117923 [4] https://bugs.launchpad.net/neutron/+bug/1626010 [5] https://bugs.launchpad.net/neutron/+bug/1593760 ** Affects: neutron Importance: Undecided Status: New ** Tags: rfe trunk -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1631371 Title: [RFE] Expose trunk details over metadata API Status in neutron: New Bug description: Enable bringup of subports via exposing trunk/subport details over the metadata API With the completion of the trunk port feature in Newton (Neutron bp/vlan-aware-vms [1]), trunk and subports are now available. But the bringup of the subports' VLAN interfaces inside an instance is not automatic. In Newton there's no easy way to pass information about the subports to the guest operating system. But using the metadata API we can change this. Problem Description ------------------- To bring up (and/or tear down) a subport the guest OS (a) must know the segmentation-type and segmentation-id of a subport as set in 'openstack network trunk create/set --subport' (b) must know the MAC address of a subport as set in 'openstack port create' (c) must know which vNIC the subport belongs to (d) may need to know when were subports added or removed (if they are added or removed during the lifetime of an instance) Since subports do not have a corresponding vNIC, the approach used for regular ports (with a vNIC) cannot work. This write-up addresses problems (a), (b) and (c), but not (d). Proposed Change --------------- Here we propose a change involving both Nova and Neutron to expose the information needed via the metadata API. Information covering (a) and (b) is already available (read-only) in the 'trunk_details' attribute of the trunk parent port (ie. the port which the instance was booted with). [2] We propose to use the MAC address of the trunk parent port to cover (c). We recognize this may occasionally be problematic, because MAC addresses (of ports belonging to different neutron networks) are not guaranteed to be unique, therefore collision may happen. But this seems to be a small price for avoiding the complexity of other solutions. The mechanism would be the following. Let's suppose we have port0 which is a trunk parent port and instance0 was booted with '--nic port-id=port0'. On every update of port0's trunk_details Neutron constructs the following JSON structure: PORT0-DETAILS = { "mac_address": PORT0-MAC-ADDRESS, "trunk_details": PORT0-TRUNK-DETAILS } Then Neutron sets a metadata key-value pair of instance0, equivalent to the following nova command: nova meta set instance0 trunk_details::PORT0-MAC-ADDRESS=PORT0-DETAILS Nova in Newton limits meta values to <= 255 characters, this limit must be raised. Assuming the current format of trunk_details roughly 150 characters/subport are needed. Alternatively meta values could have unlimited length - at least for the service tenant used by Neutron. (Though tenant-specific API validators may not be a good idea.) The 'values' column of the the 'instance_metadata' table should be altered from VARCHAR(255) to TEXT() in a Nova DB migration. (A slightly related bug report: [3]) A program could read http://169.254.169.254/openstack/2016-06-30/meta_data.json and bring up the subport VLAN interfaces accordingly. This program is not covered here, however it is worth pointing out that it could be called by cloud-init. Alternatives ------------ (1) The MAC address of a parent port can be reused for all its child ports (when creating the child ports). Then VLAN subinterfaces of a network interface will have the correct MAC address by default. Segmentation type and ID can be shared in other ways, for example as a VLAN plan embedded into a golden image. This approach could even partially solve problem (d), however it cannot solve problem (a) in the dynamic case. Use of this approach is currently blocked by an openvswitch firewall driver bug. [4][5] (2) Generate and inject a subport bringup script into the instance as user data. Cannot handle subports added or removed after VM boot. (3) An alternative solution to problem (c) could rely on the preservation of ordering between NICs passed to nova boot and NICs inside an instance. However this would turn the update of trunk_details into an instance-level operation instead of the port-level operation proposed here. Plus it would fail if this ordering is ever lost. References ---------- [1] https://blueprints.launchpad.net/neutron/+spec/vlan-aware-vms [2] https://review.openstack.org/#q,Id23ce8fc16c6ea6a405cb8febf8470a5bf3bcb43,n,z [3] https://bugs.launchpad.net/nova/+bug/1117923 [4] https://bugs.launchpad.net/neutron/+bug/1626010 [5] https://bugs.launchpad.net/neutron/+bug/1593760 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1631371/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp