Re: [openstack-dev] [nova] Adding hostId to metadata
On 2018-06-27 12:37:43 -0400 (-0400), Jay Pipes wrote: [...] > the hostId was not, in fact, the host identifier, but rather a > SHA-224 hash of the host and project_id. [...] Oh, that's slick. Yeah, it would basically take brute-forcing the UUID space to divine the actual host identifier from that (you could use it to confirm a known identifier, but not easily discover it). I too am not concerned about security in light of this, though it does open the door to users doing things like booting and deleting instances until they get one scheduled to a compute node they like (for whatever reason, be it affinity, anti-affinity, et cetera). -- Jeremy Stanley signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Adding hostId to metadata
On 06/27/2018 12:20 PM, Matt Riedemann wrote: On 6/27/2018 10:13 AM, Jay Pipes wrote: I'm -2'd the patch in question because of these concerns about crossing the line between administrative and guest/virtual domains. It may seem like a very trivial patch, but from what I can tell, it would be a very big departure from the types of information we have traditionally allowed in the metadata API. To be clear, this is exposing the exact same hashed host+project_id value via the metadata API that you can already get, as a non-admin user, from the compute REST API: https://github.com/openstack/nova/blob/c8b93fa2493dce82ef4c0b1e7a503ba9b81c2e86/nova/api/openstack/compute/views/servers.py#L135 So I don't think it's a security issue at all. My sincere apologies. I did not realize that the hostId was not, in fact, the host identifier, but rather a SHA-224 hash of the host and project_id. The one thing I would be a bit worried about is that the value would be stale from the config drive if the instance is live migrated. We also expose the availability zone the instance is in from the config drive, but as far as I know you can't live migrate your way into another availability zone (unless of course the admin force live migrates to another host in another AZ and bypasses the scheduler). OK, I'll remove my -2. Apologies! -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Adding hostId to metadata
On 06/27/18 11:20, Matt Riedemann wrote: To be clear, this is exposing the exact same hashed host+project_id value via the metadata API that you can already get, as a non-admin user, from the compute REST API: https://github.com/openstack/nova/blob/c8b93fa2493dce82ef4c0b1e7a503ba9b81c2e86/nova/api/openstack/compute/views/servers.py#L135 So I don't think it's a security issue at all. I'm not sure I understand this rationale. Strictly speaking, I would think that in order for this to be true, the set of authenticated control plane users would have to always include the set of users who can read the metadata from a guest. Which I'm pretty sure is not true in the general case. Am I missing something? -- Michael Glasgow __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Adding hostId to metadata
On 2018-06-27 11:13:04 -0400 (-0400), Jay Pipes wrote: [...] > Virtual machines and the software running in them should not need > to know what particular piece of hardware they are running on. VMs > having knowledge of the underlying hardware and host violates the > principle of least privilege and introduces attack vectors that > I'm pretty sure you (as an operator) don't want to open up. [...] I saw similar security red flags with the proposal, but didn't weigh in at the time because I was confident Nova core reviewers would arrive at the same quite quickly on their own. While it would be "nice" to have this for the Infra team to be able to give providers a heads up when we see instances crashing consistently on a particular compute node, we're not the administrators of those compute nodes and so it is not information for which we should expect to have access. It may be a pain to collect up instance UUIDs and them pass those along to the provider so they can correlate to compute nodes in their service logs, but that's ultimately the right way to go about it so that separation of concerns is preserved. -- Jeremy Stanley signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Adding hostId to metadata
On 6/27/2018 10:13 AM, Jay Pipes wrote: I'm -2'd the patch in question because of these concerns about crossing the line between administrative and guest/virtual domains. It may seem like a very trivial patch, but from what I can tell, it would be a very big departure from the types of information we have traditionally allowed in the metadata API. To be clear, this is exposing the exact same hashed host+project_id value via the metadata API that you can already get, as a non-admin user, from the compute REST API: https://github.com/openstack/nova/blob/c8b93fa2493dce82ef4c0b1e7a503ba9b81c2e86/nova/api/openstack/compute/views/servers.py#L135 So I don't think it's a security issue at all. The one thing I would be a bit worried about is that the value would be stale from the config drive if the instance is live migrated. We also expose the availability zone the instance is in from the config drive, but as far as I know you can't live migrate your way into another availability zone (unless of course the admin force live migrates to another host in another AZ and bypasses the scheduler). -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Adding hostId to metadata
On 06/25/2018 05:28 PM, Mohammed Naser wrote: Hi everyone: While working with the OpenStack infrastructure team, we noticed that we were having some intermittent issues where we wanted to identify a theory if all VMs with this issue were landing on the same hypervisor. However, there seems to be no way of directly accessing `hostId` from inside the virtual machine (such as using the metadata API). Yes, that is correct. VMs should not know (or need to know) where they are hosted. This is a very useful thing to expose over the metadata API as not only would it help for troubleshooting these types of scenarios however it would also help software that can manage anti-affinity simply by checking the API and taking scheduling decisions. We try very hard to not expose administrative operational details about the underlying hardware via the metadata API. Virtual machines and the software running in them should not need to know what particular piece of hardware they are running on. VMs having knowledge of the underlying hardware and host violates the principle of least privilege and introduces attack vectors that I'm pretty sure you (as an operator) don't want to open up. There is a bright red line between the adminstrative domain and the virtual/guest domain, and presenting host identifiers over the metadata API would definitely cross that bright red line. I've proposed the following patch to add this[1], however, this is technically an API change, and the blueprints document specifies that "API changes always require a design discussion." Also, I believe that we're in a state where getting a spec would require an exception. However, this is a very trivial change. Also, according to the notes in the metadata file, it looks like there is one "bump" per OpenStack release[3] which means that this change can just be part of that release-wide version bump of the OpenStack API. Can we include this trivial patch in the upcoming Rocky release? I'm -2'd the patch in question because of these concerns about crossing the line between administrative and guest/virtual domains. It may seem like a very trivial patch, but from what I can tell, it would be a very big departure from the types of information we have traditionally allowed in the metadata API. Best, -jay Thanks, Mohammed [1]: https://review.openstack.org/577933 [2]: https://docs.openstack.org/nova/latest/contributor/blueprints.html [3]: http://git.openstack.org/cgit/openstack/nova/tree/nova/api/metadata/base.py#n60 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Adding hostId to metadata
We only bump the version if something has changed IIRC. I think bumping when nothing has changed would create a burden for implementers of client software. So its not like you get a chance to sneak this in "for free". Does this information really need to be available in the host OS? Its trivial to look it up via our existing APIs outside the host, although possibly less trivial if the instance has already been deleted. Michael On Tue, Jun 26, 2018 at 7:30 AM Mohammed Naser wrote: > Hi everyone: > > While working with the OpenStack infrastructure team, we noticed that > we were having some intermittent issues where we wanted to identify a > theory if all VMs with this issue were landing on the same hypervisor. > > However, there seems to be no way of directly accessing `hostId` from > inside the virtual machine (such as using the metadata API). This is > a very useful thing to expose over the metadata API as not only would > it help for troubleshooting these types of scenarios however it would > also help software that can manage anti-affinity simply by checking > the API and taking scheduling decisions. > > I've proposed the following patch to add this[1], however, this is > technically an API change, and the blueprints document specifies that > "API changes always require a design discussion." > > Also, I believe that we're in a state where getting a spec would > require an exception. However, this is a very trivial change. Also, > according to the notes in the metadata file, it looks like there is > one "bump" per OpenStack release[3] which means that this change can > just be part of that release-wide version bump of the OpenStack API. > > Can we include this trivial patch in the upcoming Rocky release? > > Thanks, > Mohammed > > [1]: https://review.openstack.org/577933 > [2]: https://docs.openstack.org/nova/latest/contributor/blueprints.html > [3]: > http://git.openstack.org/cgit/openstack/nova/tree/nova/api/metadata/base.py#n60 > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Did this email leave you hoping to cause me pain? Good news! Sponsor me in city2surf 2018 and I promise to suffer greatly. http://www.madebymikal.com/city2surf-2018/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Adding hostId to metadata
Hi everyone: While working with the OpenStack infrastructure team, we noticed that we were having some intermittent issues where we wanted to identify a theory if all VMs with this issue were landing on the same hypervisor. However, there seems to be no way of directly accessing `hostId` from inside the virtual machine (such as using the metadata API). This is a very useful thing to expose over the metadata API as not only would it help for troubleshooting these types of scenarios however it would also help software that can manage anti-affinity simply by checking the API and taking scheduling decisions. I've proposed the following patch to add this[1], however, this is technically an API change, and the blueprints document specifies that "API changes always require a design discussion." Also, I believe that we're in a state where getting a spec would require an exception. However, this is a very trivial change. Also, according to the notes in the metadata file, it looks like there is one "bump" per OpenStack release[3] which means that this change can just be part of that release-wide version bump of the OpenStack API. Can we include this trivial patch in the upcoming Rocky release? Thanks, Mohammed [1]: https://review.openstack.org/577933 [2]: https://docs.openstack.org/nova/latest/contributor/blueprints.html [3]: http://git.openstack.org/cgit/openstack/nova/tree/nova/api/metadata/base.py#n60 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
