Re: [openstack-dev] [nova][trove] Protected openstack resources
On 3 June 2015 at 20:51, Amrith Kumar amr...@tesora.com wrote: Several of us including Bruno Lago, Victoria Martinez de la Cruz (vkmc), Flavio Percoco, Nikhil Manchanda (SlickNik), Vipul Sabhaya (vipul), Doug Shelley (dougshelley66), and several others from the Trove team met in Vancouver and were joined by some others (who I do not know by name) from other projects. After the summit, I summarized that meeting in the email thread here [1]. In that meeting, and in the course of other conversations, we concluded that projects like Trove require the ability to launch instances of resources from projects like Nova but bad things happen when a user directly interacts with these resources. It would therefore be highly advantageous to have a class of instances which are protected from direct user actions. The “bad things” described above stem from the fact that the guest agent that Trove uses is a component that is on the guest instance and it communicates with the other Trove controller services over an oslo.messaging message queue. If a guest instance were compromised, the fact that it has a connection path to the message queue could become a vulnerability. Deployers of Trove have addressed these concerns and are able to operate a secure Trove system by launching Nova instances in a different tenant than the end user. The changes to Trove for this are currently not part of Trove but will be made available shortly. FWIW, this is basically how Glance uses a multi-tenant Swift to store images from Nova from various tenants. I think there are more exciting ways that some folks have brewing that involve some sort of combination of two tenants, or some such. Using oslo.messaging for the communication between Trove controller and the guest agents allows deployers to choose the underlying AMQP transport. However, oslo.messaging is tightly coupled with AMQP semantics. One proposed alternative (zaqar) that could address some of Trove’s issues has no integration with oslo.messaging. Therefore, to adopt zaqar, Trove would likely have to abandon oslo.messaging and integrate tightly with zaqar which strikes many of us as more restrictive and less attractive. I know of at least one user of Trove who has deployed oslo.messaging with qpid as the underlying transport, rather than the more commonly deployed RabbitMQ. The request to create an oslo.messaging driver for zaqar (or was it a zaqar driver for oslo.messaging) met with some resistance for technical reasons. Flavio summarizes it in 2 saying, “This is probably the main reason why there's no driver for Zaqar in oslo.messaging. That is, to prevent people from actually using Zaqar as a message bus in openstack.” So you could create a REST API for your agent to talk to. They are quite well understood, but I have no idea about how your agent talks to your server, so it could be a terrible idea. Other projects like Sahara, and potentially others need a mechanism by which to protect their resources from direct manipulation by a user. Several conversations ensued with members of Nova team and Bruno drafted a write-up summarizing some aspects of the problems. To facilitate a quick review of this request, the Trove team has put together a document and it is available for review at 3. The request is to have Nova and potentially other OpenStack projects review the issues being described. They can then provide protected resources that projects like Trove can consume. Equally, if you work on some other project that could benefit from protected resources, please chime in. Please post comments on the request on the review (4) and register blueprints or work towards delivering these capabilities in your respective projects. The request is not prescriptive of how projects like Nova should implement these capabilities, it merely requests that they be created. Why is the running your Nova VMs in a trove or sahara specific tenant not good enough for your use case? I am not trying to be difficult, I am just curious about what specific issues something better would need to fix. Thanks, John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][trove] Protected openstack resources
John, Thanks for your note. I've updated the review at https://review.openstack.org/#/c/186357/ with answers to some of your questions (and I added you to that review). Trove's use-case like some of the other projects listed is different from Glance in that Trove has a guest agent. I've tried to explain that in more detail in patch set 5. I'd appreciate your comments. Thanks, -amrith | -Original Message- | From: John Garbutt [mailto:j...@johngarbutt.com] | Sent: Thursday, June 04, 2015 6:54 AM | To: OpenStack Development Mailing List (not for usage questions) | Subject: Re: [openstack-dev] [nova][trove] Protected openstack resources | | On 3 June 2015 at 20:51, Amrith Kumar amr...@tesora.com wrote: | Several of us including Bruno Lago, Victoria Martinez de la Cruz | (vkmc), Flavio Percoco, Nikhil Manchanda (SlickNik), Vipul Sabhaya | (vipul), Doug Shelley (dougshelley66), and several others from the | Trove team met in Vancouver and were joined by some others (who I do | not know by name) from other projects. After the summit, I summarized | that meeting in the email thread here [1]. | | In that meeting, and in the course of other conversations, we | concluded that projects like Trove require the ability to launch | instances of resources from projects like Nova but bad things happen | when a user directly interacts with these resources. It would | therefore be highly advantageous to have a class of instances which are | protected from direct user actions. | | The “bad things” described above stem from the fact that the guest | agent that Trove uses is a component that is on the guest instance and | it communicates with the other Trove controller services over an | oslo.messaging message queue. If a guest instance were compromised, | the fact that it has a connection path to the message queue could | become a vulnerability. Deployers of Trove have addressed these | concerns and are able to operate a secure Trove system by launching | Nova instances in a different tenant than the end user. The changes to | Trove for this are currently not part of Trove but will be made | available shortly. | | FWIW, this is basically how Glance uses a multi-tenant Swift to store | images from Nova from various tenants. | | I think there are more exciting ways that some folks have brewing that | involve some sort of combination of two tenants, or some such. | | Using oslo.messaging for the communication between Trove controller | and the guest agents allows deployers to choose the underlying AMQP | transport. | However, oslo.messaging is tightly coupled with AMQP semantics. One | proposed alternative (zaqar) that could address some of Trove’s issues | has no integration with oslo.messaging. | | Therefore, to adopt zaqar, Trove would likely have to abandon | oslo.messaging and integrate tightly with zaqar which strikes many of | us as more restrictive and less attractive. I know of at least one | user of Trove who has deployed oslo.messaging with qpid as the | underlying transport, rather than the more commonly deployed RabbitMQ. | | The request to create an oslo.messaging driver for zaqar (or was it a | zaqar driver for oslo.messaging) met with some resistance for technical | reasons. | Flavio summarizes it in 2 saying, “This is probably the main reason | why there's no driver for Zaqar in oslo.messaging. That is, to prevent | people from actually using Zaqar as a message bus in openstack.” | | So you could create a REST API for your agent to talk to. They are quite | well understood, but I have no idea about how your agent talks to your | server, so it could be a terrible idea. | | Other projects like Sahara, and potentially others need a mechanism by | which to protect their resources from direct manipulation by a user. | | Several conversations ensued with members of Nova team and Bruno | drafted a write-up summarizing some aspects of the problems. To | facilitate a quick review of this request, the Trove team has put | together a document and it is available for review at 3. | | The request is to have Nova and potentially other OpenStack projects | review the issues being described. They can then provide protected | resources that projects like Trove can consume. | | Equally, if you work on some other project that could benefit from | protected resources, please chime in. | | Please post comments on the request on the review (4) and register | blueprints or work towards delivering these capabilities in your | respective projects. The request is not prescriptive of how projects | like Nova should implement these capabilities, it merely requests that | they be created. | | Why is the running your Nova VMs in a trove or sahara specific tenant not | good enough for your use case? | | I am not trying to be difficult, I am just curious about what specific | issues something better would need to fix. | | Thanks, | John
Re: [openstack-dev] [nova][trove] Protected openstack resources
Excerpts from Amrith Kumar's message of 2015-06-04 12:46:37 +: John, Thanks for your note. I've updated the review at https://review.openstack.org/#/c/186357/ with answers to some of your questions (and I added you to that review). Trove's use-case like some of the other projects listed is different from Glance in that Trove has a guest agent. I've tried to explain that in more detail in patch set 5. I'd appreciate your comments. We solved this in Akanda by placing the service VMs in a special tenant, isolating them with security group rules, and then giving the agent running in the VM a REST API connected to a private management network owned by the same tenant that owns the VM. All communication with the agent starts from a service on the outside, through that management network. The VMs act as routers, so they are also attached to the cloud-user's networks, but the agent doesn't respond on those networks. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][trove] Protected openstack resources
Several of us including Bruno Lago, Victoria Martinez de la Cruz (vkmc), Flavio Percoco, Nikhil Manchanda (SlickNik), Vipul Sabhaya (vipul), Doug Shelley (dougshelley66), and several others from the Trove team met in Vancouver and were joined by some others (who I do not know by name) from other projects. After the summit, I summarized that meeting in the email thread here [1http://markmail.org/message/cotket76zyhbvj34]. In that meeting, and in the course of other conversations, we concluded that projects like Trove require the ability to launch instances of resources from projects like Nova but bad things happen when a user directly interacts with these resources. It would therefore be highly advantageous to have a class of instances which are protected from direct user actions. The bad things described above stem from the fact that the guest agent that Trove uses is a component that is on the guest instance and it communicates with the other Trove controller services over an oslo.messaging message queue. If a guest instance were compromised, the fact that it has a connection path to the message queue could become a vulnerability. Deployers of Trove have addressed these concerns and are able to operate a secure Trove system by launching Nova instances in a different tenant than the end user. The changes to Trove for this are currently not part of Trove but will be made available shortly. Using oslo.messaging for the communication between Trove controller and the guest agents allows deployers to choose the underlying AMQP transport. However, oslo.messaging is tightly coupled with AMQP semantics. One proposed alternative (zaqar) that could address some of Trove's issues has no integration with oslo.messaging. Therefore, to adopt zaqar, Trove would likely have to abandon oslo.messaging and integrate tightly with zaqar which strikes many of us as more restrictive and less attractive. I know of at least one user of Trove who has deployed oslo.messaging with qpid as the underlying transport, rather than the more commonly deployed RabbitMQ. The request to create an oslo.messaging driver for zaqar (or was it a zaqar driver for oslo.messaging) met with some resistance for technical reasons. Flavio summarizes it in 2http://markmail.org/message/v3vm7bux5bi6vknu saying, This is probably the main reason why there's no driver for Zaqar in oslo.messaging. That is, to prevent people from actually using Zaqar as a message bus in openstack. Other projects like Sahara, and potentially others need a mechanism by which to protect their resources from direct manipulation by a user. Several conversations ensued with members of Nova team and Bruno drafted a write-up summarizing some aspects of the problems. To facilitate a quick review of this request, the Trove team has put together a document and it is available for review at 3https://review.openstack.org/#/c/186357. The request is to have Nova and potentially other OpenStack projects review the issues being described. They can then provide protected resources that projects like Trove can consume. Equally, if you work on some other project that could benefit from protected resources, please chime in. Please post comments on the request on the review (4https://review.openstack.org/#/c/186357) and register blueprints or work towards delivering these capabilities in your respective projects. The request is not prescriptive of how projects like Nova should implement these capabilities, it merely requests that they be created. Thanks, -amrith [1] http://markmail.org/message/cotket76zyhbvj34 [2] http://markmail.org/message/v3vm7bux5bi6vknu [3] https://review.openstack.org/#/c/186357 [4] https://review.openstack.org/#/c/186357 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
