hidden configuration items
If you hit the /client/console endpoint with a vmid, it sends you back some data that contains a link to a the console proxy VM and passes an encrypted json payload that has the user, password and port for a vnc connection. Normally this is meant to load in an iframe. We want to decrypt this response to direct a VNC client to the appropriate host / port / user / pass and bypass use of the console proxy VM. The key and iv appear to be stored in the configuration table under the names security.encryption.key and security.encryption.iv, but as they are hidden we cannot get these credentials via the listConfigurations endpoint as-is. So my question is: What would be the most appropriate way to open up the possibility of showing “hidden” configuration items via this API to our middleware? Some sort of entry in a config file somewhere? An entry in the configuration table itself? Or is there some other way to get this information I’m looking for? Thanks in advance. Nathan
hidden configuration items revisited
I would like to get some feedback for a proposed addition of a feature that would allow “Hidden” configuration items to be returned from the listConfigurations endpoint. 1) There will be a new optional parameter for listConfigurations called showhidden . Defaults to false. Existing behavior is preserved unless showhidden is set to true. 2) There is a now configuration item, com.cloud.allowshowhidden , which defaults to false. This must be set to true in order for showhidden to be allowed. If showhidden=true is passed and com.cloud.allowshowhidden=false, an InvalidParameterValueException is thrown. So the web UI would still hide hidden configuration items regardless of the state of com.cloud.allowshowhidden since it will not be passing showhidden=true. The main value of this would be from API implementations / middleware, which is what our front-end talks to instead of directly to cloudstack management server. Obviously there is an explicit reason hidden configuration items are not displayed via the API at present. The Hidden configuration items contain some very sensitive data, such as private keys etc. I would like to submit a pull request that would make sense to everyone and still be secure by default and not open up pandora’s box so to speak. I have this working in our lab, but I wanted to get a bit of feedback before submitting a PR. So several questions: 1) Would it make sense for com.cloud.allowshowhidden to be a “Hidden” configuration item? The up side of this is that you could not toggle this value from the API. Marking it hidden means that a rogue root admin api key holder could not grant themselves more access. The down side is that I’m not sure how to easily change this value outside of manually going into the database and changing it, and one should hope that root admin api key holders are well trusted. Currently I have this implemented as an “Advanced” configuration item. 2) I picked com.cloud.allowshowhidden out of my hat. Is there a more appropriate name that I should use?
Re: hidden configuration items
I didn’t get how will you use that info to connect? There are three connections involved in showing console to user. ms ——authenticate—> CPVM ——for VNC console—>Hypervisor ^ | gets images from CPVM web browser Which of the above components you want to keep and which to remove? Also you can look into other implementations of Console proxy which are rarely used to get more info. Regards, Anshul > On 07-May-2016, at 11:39 AM, Nathan Johnson wrote: > > If you hit the /client/console endpoint with a vmid, it sends you back some > data that contains a link to a the console proxy VM and passes an encrypted > json payload that has the user, password and port for a vnc connection. > Normally this is meant to load in an iframe. We want to decrypt this > response to direct a VNC client to the appropriate host / port / user / > pass and bypass use of the console proxy VM. The key and iv appear to be > stored in the configuration table under the names security.encryption.key > and security.encryption.iv, but as they are hidden we cannot get these > credentials via the listConfigurations endpoint as-is. So my question is: > > What would be the most appropriate way to open up the possibility of > showing “hidden” configuration items via this API to our middleware? Some > sort of entry in a config file somewhere? An entry in the configuration > table itself? Or is there some other way to get this information I’m > looking for? > > Thanks in advance. > > Nathan DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: hidden configuration items
Anshul Gangwar wrote: > > ms ——authenticate—> CPVM ——for VNC console—>Hypervisor > ^ > | gets images from CPVM > web browser > > Which of the above components you want to keep and which to remove? Currently when you hit the management server and pass it a vmid, you get a response that is a html payload with a link in an iframe to the console proxy VM, along with a parameter passed on the get string that constitutes an encrypted JSON payload. This encrypted JSON payload includes all of the information that would be needed to connect via VNC. We want to be able to make the request from our middleware, intercept and decrypt this JSON payload and be be able to use an alternative web based VNC client. > > Also you can look into other implementations of Console proxy which are > rarely used to get more info. > Could you point me to one? I would be very interested to look at an alternative. Thank you, Nathan
Re: hidden configuration items
I know what happens but I am not sure how will you be able to connect to VNC with that info. From the diagram in previous mail which components you want to remove? Basically what I am trying to say here is that json payload is consists of data we get from management server. Only variable there is time on which token changes to make sure nobody in middle has changed anything. It is only meant for authentication purposes. This authentication code is present on management server itself. CPVM is able to connect to console because of it being present in same network. For connection purposes it doesn’t use any additional data which is not provided by management server. That being said, I can give more details only after knowing which components you want to replace. On 09/05/16, 7:27 PM, "Nathan Johnson" wrote: >Anshul Gangwar wrote: >> >> ms ——authenticate—> CPVM ——for VNC console—>Hypervisor >> ^ >> | gets images from CPVM >> web browser >> >> Which of the above components you want to keep and which to remove? > >Currently when you hit the management server and pass it a vmid, you get a >response that is a html payload with a link in an iframe to the console >proxy VM, along with a parameter passed on the get string that constitutes >an encrypted JSON payload. This encrypted JSON payload includes all of the >information that would be needed to connect via VNC. We want to be able to >make the request from our middleware, intercept and decrypt this JSON >payload and be be able to use an alternative web based VNC client. > >> >> Also you can look into other implementations of Console proxy which are >> rarely used to get more info. >> > >Could you point me to one? I would be very interested to look at an >alternative. > >Thank you, >Nathan > DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: hidden configuration items
On Sat, May 7, 2016 at 8:09 AM, Nathan Johnson wrote:
> If you hit the /client/console endpoint with a vmid, it sends you back some
> data that contains a link to a the console proxy VM and passes an encrypted
> json payload that has the user, password and port for a vnc connection.
> Normally this is meant to load in an iframe. We want to decrypt this
> response to direct a VNC client to the appropriate host / port / user /
> pass and bypass use of the console proxy VM. The key and iv appear to be
> stored in the configuration table under the names security.encryption.key
> and security.encryption.iv, but as they are hidden we cannot get these
> credentials via the listConfigurations endpoint as-is. So my question is:
>
> What would be the most appropriate way to open up the possibility of
> showing “hidden” configuration items via this API to our middleware? Some
> sort of entry in a config file somewhere? An entry in the configuration
> table itself? Or is there some other way to get this information I’m
> looking for?
>
>
Just tested:
mysql> update configuration set category='Secure' where name in
('security.encryption.iv', 'security.encryption.key');
(default) > list configurations name=security.encryption.iv
count = 1
configuration:
+--++++
| category | name | value |
description |
+--++++
| Secure | security.encryption.iv | | base64
encoded IV data |
+--++++
--
Erik
RE: hidden configuration items revisited
Nathan, You can use "Secure" category instead of "Hidden". Config items with "Secure" category are encrypted and also included in listConfigurations API response. ~kishan -Original Message- From: Nathan Johnson [mailto:njohn...@ena.com] Sent: 08 May 2016 22:01 To: dev@cloudstack.apache.org Subject: hidden configuration items revisited I would like to get some feedback for a proposed addition of a feature that would allow “Hidden” configuration items to be returned from the listConfigurations endpoint. 1) There will be a new optional parameter for listConfigurations called showhidden . Defaults to false. Existing behavior is preserved unless showhidden is set to true. 2) There is a now configuration item, com.cloud.allowshowhidden , which defaults to false. This must be set to true in order for showhidden to be allowed. If showhidden=true is passed and com.cloud.allowshowhidden=false, an InvalidParameterValueException is thrown. So the web UI would still hide hidden configuration items regardless of the state of com.cloud.allowshowhidden since it will not be passing showhidden=true. The main value of this would be from API implementations / middleware, which is what our front-end talks to instead of directly to cloudstack management server. Obviously there is an explicit reason hidden configuration items are not displayed via the API at present. The Hidden configuration items contain some very sensitive data, such as private keys etc. I would like to submit a pull request that would make sense to everyone and still be secure by default and not open up pandora’s box so to speak. I have this working in our lab, but I wanted to get a bit of feedback before submitting a PR. So several questions: 1) Would it make sense for com.cloud.allowshowhidden to be a “Hidden” configuration item? The up side of this is that you could not toggle this value from the API. Marking it hidden means that a rogue root admin api key holder could not grant themselves more access. The down side is that I’m not sure how to easily change this value outside of manually going into the database and changing it, and one should hope that root admin api key holders are well trusted. Currently I have this implemented as an “Advanced” configuration item. 2) I picked com.cloud.allowshowhidden out of my hat. Is there a more appropriate name that I should use? DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: hidden configuration items revisited
Kishan Kavala wrote: > Nathan, > You can use "Secure" category instead of "Hidden". Config items with > "Secure" category are encrypted and also included in listConfigurations API > response. The data that I need (specifically security.encryption.iv and security.encryption.key) are already marked “Hidden". These are the keys that are used to encrypt the response that I need to decrypt in the middleware. The configuration item I’m proposing to add is a boolean, so that doesn’t make sense to be “Secure” either. Unless I’m misunderstanding? Thanks for the reply Nathan > > ~kishan > > -Original Message- > From: Nathan Johnson [mailto:njohn...@ena.com] > Sent: 08 May 2016 22:01 > To: dev@cloudstack.apache.org > Subject: hidden configuration items revisited > > I would like to get some feedback for a proposed addition of a feature > that would allow “Hidden” configuration items to be returned from the > listConfigurations endpoint. > > 1) There will be a new optional parameter for listConfigurations called > showhidden . Defaults to false. Existing behavior is preserved unless > showhidden is set to true. > > 2) There is a now configuration item, com.cloud.allowshowhidden , which > defaults to false. This must be set to true in order for showhidden to > be allowed. If showhidden=true is passed and > com.cloud.allowshowhidden=false, an InvalidParameterValueException is > thrown. > > So the web UI would still hide hidden configuration items regardless of > the state of com.cloud.allowshowhidden since it will not be passing > showhidden=true. The main value of this would be from API > implementations / middleware, which is what our front-end talks to > instead of directly to cloudstack management server. > > Obviously there is an explicit reason hidden configuration items are not > displayed via the API at present. The Hidden configuration items contain > some very sensitive data, such as private keys etc. I would like to > submit a pull request that would make sense to everyone and still be > secure by default and not open up pandora’s box so to speak. I have this > working in our lab, but I wanted to get a bit of feedback before > submitting a PR. > > So several questions: > > 1) Would it make sense for com.cloud.allowshowhidden to be a “Hidden” > configuration item? The up side of this is that you could not toggle > this value from the API. Marking it hidden means that a rogue root admin > api key holder could not grant themselves more access. The down side is > that I’m not sure how to easily change this value outside of manually > going into the database and changing it, and one should hope that root > admin api key holders are well trusted. Currently I have this > implemented as an “Advanced” configuration item. > > 2) I picked com.cloud.allowshowhidden out of my hat. Is there a more > appropriate name that I should use? > > > > > DISCLAIMER > == > This e-mail may contain privileged and confidential information which is > the property of Accelerite, a Persistent Systems business. It is intended > only for the use of the individual or entity to which it is addressed. If > you are not the intended recipient, you are not authorized to read, > retain, copy, print, distribute or use this message. If you have received > this communication in error, please notify the sender and delete all > copies of this message. Accelerite, a Persistent Systems business does > not accept any liability for virus infected mails.
Re: hidden configuration items revisited
I believe Kishan suggested that you could change those Hidden config items to Secure (an existing category), as Secure items are returned with the listConfiguration API. I don't know if you can do an in-place switch, or if the value has to be encrypted first for it to work, but you should be able to test that in a test environment. -- Erik On Mon, May 9, 2016 at 3:53 PM, Nathan Johnson wrote: > Kishan Kavala wrote: > > > Nathan, > > You can use "Secure" category instead of "Hidden". Config items with > "Secure" category are encrypted and also included in listConfigurations API > response. > > The data that I need (specifically security.encryption.iv and > security.encryption.key) are already marked “Hidden". These are the keys > that are used to encrypt the response that I need to decrypt in the > middleware. The configuration item I’m proposing to add is a boolean, so > that doesn’t make sense to be “Secure” either. Unless I’m > misunderstanding? > > Thanks for the reply > > Nathan > > > > > > ~kishan > > > > -Original Message- > > From: Nathan Johnson [mailto:njohn...@ena.com] > > Sent: 08 May 2016 22:01 > > To: dev@cloudstack.apache.org > > Subject: hidden configuration items revisited > > > > I would like to get some feedback for a proposed addition of a feature > > that would allow “Hidden” configuration items to be returned from the > > listConfigurations endpoint. > > > > 1) There will be a new optional parameter for listConfigurations called > > showhidden . Defaults to false. Existing behavior is preserved unless > > showhidden is set to true. > > > > 2) There is a now configuration item, com.cloud.allowshowhidden , which > > defaults to false. This must be set to true in order for showhidden to > > be allowed. If showhidden=true is passed and > > com.cloud.allowshowhidden=false, an InvalidParameterValueException is > > thrown. > > > > So the web UI would still hide hidden configuration items regardless of > > the state of com.cloud.allowshowhidden since it will not be passing > > showhidden=true. The main value of this would be from API > > implementations / middleware, which is what our front-end talks to > > instead of directly to cloudstack management server. > > > > Obviously there is an explicit reason hidden configuration items are not > > displayed via the API at present. The Hidden configuration items contain > > some very sensitive data, such as private keys etc. I would like to > > submit a pull request that would make sense to everyone and still be > > secure by default and not open up pandora’s box so to speak. I have this > > working in our lab, but I wanted to get a bit of feedback before > > submitting a PR. > > > > So several questions: > > > > 1) Would it make sense for com.cloud.allowshowhidden to be a “Hidden” > > configuration item? The up side of this is that you could not toggle > > this value from the API. Marking it hidden means that a rogue root admin > > api key holder could not grant themselves more access. The down side is > > that I’m not sure how to easily change this value outside of manually > > going into the database and changing it, and one should hope that root > > admin api key holders are well trusted. Currently I have this > > implemented as an “Advanced” configuration item. > > > > 2) I picked com.cloud.allowshowhidden out of my hat. Is there a more > > appropriate name that I should use? > > > > > > > > > > DISCLAIMER > > == > > This e-mail may contain privileged and confidential information which is > > the property of Accelerite, a Persistent Systems business. It is intended > > only for the use of the individual or entity to which it is addressed. If > > you are not the intended recipient, you are not authorized to read, > > retain, copy, print, distribute or use this message. If you have received > > this communication in error, please notify the sender and delete all > > copies of this message. Accelerite, a Persistent Systems business does > > not accept any liability for virus infected mails. > > >
Re: hidden configuration items revisited
Erik Weber wrote: > I believe Kishan suggested that you could change those Hidden config items > to Secure (an existing category), as Secure items are returned with the > listConfiguration API. This is chicken and egg. I need the unencrypted values so I can decrypt other payloads. I need the keys plaintext, one way or another.
Re: hidden configuration items revisited
listConfiguration returns unencrypted values for Secure items, but they need to be stored encrypted in the db. You'd need to check If those values ever change, If they don't you may try encrypting the value and change category to Secure Erik Den mandag 9. mai 2016 skrev Nathan Johnson følgende: > Erik Weber > wrote: > > > I believe Kishan suggested that you could change those Hidden config > items > > to Secure (an existing category), as Secure items are returned with the > > listConfiguration API. > > This is chicken and egg. I need the unencrypted values so I can decrypt > other payloads. I need the keys plaintext, one way or another. >
Re: hidden configuration items revisited
'Hidden' and 'Secure' are both encrypted in db only difference being hidden values are not shown. you could just change the category in configuration table. ~Rajani On Mon, May 9, 2016 at 10:56 PM, Erik Weber wrote: > listConfiguration returns unencrypted values for Secure items, but they > need to be stored encrypted in the db. > > You'd need to check If those values ever change, If they don't you may try > encrypting the value and change category to Secure > > Erik > > Den mandag 9. mai 2016 skrev Nathan Johnson følgende: > > > Erik Weber > wrote: > > > > > I believe Kishan suggested that you could change those Hidden config > > items > > > to Secure (an existing category), as Secure items are returned with the > > > listConfiguration API. > > > > This is chicken and egg. I need the unencrypted values so I can decrypt > > other payloads. I need the keys plaintext, one way or another. > > >
Re: hidden configuration items revisited
Rajani is right, I didn't look at the value in db, just the category. -- Erik On Tue, May 10, 2016 at 8:30 AM, Rajani Karuturi wrote: > 'Hidden' and 'Secure' are both encrypted in db only difference being hidden > values are not shown. > you could just change the category in configuration table. > > ~Rajani > > On Mon, May 9, 2016 at 10:56 PM, Erik Weber wrote: > > > listConfiguration returns unencrypted values for Secure items, but they > > need to be stored encrypted in the db. > > > > You'd need to check If those values ever change, If they don't you may > try > > encrypting the value and change category to Secure > > > > Erik > > > > Den mandag 9. mai 2016 skrev Nathan Johnson følgende: > > > > > Erik Weber > wrote: > > > > > > > I believe Kishan suggested that you could change those Hidden config > > > items > > > > to Secure (an existing category), as Secure items are returned with > the > > > > listConfiguration API. > > > > > > This is chicken and egg. I need the unencrypted values so I can > decrypt > > > other payloads. I need the keys plaintext, one way or another. > > > > > >
