Re: [MERGE] CS-2163
This has been merged into master now. On 12/05/13 2:40 PM, "Nitin Mehta" wrote: >Chip - The change is not significant but a subset of the work already >done. Going this path makes the model of adding semantic metadata more >flexible and can be extended to have a more flexible ACL in future as >well. >I am already done with the changes and have added the unit and integration >tests as well. The RAT build passes as well. > >On 11/05/13 11:02 PM, "Chip Childers" wrote: > >>On Thu, May 09, 2013 at 09:55:02AM +, Nitin Mehta wrote: >>> Updated the FS [1] as per the discussion. So the plan is to introduce a >>> generic api to >>> add semantic meta data to the first class objects. >>> >>> AddResourceDetail - Add details to resource >>> >>> * resourceIds - List of resource ids >>> * resourceType - Example - volume, nic etc. >>> * details - Map of key/value pairs >>> >>> DeleteResourceDetail - Remove detail to resource >>> >>> * resourceIds - List of resource ids >>> * resourceType - Example - volume, nic etc. >>> * details - Map of key/value pairs >>> >>> ListResourceDetails - lists details for a resource >>> >>> * resourceId >>> * resourceType - Example - volume, nic etc. >>> * key >>> * value >>> >>> >>> >>> [1] - >>> >>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+b >>>e >>>tt >>> er+control+over+first+class+objects+in+CS >> >>Is your plan to re-work the feature now? >
Re: [MERGE] CS-2163
Chip - The change is not significant but a subset of the work already done. Going this path makes the model of adding semantic metadata more flexible and can be extended to have a more flexible ACL in future as well. I am already done with the changes and have added the unit and integration tests as well. The RAT build passes as well. On 11/05/13 11:02 PM, "Chip Childers" wrote: >On Thu, May 09, 2013 at 09:55:02AM +, Nitin Mehta wrote: >> Updated the FS [1] as per the discussion. So the plan is to introduce a >> generic api to >> add semantic meta data to the first class objects. >> >> AddResourceDetail - Add details to resource >> >> * resourceIds - List of resource ids >> * resourceType - Example - volume, nic etc. >> * details - Map of key/value pairs >> >> DeleteResourceDetail - Remove detail to resource >> >> * resourceIds - List of resource ids >> * resourceType - Example - volume, nic etc. >> * details - Map of key/value pairs >> >> ListResourceDetails - lists details for a resource >> >> * resourceId >> * resourceType - Example - volume, nic etc. >> * key >> * value >> >> >> >> [1] - >> >>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+be >>tt >> er+control+over+first+class+objects+in+CS > >Is your plan to re-work the feature now?
Re: [MERGE] CS-2163
On Thu, May 09, 2013 at 09:55:02AM +, Nitin Mehta wrote: > Updated the FS [1] as per the discussion. So the plan is to introduce a > generic api to > add semantic meta data to the first class objects. > > AddResourceDetail - Add details to resource > > * resourceIds - List of resource ids > * resourceType - Example - volume, nic etc. > * details - Map of key/value pairs > > DeleteResourceDetail - Remove detail to resource > > * resourceIds - List of resource ids > * resourceType - Example - volume, nic etc. > * details - Map of key/value pairs > > ListResourceDetails - lists details for a resource > > * resourceId > * resourceType - Example - volume, nic etc. > * key > * value > > > > [1] - > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+bett > er+control+over+first+class+objects+in+CS Is your plan to re-work the feature now?
Re: [MERGE] CS-2163
Updated the FS [1] as per the discussion. So the plan is to introduce a generic api to add semantic meta data to the first class objects. AddResourceDetail - Add details to resource * resourceIds - List of resource ids * resourceType - Example - volume, nic etc. * details - Map of key/value pairs DeleteResourceDetail - Remove detail to resource * resourceIds - List of resource ids * resourceType - Example - volume, nic etc. * details - Map of key/value pairs ListResourceDetails - lists details for a resource * resourceId * resourceType - Example - volume, nic etc. * key * value [1] - https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+bett er+control+over+first+class+objects+in+CS On 09/05/13 3:04 PM, "Nitin Mehta" wrote: > > >On 03/05/13 7:58 PM, "Chip Childers" wrote: > >>On Fri, May 03, 2013 at 04:11:21AM +, Murali Reddy wrote: >>> On 02/05/13 11:20 PM, "Chiradeep Vittal" >>> wrote: >>> >>> > >>> > >>> >On 5/2/13 6:24 AM, "Chip Childers" wrote: >>> > >>> >>On Thu, May 02, 2013 at 12:40:45PM +0530, Abhinandan Prateek wrote: >>> >>> >>> >>> The tags are name-value pairs basically meant to label resources by >>>a >>> >>>user. >>> >>> The purpose of the current API is to add useful meta information to >>> >>> resources so that they can be meaningfully controlled by external >>> >>>tools. >>> >>> >>> >>> The current implementation of meta info API may look similar to >>>tags >>> >>>with >>> >>> possible edition of a "system" or a "user" qualifier/flag in order >>>to >>> >>> control visibility. >>> >>> >>> >>> But it is not so. Tags by definition are name value pairs where >>>both >>> >>>name >>> >>> and value is a string. If we try to set the meta data information >>>in >>> >>>the >>> >>> tags we are severely restricting the type of meta information that >>>can >>> >>>be >>> >>> contained in the tags. For example if the "data-type" information >>>is >>> >>> required in this meta-data we will end up tweaking the tag system >>>to >>> >>>serve >>> >>> some purpose that it is not meant for. >>> >>> >>> >>> I strongly suggest that we do not try to contain the meta >>>information >>> >>>in a >>> >>> restricted framework provided by tags. >>> >>> >>> >>> -abhi >>> >> >>> >>I don't understand this argument. Most primary data types have >>>string >>> >>representations, right? (ref: JSON) >>> >> >>> >>Are you talking about complex types? Or things like integers, long, >>> >>datetime, etc...? >>> > >>> >I didn't understand it either. >>> >Perhaps the issue is whether this meta-data is available to the >>>end-user >>> >or not? At least, tags are visible to the end-user. Is it the >>>intention >>> >that this 3rd party service do its magic in cahoots with the admin? >>> > >>> > >>> >>> I would like to see this as, meta-data that has semantic meaning to >>> CloudStack vs meta-data that's purely user meta-data that has no >>>meaning >>> to CloudStack and its plug-ins. 'Tags' falls in second category. >> >>Aren't tags used quite a bit for placement functions? >> >>> From e.g. >>> use-case Nitin has put in FS, I do see valid case where a >>>plug-in/feature >>> of CloudStack or external systems like to book keep information of an >>> entity. We already have a pattern of storing such meta data of entities >>> (user_vm_details, cluster_details etc) in the details table which is >>> basically a key-value pair. IMO, I see no harm if we can generalise it. >> >>I agree with generalizing it, absolutely. >> >>I think my question is: Are we talking about CS internals only? Or are >>we talking about a truly generalized system that can contain metadata >>for the system, the admins and the users? When I read the functional >>spec, >>it seems confusing as to which of these are being focused on, and if not >>all of them are we getting ourselves into another situation where the >>data authorization model is limited to some specific use case and not >>flexible enough to extend in other ways? >> >>One other FS question: >> >>I see this: "DeployVM/UpdateVM/ - Add flag whether to display the VM to >>end user". What's the definition of "end user" in this scenario? Are >>account admins end users? How about domain admins? Is root admin the >>only one that can see these things? > >Yes, currently as per the use case root admin is the only one allowed to >give him control over the first class objects, but again this can be >customized in future. > >> >>It seems that the implementation is limited in scope yet again to a >>boolean flag. Similar to the question of metadata scope, why would this >>not be designed to be a more robust data authorization model? >> >>It seems like this whole feature should be approached as: >> >>1 - Define a data structure (possibly extending tags) that allows for >>flexible key/value pairs being attached to virtual entities within CS. > >As per suggestions by Murali, I am planning to introduce a generic api to >add semantic meta data to the first class objects. > >>
Re: [MERGE] CS-2163
On 03/05/13 7:58 PM, "Chip Childers" wrote: >On Fri, May 03, 2013 at 04:11:21AM +, Murali Reddy wrote: >> On 02/05/13 11:20 PM, "Chiradeep Vittal" >> wrote: >> >> > >> > >> >On 5/2/13 6:24 AM, "Chip Childers" wrote: >> > >> >>On Thu, May 02, 2013 at 12:40:45PM +0530, Abhinandan Prateek wrote: >> >>> >> >>> The tags are name-value pairs basically meant to label resources by >>a >> >>>user. >> >>> The purpose of the current API is to add useful meta information to >> >>> resources so that they can be meaningfully controlled by external >> >>>tools. >> >>> >> >>> The current implementation of meta info API may look similar to tags >> >>>with >> >>> possible edition of a "system" or a "user" qualifier/flag in order >>to >> >>> control visibility. >> >>> >> >>> But it is not so. Tags by definition are name value pairs where both >> >>>name >> >>> and value is a string. If we try to set the meta data information in >> >>>the >> >>> tags we are severely restricting the type of meta information that >>can >> >>>be >> >>> contained in the tags. For example if the "data-type" information is >> >>> required in this meta-data we will end up tweaking the tag system to >> >>>serve >> >>> some purpose that it is not meant for. >> >>> >> >>> I strongly suggest that we do not try to contain the meta >>information >> >>>in a >> >>> restricted framework provided by tags. >> >>> >> >>> -abhi >> >> >> >>I don't understand this argument. Most primary data types have string >> >>representations, right? (ref: JSON) >> >> >> >>Are you talking about complex types? Or things like integers, long, >> >>datetime, etc...? >> > >> >I didn't understand it either. >> >Perhaps the issue is whether this meta-data is available to the >>end-user >> >or not? At least, tags are visible to the end-user. Is it the intention >> >that this 3rd party service do its magic in cahoots with the admin? >> > >> > >> >> I would like to see this as, meta-data that has semantic meaning to >> CloudStack vs meta-data that's purely user meta-data that has no meaning >> to CloudStack and its plug-ins. 'Tags' falls in second category. > >Aren't tags used quite a bit for placement functions? > >> From e.g. >> use-case Nitin has put in FS, I do see valid case where a >>plug-in/feature >> of CloudStack or external systems like to book keep information of an >> entity. We already have a pattern of storing such meta data of entities >> (user_vm_details, cluster_details etc) in the details table which is >> basically a key-value pair. IMO, I see no harm if we can generalise it. > >I agree with generalizing it, absolutely. > >I think my question is: Are we talking about CS internals only? Or are >we talking about a truly generalized system that can contain metadata >for the system, the admins and the users? When I read the functional >spec, >it seems confusing as to which of these are being focused on, and if not >all of them are we getting ourselves into another situation where the >data authorization model is limited to some specific use case and not >flexible enough to extend in other ways? > >One other FS question: > >I see this: "DeployVM/UpdateVM/ - Add flag whether to display the VM to >end user". What's the definition of "end user" in this scenario? Are >account admins end users? How about domain admins? Is root admin the >only one that can see these things? Yes, currently as per the use case root admin is the only one allowed to give him control over the first class objects, but again this can be customized in future. > >It seems that the implementation is limited in scope yet again to a >boolean flag. Similar to the question of metadata scope, why would this >not be designed to be a more robust data authorization model? > >It seems like this whole feature should be approached as: > >1 - Define a data structure (possibly extending tags) that allows for >flexible key/value pairs being attached to virtual entities within CS. As per suggestions by Murali, I am planning to introduce a generic api to add semantic meta data to the first class objects. > >2 - Build an ACL model that's actually able to handle different ACL >requirements for that metadata AS WELL AS for the foundational virtual >resources. Valid point. I am going to currently enable it for root admin but certainly the design can be extended to handle ACL requirements in future as well. > >Perhaps I'm over complicating things, but this FS was only first created >on April 25 and I think that these concerns need to be discussed / >debated. > >-chip
Re: [MERGE] CS-2163
On 06/05/13 5:20 PM, "Prasanna Santhanam" wrote: >On Mon, May 06, 2013 at 11:39:49AM +, Murali Reddy wrote: >> On 03/05/13 7:47 PM, "Chip Childers" wrote: >> >> >On Fri, May 03, 2013 at 04:11:21AM +, Murali Reddy wrote: >> >> >> >> >> >> I would like to see this as, meta-data that has semantic meaning to >> >> CloudStack vs meta-data that's purely user meta-data that has no >>meaning >> >> to CloudStack and its plug-ins. 'Tags' falls in second category. >> > >> >Aren't tags used quite a bit for placement functions? >> > >To be clear - host tags cause affinity towards hosts that bear the >same tag. This tagging can be done for storage pools too. The current proposal is to associate similar data to resources so that cloudstack can interpret it. e.g. the do not display flag affects whether the VM can be returned to the end user or not. The do not display flag will be used to not display the shadow Vms as in a DR solution. > >Resource Tags are simply meta-data to which a user can attribute >meaning outside of CloudStack's knowledge and function. > >-- >Prasanna., > > >Powered by BigRock.com >
Re: [MERGE] CS-2163
On 06/05/13 5:18 PM, "Prasanna Santhanam" wrote: >> >> Agree. There could be use cases for both services that are cloud >>provider >> managed (admin) vs self-service that need to access/store meta-data. We >> need to keep design to be flexible. Also current proposal to add new API >> for each of the action (add/update/remove) and corresponding entity >> (AddNicDetail, UpdateVolumeDetail etc) looks too redundant. IMO, we can >> just have create/delete/List/Update meta data API's with resource type >>and >> id. We can keep API's both user and admin accessible. > >Like createTags, listTags, deleteTags? >From the design of API perspective, yes. But as I mentioned earlier, I would like think resource meta-data as separate from the 'tags'. > >-- >Prasanna., > > >Powered by BigRock.com > >
Re: [MERGE] CS-2163
On Mon, May 06, 2013 at 11:39:49AM +, Murali Reddy wrote: > On 03/05/13 7:47 PM, "Chip Childers" wrote: > > >On Fri, May 03, 2013 at 04:11:21AM +, Murali Reddy wrote: > >> > >> > >> I would like to see this as, meta-data that has semantic meaning to > >> CloudStack vs meta-data that's purely user meta-data that has no meaning > >> to CloudStack and its plug-ins. 'Tags' falls in second category. > > > >Aren't tags used quite a bit for placement functions? > To be clear - host tags cause affinity towards hosts that bear the same tag. This tagging can be done for storage pools too. Resource Tags are simply meta-data to which a user can attribute meaning outside of CloudStack's knowledge and function. -- Prasanna., Powered by BigRock.com
Re: [MERGE] CS-2163
On Mon, May 06, 2013 at 11:39:49AM +, Murali Reddy wrote: > On 03/05/13 7:47 PM, "Chip Childers" wrote: > > >On Fri, May 03, 2013 at 04:11:21AM +, Murali Reddy wrote: > >> > >> > >> I would like to see this as, meta-data that has semantic meaning to > >> CloudStack vs meta-data that's purely user meta-data that has no meaning > >> to CloudStack and its plug-ins. 'Tags' falls in second category. > > > >Aren't tags used quite a bit for placement functions? > > I got confused as well, actually host tags [1] is different from AWS like > tags [2]. In this case I can see host tags as meta-data (of hosts) that > has semantic meaning (for deployment planner) to CloudStack. Adding > key/value pair meta-data to virtual entities within CS is easy way to > extend the properties of CS objects. Those two are unrelated features - host tags and resource tags. We call too many things tags IMO. The tags I meant were Resource Tags, these you will see in the UI against every resource that takes key,value pair meta-data. Alena developed this feature IIRC. > > > > >> From e.g. > >> use-case Nitin has put in FS, I do see valid case where a > >>plug-in/feature > >> of CloudStack or external systems like to book keep information of an > >> entity. We already have a pattern of storing such meta data of entities > >> (user_vm_details, cluster_details etc) in the details table which is > >> basically a key-value pair. IMO, I see no harm if we can generalise it. > > > >I agree with generalizing it, absolutely. > > > >I think my question is: Are we talking about CS internals only? Or are > >we talking about a truly generalized system that can contain metadata > >for the system, the admins and the users? When I read the functional > >spec, > >it seems confusing as to which of these are being focused on, and if not > >all of them are we getting ourselves into another situation where the > >data authorization model is limited to some specific use case and not > >flexible enough to extend in other ways? > > Agree. There could be use cases for both services that are cloud provider > managed (admin) vs self-service that need to access/store meta-data. We > need to keep design to be flexible. Also current proposal to add new API > for each of the action (add/update/remove) and corresponding entity > (AddNicDetail, UpdateVolumeDetail etc) looks too redundant. IMO, we can > just have create/delete/List/Update meta data API's with resource type and > id. We can keep API's both user and admin accessible. Like createTags, listTags, deleteTags? -- Prasanna., Powered by BigRock.com
Re: [MERGE] CS-2163
On 03/05/13 7:47 PM, "Chip Childers" wrote: >On Fri, May 03, 2013 at 04:11:21AM +, Murali Reddy wrote: >> >> >> I would like to see this as, meta-data that has semantic meaning to >> CloudStack vs meta-data that's purely user meta-data that has no meaning >> to CloudStack and its plug-ins. 'Tags' falls in second category. > >Aren't tags used quite a bit for placement functions? I got confused as well, actually host tags [1] is different from AWS like tags [2]. In this case I can see host tags as meta-data (of hosts) that has semantic meaning (for deployment planner) to CloudStack. Adding key/value pair meta-data to virtual entities within CS is easy way to extend the properties of CS objects. > >> From e.g. >> use-case Nitin has put in FS, I do see valid case where a >>plug-in/feature >> of CloudStack or external systems like to book keep information of an >> entity. We already have a pattern of storing such meta data of entities >> (user_vm_details, cluster_details etc) in the details table which is >> basically a key-value pair. IMO, I see no harm if we can generalise it. > >I agree with generalizing it, absolutely. > >I think my question is: Are we talking about CS internals only? Or are >we talking about a truly generalized system that can contain metadata >for the system, the admins and the users? When I read the functional >spec, >it seems confusing as to which of these are being focused on, and if not >all of them are we getting ourselves into another situation where the >data authorization model is limited to some specific use case and not >flexible enough to extend in other ways? Agree. There could be use cases for both services that are cloud provider managed (admin) vs self-service that need to access/store meta-data. We need to keep design to be flexible. Also current proposal to add new API for each of the action (add/update/remove) and corresponding entity (AddNicDetail, UpdateVolumeDetail etc) looks too redundant. IMO, we can just have create/delete/List/Update meta data API's with resource type and id. We can keep API's both user and admin accessible. > >One other FS question: > >I see this: "DeployVM/UpdateVM/ - Add flag whether to display the VM to >end user". What's the definition of "end user" in this scenario? Are >account admins end users? How about domain admins? Is root admin the >only one that can see these things? > >It seems that the implementation is limited in scope yet again to a >boolean flag. Similar to the question of metadata scope, why would this >not be designed to be a more robust data authorization model? > >It seems like this whole feature should be approached as: > >1 - Define a data structure (possibly extending tags) that allows for >flexible key/value pairs being attached to virtual entities within CS. > >2 - Build an ACL model that's actually able to handle different ACL >requirements for that metadata AS WELL AS for the foundational virtual >resources. > >Perhaps I'm over complicating things, but this FS was only first created >on April 25 and I think that these concerns need to be discussed / >debated. > >-chip [1] http://s.apache.org/bG2 [2] http://s.apache.org/16q
Re: [MERGE] CS-2163
On Fri, May 03, 2013 at 04:11:21AM +, Murali Reddy wrote: > On 02/05/13 11:20 PM, "Chiradeep Vittal" > wrote: > > > > > > >On 5/2/13 6:24 AM, "Chip Childers" wrote: > > > >>On Thu, May 02, 2013 at 12:40:45PM +0530, Abhinandan Prateek wrote: > >>> > >>> The tags are name-value pairs basically meant to label resources by a > >>>user. > >>> The purpose of the current API is to add useful meta information to > >>> resources so that they can be meaningfully controlled by external > >>>tools. > >>> > >>> The current implementation of meta info API may look similar to tags > >>>with > >>> possible edition of a "system" or a "user" qualifier/flag in order to > >>> control visibility. > >>> > >>> But it is not so. Tags by definition are name value pairs where both > >>>name > >>> and value is a string. If we try to set the meta data information in > >>>the > >>> tags we are severely restricting the type of meta information that can > >>>be > >>> contained in the tags. For example if the "data-type" information is > >>> required in this meta-data we will end up tweaking the tag system to > >>>serve > >>> some purpose that it is not meant for. > >>> > >>> I strongly suggest that we do not try to contain the meta information > >>>in a > >>> restricted framework provided by tags. > >>> > >>> -abhi > >> > >>I don't understand this argument. Most primary data types have string > >>representations, right? (ref: JSON) > >> > >>Are you talking about complex types? Or things like integers, long, > >>datetime, etc...? > > > >I didn't understand it either. > >Perhaps the issue is whether this meta-data is available to the end-user > >or not? At least, tags are visible to the end-user. Is it the intention > >that this 3rd party service do its magic in cahoots with the admin? > > > > > > I would like to see this as, meta-data that has semantic meaning to > CloudStack vs meta-data that's purely user meta-data that has no meaning > to CloudStack and its plug-ins. 'Tags' falls in second category. Aren't tags used quite a bit for placement functions? > From e.g. > use-case Nitin has put in FS, I do see valid case where a plug-in/feature > of CloudStack or external systems like to book keep information of an > entity. We already have a pattern of storing such meta data of entities > (user_vm_details, cluster_details etc) in the details table which is > basically a key-value pair. IMO, I see no harm if we can generalise it. I agree with generalizing it, absolutely. I think my question is: Are we talking about CS internals only? Or are we talking about a truly generalized system that can contain metadata for the system, the admins and the users? When I read the functional spec, it seems confusing as to which of these are being focused on, and if not all of them are we getting ourselves into another situation where the data authorization model is limited to some specific use case and not flexible enough to extend in other ways? One other FS question: I see this: "DeployVM/UpdateVM/ - Add flag whether to display the VM to end user". What's the definition of "end user" in this scenario? Are account admins end users? How about domain admins? Is root admin the only one that can see these things? It seems that the implementation is limited in scope yet again to a boolean flag. Similar to the question of metadata scope, why would this not be designed to be a more robust data authorization model? It seems like this whole feature should be approached as: 1 - Define a data structure (possibly extending tags) that allows for flexible key/value pairs being attached to virtual entities within CS. 2 - Build an ACL model that's actually able to handle different ACL requirements for that metadata AS WELL AS for the foundational virtual resources. Perhaps I'm over complicating things, but this FS was only first created on April 25 and I think that these concerns need to be discussed / debated. -chip
Re: [MERGE] CS-2163
On 02/05/13 11:20 PM, "Chiradeep Vittal" wrote: > > >On 5/2/13 6:24 AM, "Chip Childers" wrote: > >>On Thu, May 02, 2013 at 12:40:45PM +0530, Abhinandan Prateek wrote: >>> >>> The tags are name-value pairs basically meant to label resources by a >>>user. >>> The purpose of the current API is to add useful meta information to >>> resources so that they can be meaningfully controlled by external >>>tools. >>> >>> The current implementation of meta info API may look similar to tags >>>with >>> possible edition of a "system" or a "user" qualifier/flag in order to >>> control visibility. >>> >>> But it is not so. Tags by definition are name value pairs where both >>>name >>> and value is a string. If we try to set the meta data information in >>>the >>> tags we are severely restricting the type of meta information that can >>>be >>> contained in the tags. For example if the "data-type" information is >>> required in this meta-data we will end up tweaking the tag system to >>>serve >>> some purpose that it is not meant for. >>> >>> I strongly suggest that we do not try to contain the meta information >>>in a >>> restricted framework provided by tags. >>> >>> -abhi >> >>I don't understand this argument. Most primary data types have string >>representations, right? (ref: JSON) >> >>Are you talking about complex types? Or things like integers, long, >>datetime, etc...? > >I didn't understand it either. >Perhaps the issue is whether this meta-data is available to the end-user >or not? At least, tags are visible to the end-user. Is it the intention >that this 3rd party service do its magic in cahoots with the admin? > > I would like to see this as, meta-data that has semantic meaning to CloudStack vs meta-data that's purely user meta-data that has no meaning to CloudStack and its plug-ins. 'Tags' falls in second category. From e.g. use-case Nitin has put in FS, I do see valid case where a plug-in/feature of CloudStack or external systems like to book keep information of an entity. We already have a pattern of storing such meta data of entities (user_vm_details, cluster_details etc) in the details table which is basically a key-value pair. IMO, I see no harm if we can generalise it.
Re: [MERGE] CS-2163
On Thu, May 02, 2013 at 10:50:25AM -0700, Chiradeep Vittal wrote: > > > On 5/2/13 6:24 AM, "Chip Childers" wrote: > > >On Thu, May 02, 2013 at 12:40:45PM +0530, Abhinandan Prateek wrote: > >> > >> The tags are name-value pairs basically meant to label resources by a > >>user. > >> The purpose of the current API is to add useful meta information to > >> resources so that they can be meaningfully controlled by external tools. > >> > >> The current implementation of meta info API may look similar to tags > >>with > >> possible edition of a "system" or a "user" qualifier/flag in order to > >> control visibility. > >> > >> But it is not so. Tags by definition are name value pairs where both > >>name > >> and value is a string. If we try to set the meta data information in the > >> tags we are severely restricting the type of meta information that can > >>be > >> contained in the tags. For example if the "data-type" information is > >> required in this meta-data we will end up tweaking the tag system to > >>serve > >> some purpose that it is not meant for. > >> > >> I strongly suggest that we do not try to contain the meta information > >>in a > >> restricted framework provided by tags. > >> > >> -abhi > > > >I don't understand this argument. Most primary data types have string > >representations, right? (ref: JSON) > > > >Are you talking about complex types? Or things like integers, long, > >datetime, etc...? > > I didn't understand it either. > Perhaps the issue is whether this meta-data is available to the end-user > or not? At least, tags are visible to the end-user. Is it the intention > that this 3rd party service do its magic in cahoots with the admin? > > I'd be OK if we were talking about an ACL for that, but I can see plenty of reasons for a user to want meta-data about their instances.
Re: [MERGE] CS-2163
On 5/2/13 6:24 AM, "Chip Childers" wrote: >On Thu, May 02, 2013 at 12:40:45PM +0530, Abhinandan Prateek wrote: >> >> The tags are name-value pairs basically meant to label resources by a >>user. >> The purpose of the current API is to add useful meta information to >> resources so that they can be meaningfully controlled by external tools. >> >> The current implementation of meta info API may look similar to tags >>with >> possible edition of a "system" or a "user" qualifier/flag in order to >> control visibility. >> >> But it is not so. Tags by definition are name value pairs where both >>name >> and value is a string. If we try to set the meta data information in the >> tags we are severely restricting the type of meta information that can >>be >> contained in the tags. For example if the "data-type" information is >> required in this meta-data we will end up tweaking the tag system to >>serve >> some purpose that it is not meant for. >> >> I strongly suggest that we do not try to contain the meta information >>in a >> restricted framework provided by tags. >> >> -abhi > >I don't understand this argument. Most primary data types have string >representations, right? (ref: JSON) > >Are you talking about complex types? Or things like integers, long, >datetime, etc...? I didn't understand it either. Perhaps the issue is whether this meta-data is available to the end-user or not? At least, tags are visible to the end-user. Is it the intention that this 3rd party service do its magic in cahoots with the admin?
Re: [MERGE] CS-2163
On Thu, May 02, 2013 at 12:40:45PM +0530, Abhinandan Prateek wrote: > > The tags are name-value pairs basically meant to label resources by a user. > The purpose of the current API is to add useful meta information to > resources so that they can be meaningfully controlled by external tools. > > The current implementation of meta info API may look similar to tags with > possible edition of a "system" or a "user" qualifier/flag in order to > control visibility. > > But it is not so. Tags by definition are name value pairs where both name > and value is a string. If we try to set the meta data information in the > tags we are severely restricting the type of meta information that can be > contained in the tags. For example if the "data-type" information is > required in this meta-data we will end up tweaking the tag system to serve > some purpose that it is not meant for. > > I strongly suggest that we do not try to contain the meta information in a > restricted framework provided by tags. > > -abhi I don't understand this argument. Most primary data types have string representations, right? (ref: JSON) Are you talking about complex types? Or things like integers, long, datetime, etc...?
Re: [MERGE] CS-2163
The tags are name-value pairs basically meant to label resources by a user. The purpose of the current API is to add useful meta information to resources so that they can be meaningfully controlled by external tools. The current implementation of meta info API may look similar to tags with possible edition of a "system" or a "user" qualifier/flag in order to control visibility. But it is not so. Tags by definition are name value pairs where both name and value is a string. If we try to set the meta data information in the tags we are severely restricting the type of meta information that can be contained in the tags. For example if the "data-type" information is required in this meta-data we will end up tweaking the tag system to serve some purpose that it is not meant for. I strongly suggest that we do not try to contain the meta information in a restricted framework provided by tags. -abhi On Tue, Apr 30, 2013 at 09:25:40AM +, Nitin Mehta wrote: > Prasanna - Thanks for your question. The reasons for not using createTags > are as follows :- > > - tags functionality is for grouping resources than adding meta > information and is allowed to all users. What I am proposing is for the > admin The FS of tags [1] states the same objectives: """ Tag is the first class object in the cloudStack """ The resource tags is based on AWS tags that carry meta-information for users to do nifty things [2] like managing billing, filtering, external services etc. In the AWS case it is up to the user to determine what she does with a tag. > to have better control over the first class objects in CS. This ability > should be with admin only. Why only admin? Like I pointed, I as a user could decide to fire APIs that do the same failure/backup implementation across my regions if my provider hasn't enabled any service as you describe in your use case. > - Also list* (* == volume, vm etc.) APIs can list the tags for the enduser > and mixing tags with the meta information wouldn't go well as they > are admin only. I don't want to mix the meta information with the resource > response because they are not the attributes of the resource, we should > try to get to this model in future. That's a scope/access control issue of the API. As an admin I could reserve a set of tags for myself to use that my users are forbidden to create. AWS reserves the 'aws' tag. > (Like we don't want to add stats to vm response ) > - Lets not use something just because they seem similar, because they both > might evolve in a different way in future. I'm just trying to understand if the existence of tags was overlooked before developing this feature. To me it sounds like one can enhance the existing APIs than have to introduce new ones that can be potentially confusing. [1] https://cwiki.apache.org/confluence/x/PYnlAQ [2] http://s.apache.org/v4 -- Prasanna., Powered by BigRock.com
Re: [MERGE] CS-2163
Min - Thanks for your questions. I have updated the FS with the list Apis. I also updated the FS with the DB view changes. This was already done while submitting the merge request.Thanks for pointing this out. Thanks, -Nitin On 29/04/13 11:07 PM, "Min Chen" wrote: >Nitin, > > I have several questions after reading your FS: >1. The API to be modified should also include some list APIs to control >which resources to be displayed to the end user based on your new flag, >right? You didn't mention such APIs in your FS, so I am asking for >clarification. >2. If answer to #1 is true, then you may need to modify new DB views >introduced in 4.1 to reflect that flag as well so that you can filter >resources for end user. > Thanks > -min > >On 4/28/13 9:01 PM, "Nitin Mehta" wrote: > >>I would like to merge the feature which will add the ability in CS so >>that the user can have better control over first class objects in CS >> >>- The feature can be found in the branch cs-2163 and has been rebased >>with master. >>- Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2163 >>- The FS can be found @ >>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+be >>t >>ter+control+over+first+class+objects+in+CS >>- I have added the unit tests and marvin tests as well. >>- I ran the RAT check as well with no failures. >> >>Thanks, >>Nitin >
Re: [MERGE] CS-2163
+1 to Prasanna's argument. Tags provide more flexibility and extensibility than adding column to existing db schema. What happens if later on we want to add finer control to these resources not specific to end user? Adding another column? Also, I haven't seen your answer to my questions in this thread. -min On 4/30/13 7:58 AM, "Prasanna Santhanam" wrote: >Also - tags supports many more resources in CloudStack already as >compared to the resources you are adding for meta-information. All the >more reason to enhance existing APIs? > >On Tue, Apr 30, 2013 at 08:21:39PM +0530, Prasanna Santhanam wrote: >> On Tue, Apr 30, 2013 at 09:25:40AM +, Nitin Mehta wrote: >> > Prasanna - Thanks for your question. The reasons for not using >>createTags >> > are as follows :- >> > >> > - tags functionality is for grouping resources than adding meta >> > information and is allowed to all users. What I am proposing is for >>the >> > admin >> The FS of tags [1] states the same objectives: >> >> """ Tag is the first class object in the cloudStack """ >> >> The resource tags is based on AWS tags that carry meta-information for >> users to do nifty things [2] like managing billing, filtering, external >> services etc. In the AWS case it is up to the user to determine what >> she does with a tag. >> >> > to have better control over the first class objects in CS. This >>ability >> > should be with admin only. >> Why only admin? Like I pointed, I as a user could decide to fire APIs >> that do the same failure/backup implementation across my regions if my >> provider hasn't enabled any service as you describe in your use case. >> >> > - Also list* (* == volume, vm etc.) APIs can list the tags for the >>enduser >> > and mixing tags with the meta information wouldn't go well as they >> > are admin only. I don't want to mix the meta information with the >>resource >> > response because they are not the attributes of the resource, we >>should >> > try to get to this model in future. >> That's a scope/access control issue of the API. As an admin I could >> reserve a set of tags for myself to use that my users are forbidden to >> create. AWS reserves the 'aws' tag. >> >> > (Like we don't want to add stats to vm response ) >> > - Lets not use something just because they seem similar, because they >>both >> > might evolve in a different way in future. >> >> I'm just trying to understand if the existence of tags was overlooked >> before developing this feature. To me it sounds like one can enhance >> the existing APIs than have to introduce new ones that can be >> potentially confusing. >> >> [1] https://cwiki.apache.org/confluence/x/PYnlAQ >> [2] http://s.apache.org/v4 >> >> -- >> Prasanna., >> >> >> Powered by BigRock.com > >-- >Prasanna., > > >Powered by BigRock.com >
Re: [MERGE] CS-2163
Also - tags supports many more resources in CloudStack already as compared to the resources you are adding for meta-information. All the more reason to enhance existing APIs? On Tue, Apr 30, 2013 at 08:21:39PM +0530, Prasanna Santhanam wrote: > On Tue, Apr 30, 2013 at 09:25:40AM +, Nitin Mehta wrote: > > Prasanna - Thanks for your question. The reasons for not using createTags > > are as follows :- > > > > - tags functionality is for grouping resources than adding meta > > information and is allowed to all users. What I am proposing is for the > > admin > The FS of tags [1] states the same objectives: > > """ Tag is the first class object in the cloudStack """ > > The resource tags is based on AWS tags that carry meta-information for > users to do nifty things [2] like managing billing, filtering, external > services etc. In the AWS case it is up to the user to determine what > she does with a tag. > > > to have better control over the first class objects in CS. This ability > > should be with admin only. > Why only admin? Like I pointed, I as a user could decide to fire APIs > that do the same failure/backup implementation across my regions if my > provider hasn't enabled any service as you describe in your use case. > > > - Also list* (* == volume, vm etc.) APIs can list the tags for the enduser > > and mixing tags with the meta information wouldn't go well as they > > are admin only. I don't want to mix the meta information with the resource > > response because they are not the attributes of the resource, we should > > try to get to this model in future. > That's a scope/access control issue of the API. As an admin I could > reserve a set of tags for myself to use that my users are forbidden to > create. AWS reserves the 'aws' tag. > > > (Like we don't want to add stats to vm response ) > > - Lets not use something just because they seem similar, because they both > > might evolve in a different way in future. > > I'm just trying to understand if the existence of tags was overlooked > before developing this feature. To me it sounds like one can enhance > the existing APIs than have to introduce new ones that can be > potentially confusing. > > [1] https://cwiki.apache.org/confluence/x/PYnlAQ > [2] http://s.apache.org/v4 > > -- > Prasanna., > > > Powered by BigRock.com -- Prasanna., Powered by BigRock.com
Re: [MERGE] CS-2163
On Tue, Apr 30, 2013 at 09:25:40AM +, Nitin Mehta wrote: > Prasanna - Thanks for your question. The reasons for not using createTags > are as follows :- > > - tags functionality is for grouping resources than adding meta > information and is allowed to all users. What I am proposing is for the > admin The FS of tags [1] states the same objectives: """ Tag is the first class object in the cloudStack """ The resource tags is based on AWS tags that carry meta-information for users to do nifty things [2] like managing billing, filtering, external services etc. In the AWS case it is up to the user to determine what she does with a tag. > to have better control over the first class objects in CS. This ability > should be with admin only. Why only admin? Like I pointed, I as a user could decide to fire APIs that do the same failure/backup implementation across my regions if my provider hasn't enabled any service as you describe in your use case. > - Also list* (* == volume, vm etc.) APIs can list the tags for the enduser > and mixing tags with the meta information wouldn't go well as they > are admin only. I don't want to mix the meta information with the resource > response because they are not the attributes of the resource, we should > try to get to this model in future. That's a scope/access control issue of the API. As an admin I could reserve a set of tags for myself to use that my users are forbidden to create. AWS reserves the 'aws' tag. > (Like we don't want to add stats to vm response ) > - Lets not use something just because they seem similar, because they both > might evolve in a different way in future. I'm just trying to understand if the existence of tags was overlooked before developing this feature. To me it sounds like one can enhance the existing APIs than have to introduce new ones that can be potentially confusing. [1] https://cwiki.apache.org/confluence/x/PYnlAQ [2] http://s.apache.org/v4 -- Prasanna., Powered by BigRock.com
Re: [MERGE] CS-2163
Prasanna - Thanks for your question. The reasons for not using createTags are as follows :- - tags functionality is for grouping resources than adding meta information and is allowed to all users. What I am proposing is for the admin to have better control over the first class objects in CS. This ability should be with admin only. - Also list* (* == volume, vm etc.) APIs can list the tags for the enduser and mixing tags with the meta information wouldn't go well as they are admin only. I don't want to mix the meta information with the resource response because they are not the attributes of the resource, we should try to get to this model in future. (Like we don't want to add stats to vm response ) - Lets not use something just because they seem similar, because they both might evolve in a different way in future. Hope you agree. Let me know your thoughts on this. Thanks, -Nitin On 30/04/13 11:06 AM, "Prasanna Santhanam" wrote: >On Mon, Apr 29, 2013 at 04:01:27AM +, Nitin Mehta wrote: >> I would like to merge the feature which will add the ability in CS so >>that the user can have better control over first class objects in CS >> >> - The feature can be found in the branch cs-2163 and has been rebased >>with master. >> - Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2163 >> - The FS can be found @ >>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+be >>tter+control+over+first+class+objects+in+CS >> - I have added the unit tests and marvin tests as well. >> - I ran the RAT check as well with no failures. > >Why not just use resource tags to convey this meta-information? All it >would mean for the third-party subscriber would be to get this from >the event bus as you mentioned. > >createTags & resourceType = VM/Disk/Snapshot etc & key=k, value=v? > >-- >Prasanna., > > >Powered by BigRock.com
Re: [MERGE] CS-2163
On Mon, Apr 29, 2013 at 04:01:27AM +, Nitin Mehta wrote: > I would like to merge the feature which will add the ability in CS so that > the user can have better control over first class objects in CS > > - The feature can be found in the branch cs-2163 and has been rebased with > master. > - Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2163 > - The FS can be found @ > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+better+control+over+first+class+objects+in+CS > - I have added the unit tests and marvin tests as well. > - I ran the RAT check as well with no failures. Why not just use resource tags to convey this meta-information? All it would mean for the third-party subscriber would be to get this from the event bus as you mentioned. createTags & resourceType = VM/Disk/Snapshot etc & key=k, value=v? -- Prasanna., Powered by BigRock.com
Re: [MERGE] CS-2163
Nitin, I have several questions after reading your FS: 1. The API to be modified should also include some list APIs to control which resources to be displayed to the end user based on your new flag, right? You didn't mention such APIs in your FS, so I am asking for clarification. 2. If answer to #1 is true, then you may need to modify new DB views introduced in 4.1 to reflect that flag as well so that you can filter resources for end user. Thanks -min On 4/28/13 9:01 PM, "Nitin Mehta" wrote: >I would like to merge the feature which will add the ability in CS so >that the user can have better control over first class objects in CS > >- The feature can be found in the branch cs-2163 and has been rebased >with master. >- Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2163 >- The FS can be found @ >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+bet >ter+control+over+first+class+objects+in+CS >- I have added the unit tests and marvin tests as well. >- I ran the RAT check as well with no failures. > >Thanks, >Nitin
Re: [MERGE] CS-2163
On Mon, Apr 29, 2013 at 04:01:27AM +, Nitin Mehta wrote: > I would like to merge the feature which will add the ability in CS so that > the user can have better control over first class objects in CS > > - The feature can be found in the branch cs-2163 and has been rebased with > master. > - Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2163 > - The FS can be found @ > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+better+control+over+first+class+objects+in+CS > - I have added the unit tests and marvin tests as well. > - I ran the RAT check as well with no failures. > > Thanks, > Nitin +1 from me.
[MERGE] CS-2163
I would like to merge the feature which will add the ability in CS so that the user can have better control over first class objects in CS - The feature can be found in the branch cs-2163 and has been rebased with master. - Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-2163 - The FS can be found @ https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ability+to+have+better+control+over+first+class+objects+in+CS - I have added the unit tests and marvin tests as well. - I ran the RAT check as well with no failures. Thanks, Nitin
