Thanks for the detailed response, John! When Chris Suich (formerly from NetApp) and I talked about this several months ago, we were thinking that this would replace the need for storage tagging. However, as you point out, I now think it does not.
We were originally figuring that by specifically selecting a vendor (by picking the applicable storage plug-in) that this removed the need for storage tagging, but I see now that you might still want to further refine within a given storage provider which primary storage CloudStack selects. That being the case, if the user selects a specific storage plug-in, that will be the first step in filtering and storage tagging will be the next. Once I update the design, maybe you can take a look at it again. I can include the new data types you listed. Thanks! On Fri, Jun 13, 2014 at 8:38 AM, John Burwell <jburw...@basho.com> wrote: > Mike, > > I completely agree with the need to attach driver-specified key-value > pairs to a storage driver (we have been batting this around for nearly a > year). However, I think this facility should be generalized to support all > drivers (e.g. network, compute, etc) where storage maybe the first layer to > implement it. For example, we have a raft of places where vendor specific > data/concepts leak into the object model and schema. For example, in > networking, we have vendor specific API calls. One of the main value > propositions of a cloud platform is to provide a unified abstraction for > underlying infrastructure components. Furthermore, requiring vendor > implements to add API calls further increases the complexity and effort > required to support CloudStack — discouraging their participation. > > In terms of the design of itself, I don’t feel that the Map<String, > String> getPropertiesAndTypes is a rich enough semantic for such an > extension facility. First, it does not provide a mechanism to validate > that the form and content of the values being set. Second, it is > incredible lose contract that does not exploit the Java compiler to ensure > drivers are providing well-formed metadata to the CloudStack orchestration > components. Finally, it does not support notions such as hinting which > fields are required (very useful for creating useful UIs) and constrained > values (rendered as drop downs). My thoughts are as follows: > > > - Define an ExtenstionPropertyType enumeration with the following > types permitted: > - STRING > - DATE > - NUMBER > - BOOLEAN > - The following methods as well: > - Class getType(): The underlying Java type used to represent > the type > - Object convertFromString(String aValue): Converts a value from > a string to the underlying Java type > - String convertToString(Object aValue): Converts a value to a > string from the underlying Java type > - Define a ExtensionPropertyDefinition interface that describes the > following attributes about a extension property: > - String getName(): The name of the property conforming to JavaBean > standards (while we don’t support reflection, it’s a well known > standard by > Java developers …) > - ExtensionPropertyType getType(): The type of the property > - boolean isRequired(): Whether or not a value for the property is > required > - boolean isCollected(): Whether or not the property’s value is a > single value or collected (default to a list) > - Set<String> getConstraints: A list of the values that constrain > the values of this extension property. > - List<String> validate(Object aValue): A callback to validate a > value for this property definition — returning a list of strings > describing > the validation errors. An empty list implies a valid value. An > additional > nicety for this interface would be to define a type which the definition > would expect of the underlying value to be at runtime. > - Introduce an Extendable interface that indicates that a type can > have its data extended with a set of key-value pairs. I would define the > following methods: > - Set<ExtenstionPropertyDefinition> > getExtenstionPropertyDefinitions(): The set of extension property > descriptions > - Object getExtensionPropertyValue(String aPropertyName): Get a > value for an extension property > - List<String> validateExtensionProperties(): Validates the values > of the extension properties > > > There are likely pieces I missed and refinements to improve this design — > I jotted it down off the top of my head. However, we have to ensue that > the semantic is rich enough to render friendly UIs, as well as, protect > against GIGO (garbage in, garbage out). > > Finally, I would like to see the design expanded to explain when and where > this information will pass into drivers for use. As the design stands now, > it does not explain how the data would actually be used during operation. > The scope of these changes is important to understand for both impact > analysis and driver developers and implementors. Also, I don’t understand > how the addition of generic key-values to drivers would eliminate tags. As > I understand them, tags are used to establish valid combinations of drivers > (e.g. mutual exclusion) to avoid defining an unimplementable infrastructure > configuration. While I agree with the desire to eliminate them, I don’t > understand how such a facility would contribute to that goal. > > Great work putting together an initial design, and getting this important > conversation started. > > Thanks, > -John > > On Jun 10, 2014, at 6:26 PM, Mike Tutkowski <mike.tutkow...@solidfire.com> > wrote: > > Hi, > > Please feel free to review the following proposal for 4.5: > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=42566111 > > Here is the summary: > > Today the way you associate a Compute Offering (CO) or a Disk Offering > (DO) with a Primary Storage (PS) is via storage tagging. > > This has some benefits and drawbacks. > > One benefit is being able to have some level of vendor independence from > the point of view of the CO or DO. For example, if the storage tag of a DO > is "Fast", then this can be satisfied by PS that describes itself as > "Fast", regardless of vendor. > > A major drawback with the storage-tagging approach, however, is that you > are not easily able to leverage vendor-specific features, which is often > why you bought storage from the vendor in question to begin with. > > Ideally we do not want to add each vendor's features into the system as > properties that can be seen by the admin regardless of whether or not the > underlying storage he's actually using supports the feature in question. > Traditionally, however, this has been business as usual in the CloudStack > codebase. > > Going forward, we want to implement a more fine-grain and generic approach. > > For example, in the GUI we would like to have a storage provider field for > the CO and DO windows (this equates to the name of one and only one storage > provider). If the admin inputs a specific storage provider, he can enter in > an arbitrary number of key/value pairs in another text field (perhaps we > would provide a nice entry dialog to make this easier in the GUI). These > key value pairs can be passed into the storage driver when it's asked to > create (or update) a volume and the storage driver can decide what each and > every key/value pair means, if anything. > > Thanks! > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the cloud > <http://solidfire.com/solution/overview/?video=play>*™* > > > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud <http://solidfire.com/solution/overview/?video=play>*™*
