Comments embedded in below. From: Vladimir Popovski [mailto:vladi...@zadarastorage.com] Sent: Wednesday, March 27, 2013 12:22 PM To: Mike Tutkowski; Edison Su Cc: cloudstack-...@incubator.apache.org Subject: RE: Storage Subsystem 2.0 plugin docs
Hi All, I was away for couple of days, so sorry for the delay. I'm completely with Mike & John (& others) on separating storage plugins from hypervisor-related functions. If every plugin will need to implement hypervisor-related code, it will be a big mess. Our use-case is very similar to Mike's - we would like to be able to provision volumes with different QoS characteristics directly to VMs, rather than adding them into "shared" datastores. It might be achieved in two ways: - either to create separate data stores per each volume (storage LUN), and from there to create volumes & assign to instances. - or to assign devices recognized by iSCSI Initiators directly to instances (I'm not sure if it will be possible in VMware without datastores) It looks like Mike started to work on the 1st approach. In this case, for every volume there will be a separate datastore. If this is the preferred direction for all storage plugins, then the hypervisor-specific logic of datastore creation and creating/assigning volumes from the datastore will be fairly common for all storage plug-ins. At the same time, the storage plugin should have the control over the datastore management. It will depend on the plugin, if dedicated or shared datastores should be created. For the 2nd option (skipping the datastore layer) there might be plenty of common code as well. So, my questions are: - do you think it is beneficial to support both options for the CS (or are we good with potentially huge number of datastores)? [Edison] CloudStack will not enforce any of these options, it's all up to provider's implementation. Either way is OK to me. Do you think, from architecture point of view, Is the current storage API enough for both options? If no, we can come up some new APIs. - are we all agree that HV-dependent storage code should be generic and appropriate interfaces to be exposed? [Edison] If there are HV-dependent storage code there(I assume it should have some code can be shared between different storage provider at hypervisor side), we can generalize them and expose them. As Mike said, the code dealing with storage pool at the hypervisor side, can be shared. Regards, -Vladimir From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>] Sent: Wednesday, March 27, 2013 10:21 AM To: Edison Su Cc: cloudstack-...@incubator.apache.org<mailto:cloudstack-...@incubator.apache.org>; Vladimir Popovski Subject: Re: Storage Subsystem 2.0 plugin docs Sounds good, Edison Last night I finished up code that uses the VI Java API to create a VMware Datastore. I want to test it a bit more before I have you look at it. Today there is a Citrix CloudPlatform demo I'm participating in to handle part of the SolidFire section of the demo, so I might not have time to do my Datastore testing, but I should be done with it tomorrow. Talk to you later! On Wed, Mar 27, 2013 at 11:15 AM, Edison Su <edison...@citrix.com<mailto:edison...@citrix.com>> wrote: For vmware, current cloudstack doesn't create a vmware datastore through vmware's API, admin needs to create the datastore in Vcenter at first, then add it back into cloudstack. I am not familiar with how to create a VMware datastore through Vmware's API, but regarding to add a new host into a cluster, the current framework lets storage provider adding a listener which can listen on adding host event. From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>] Sent: Tuesday, March 26, 2013 6:45 PM To: Edison Su Cc: cloudstack-...@incubator.apache.org<mailto:cloudstack-...@incubator.apache.org>; Vladimir Popovski Subject: Re: Storage Subsystem 2.0 plugin docs Great - thanks, Edison! I can take a look at that code. I've almost gotten the VMware code written. It's a little more involved than the XenServer code because you have to add static IQNs for discovery to each host in a VMware cluster (this is somehow handled behind the scenes, I suppose, with XenServer) before you can create a Datastore based on your iSCSI target. One thing I was wondering, though, is when you add a new host to this VMware cluster. It will need to "inherit" the list of IQNs to discover. I image this is the case today. Do you know anything about that? I might just try it out and see if that works today. On Tue, Mar 26, 2013 at 5:18 PM, Edison Su <edison...@citrix.com<mailto:edison...@citrix.com>> wrote: Thanks! FYI, there are some code at both xen and kvm hypervisor resource code to deal with storage pool creation. For example, in CitrixResourceBase-> getNfsSR or getIscsiSR to create a nfs SR or ISCSI SR. In LibvirtStorageAdaptor, which can create storage pool in libvirt. From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>] Sent: Tuesday, March 26, 2013 1:52 PM To: Edison Su Cc: cloudstack-...@incubator.apache.org<mailto:cloudstack-...@incubator.apache.org>; Vladimir Popovski Subject: Re: Storage Subsystem 2.0 plugin docs Hi Edison, Sounds good. I already have code to create a XenServer Storage Repository (and optionally use CHAP credentials) and I'm working right now on creating a vSphere Datastore. When I have this working and in a nicer state, I can check in with you to review it and provide comments. Once those two hypervisors are handled, I'll move on to KVM and OVM. Thanks! On Tue, Mar 26, 2013 at 2:33 PM, Edison Su <edison...@citrix.com<mailto:edison...@citrix.com>> wrote: Yes, code is welcome!!! Currently Only the interface at the management server side is defined. At the hypervisor resource side, we may need some kind of utility library or another plugin framework, as John proposed few months ago. From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>] Sent: Monday, March 25, 2013 2:37 PM To: Edison Su; cloudstack-...@incubator.apache.org<mailto:cloudstack-...@incubator.apache.org>; Vladimir Popovski Subject: Re: Storage Subsystem 2.0 plugin docs Hey Edison, So...if you think my understanding is correct (please check out the e-mail below), then I have a question. Do we really want to have the storage plug-ins taking on the responsibility of talking to the hypervisors to hook up the storage that they just created? I'm a bit familiar with how OpenStack does this and it seems that it only has its storage plug-ins create a volume (LUN, whatever) and then the framework handles the process of dealing with the hypervisor in question to hook up the storage. It seems like otherwise we'd need to create a utility for all storage plug-ins to share otherwise they'd be duplicating efforts in talking to hypervisors. What do you think? On Thu, Mar 21, 2013 at 7:52 PM, Mike Tutkowski <mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>> wrote: Hi Edison, I believe I understand the requirements for the plug-in better now. It sounds like the flow will be as such: * The user executes a Compute or Disk Offering that is tied via a storage tag to a Primary Storage that is associated with a plug-in. * The storage framework will ask the plug-in to create a volume. The plug-in will create a volume and hook the volume up to the appropriate hypervisor. For VMware, this means the plug-in will create a Datastore. For XenServer, this means the plug-in will create a Storage Repository. (So on and so forth for other hypervisors.) * The VM or data disk is then deployed to the hypervisor. Does that sound correct, Edison? Thanks! On Thu, Mar 21, 2013 at 5:44 PM, Edison Su <edison...@citrix.com<mailto:edison...@citrix.com>> wrote: From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>] Sent: Thursday, March 21, 2013 4:18 PM To: Edison Su Subject: Re: Storage Subsystem 2.0 plugin docs Hi Edison, I wanted to dive into these comments a bit more: [Edison] plugin's driver->createasync will be called when mgt server want to create a volume on the storage. In the driver's implementation, it can directly call storage box's api, or send a command to hypervisor host, then call storage box's api to create an iscsi. Then create a datastore(for vmware), SR(for xenserver), or storage pool(for KVM) on hypervisor host, based on the iscsi iqn. If the volume is created from a template(for root disk), need to find a way to import that template(which is nfs based currently, it will be just a plain http url the future) into the root disk. The part about creating a datastore or a storage repository...is that something the plug-in will be responsible for doing or will the storage framework cover that piece? I'm thinking the storage framework will since all sorts of plug-ins would seem to need that ability (to have their storage hooked up to a datastore or a storage repository). [Edison] It's a specific requirement for per volume per LUN case, and specific for certain hypervisors(seems KVM doesn't need to create a storage pool when using iscsi LUN), so the storage framework will not deal with it right now. Thanks for your time, Edison! :) On Thu, Mar 21, 2013 at 4:45 PM, Edison Su <edison...@citrix.com<mailto:edison...@citrix.com>> wrote: Feedback/comments are appreciated, need to know your input from storage vendor point of view. From: Vladimir Popovski [mailto:vladi...@zadarastorage.com<mailto:vladi...@zadarastorage.com>] Sent: Thursday, March 21, 2013 11:52 AM To: Edison Su; cloudstack Cc: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> Subject: RE: Storage Subsystem 2.0 plugin docs Hi Edison, Thank you for the reply. We will check it out. Regards, -Vladimir From: Edison Su [mailto:edison...@citrix.com<mailto:edison...@citrix.com>] Sent: Thursday, March 21, 2013 11:36 AM To: 'Vladimir Popovski'; cloudstack Cc: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> Subject: RE: Storage Subsystem 2.0 plugin docs From: Vladimir Popovski [mailto:vladi...@zadarastorage.com] Sent: Wednesday, March 20, 2013 9:05 AM To: cloudstack Cc: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com>; Edison Su Subject: Storage Subsystem 2.0 plugin docs Hi All, Thank you for a great work on CloudStack! We are interested in integrating CS with our storage system and started to look at your documentation and storage-related code. I see that Mike from SolidFire started working on something similar some time ago and Edison even created an empty plugin for it (in Nov'12?). We have couple of questions related to that: - Is there any documentation about plugins (except of https://cwiki.apache.org/CLOUDSTACK/storage-subsystem-20.html) [Edison] There are not much docs about the plugins other than the above link. See below. - Are there any exemplary plugins for primary & secondary datastores? Was the SolidFire plugin ever finished? [Edison] yesterday, I checked in some code to separate existing cloudstack storage code into a standalone maven project, called: cloud-plugin-storage-volume-default, which can give you an example how a storage plugin will look like. - How to activate a new plugin and use it (at least through CLIs/APIs) [Edison] First, put a bean configuration in client/tomcatconf/componentContext.xml.in<http://componentContext.xml.in> for your plugin provider class, like: <bean id="ClassicalPrimaryDataStoreProvider" class="org.apache.cloudstack.storage.datastore.provider.CloudStackPrimaryDataStoreProviderImpl"> </bean> Second, when adding a data store into cloudstack, with an extra parameter in createstoragepoolcmd: provider=your-provider-name, liststorageproviderscmd can list all the registered providers in mgt server. - How to integrate it with the UI There is no UI part of example code for storage yet, the idea is to use pluggable UI(https://cwiki.apache.org/confluence/display/CLOUDSTACK/UI+Plugin+Tutorial), for each storage provider may need a separate UI to add a storage. For example, in adding primary storage ui, there will be a drop down list, show all the registered providers, if user selects one of the drop down list, then UI will pop up a diagram, based on providers' pluggable ui, then user can type whatever information needed for a storage(e.g. nfs server, nfs path, if its nfs). At the end, UI will call createstoragepoolcmd to register a storage into cloudstack. Thanks, -Vladimir ------- Vladimir Popovski VP, Cloud Operations Zadara Storage (949) 677-2095<tel:%28949%29%20677-2095> vladi...@zadarastorage.com<mailto:vladi...@zadarastorage.com> www.zadarastorage.com<http://www.zadarastorage.com/> -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc. e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> o: 303.746.7302<tel:303.746.7302> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm) -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc. e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> o: 303.746.7302<tel:303.746.7302> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm) -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc. e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> o: 303.746.7302<tel:303.746.7302> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm) -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc. e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> o: 303.746.7302<tel:303.746.7302> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm) -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc. e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> o: 303.746.7302<tel:303.746.7302> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm) -- Mike Tutkowski Senior CloudStack Developer, SolidFire Inc. e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>(tm)
