Hi Mike, The idea of introducing a new API: StorageSnapshot for managed storage is because the VolumeSnapshot default, or expected, behavior is to archive snapshots into the Secondary Storage. So a StorageSnapshot API would be for snapshot that remain on the managed storage appliance.
Quickly looking at the API doc and I don't see a strong requirement for volume snapshots to be moved into secondary storage. So, maybe StorageSnapshot API is not useful, but both use cases are required. A snapshot that remain on the managed storage, and another type of snapshot that end up into the secondary storage. Since you've done a lot of work, might easier to just add a parameter to the current snapshot API that would trigger an extraction of the storage snapshot into the secondary storage? PL On Thu, Feb 4, 2016 at 9:02 PM, Mike Tutkowski <mike.tutkow...@solidfire.com > wrote: > I think that all sounds reasonable then - thanks! > > On Thu, Feb 4, 2016 at 6:52 PM, Syed Mushtaq <syed1.mush...@gmail.com> > wrote: > >> You are correct Mike in terms of the requirements. One of our earlier >> iterations on this was to have an argument to the create snapshot API which >> decides whether to backup the volume to sec storage but we realized it >> would make management of snapshots quite messy so we proposed a new api >> instead. >> >> On Thu, Feb 4, 2016, 8:29 PM Mike Tutkowski <mike.tutkow...@solidfire.com> >> wrote: >> >>> Hi, >>> >>> Just to make sure I understand all the requirements here: >>> >>> 1) This relates only to managed storage (1:1 mapping between a virtual >>> disk and a backend SAN volume). >>> >>> 2) We want to take the current (introduced in 4.6) functionality, which >>> creates a snapshot on the SAN, and extend it via a config option (or >>> something) to not only take the SAN snapshot, but to copy the underlying >>> VHD (XenServer only) to NFS. >>> >>> 3) The SAN snapshot is always taken. It's the backup to NFS that is >>> optional. >>> >>> 4) Templates can be created from the snapshot that's on the SAN (already >>> works). >>> >>> 5) CloudStack volumes can be created from the snapshot that's on the SAN >>> (already works as long as the new CloudStack volume ends up on the same >>> primary storage). >>> >>> Would we have a need for a storage snapshot API then or would that just >>> be the standard volume snapshot without the backup to NFS? >>> >>> Thanks! >>> Mike >>> >>> On Thu, Feb 4, 2016 at 5:24 PM, Syed Mushtaq <syed1.mush...@gmail.com> >>> wrote: >>> >>>> Is it possible to have both functionalities (snapshot on SAN & Sec >>>> Storage) coexist? Because Ideally, we would like to have both. >>>> For example, some of our customers want to implement their own backup >>>> strategies and do encryption to their backups which is a perfect >>>> use case for Storage Snapshot while our other customers will still keep >>>> using the standard volume snapshot. >>>> >>>> To keep things backward compatible, we can add a setting which says to >>>> not upload on secondary storage, because, after all, you would take a >>>> SAN snapshot first when doing a Volume Snapshot. You could stop the >>>> process there and not do the upload. >>>> >>>> What do you think about this approach? >>>> >>>> Thanks, >>>> -Syed >>>> >>>> On Thu, Feb 4, 2016 at 4:25 PM, Mike Tutkowski < >>>> mike.tutkow...@solidfire.com> wrote: >>>> >>>>> So, this is just me thinking out load here, but if a given CloudStack >>>>> cloud doesn't actually need to provide both the ability to take a SAN >>>>> snapshot and export it to NFS (if just taking a SAN snapshot is OK), then >>>>> we might be able to get away with no new API calls and simply implement a >>>>> new custom snapshot strategy and data motion strategy to handle the case >>>>> where the CloudStack cloud does want both a SAN snapshot and >>>>> exported-to-NFS backup. >>>>> >>>>> In other words, the "default" behavior would be to use the snapshot >>>>> strategy and data motion strategy that we already have (the one that only >>>>> takes a SAN snapshot). >>>>> >>>>> If your CloudStack cloud, however, wants to take a SAN snapshot and >>>>> have the data exported to NFS, then we could have you manipulate a Swing >>>>> config file to make use of a new snapshot strategy and data motion >>>>> strategy >>>>> that performs both of these activities. >>>>> >>>>> This way, the old behavior is still the default for users, but >>>>> CloudStack admins can change this behavior via configuration. >>>>> >>>>> Thoughts? >>>>> >>>>> On Thu, Feb 4, 2016 at 11:55 AM, Mike Tutkowski < >>>>> mike.tutkow...@solidfire.com> wrote: >>>>> >>>>>> Right...I think we will need to come up with a viable upgrade path or >>>>>> some reasonable way for them to move from the old way to the new way (and >>>>>> some obvious way that they will know they need to do this). >>>>>> >>>>>> On Thu, Feb 4, 2016 at 11:45 AM, Syed Mushtaq < >>>>>> syed1.mush...@gmail.com> wrote: >>>>>> >>>>>>> I'm not really sure about the upgrade path however, customers who >>>>>>> are using 4.6 and are on a managed storage would no longer have the same >>>>>>> functionality with Volume Snapshots. >>>>>>> >>>>>>> On Thu, Feb 4, 2016 at 1:43 PM, Syed Mushtaq < >>>>>>> syed1.mush...@gmail.com> wrote: >>>>>>> >>>>>>>> So if I understand correctly, currently taking a Volume Snapshots >>>>>>>> of a volume on a managed storage keeps it on the storage array. As a >>>>>>>> part >>>>>>>> of this feature, we can make sure that Volume Snapshots on managed >>>>>>>> storage >>>>>>>> are uploaded to the secondary storage. This would make the Volume >>>>>>>> Snapshot >>>>>>>> feature behave the same regardless of the storage (managed or >>>>>>>> non-managed) >>>>>>>> And, for utilizing the efficient backend storage capabilities, we can >>>>>>>> use >>>>>>>> the new storage snapshots API. >>>>>>>> >>>>>>>> On Thu, Feb 4, 2016 at 1:36 PM, Mike Tutkowski < >>>>>>>> mike.tutkow...@solidfire.com> wrote: >>>>>>>> >>>>>>>>> Hi everyone, >>>>>>>>> >>>>>>>>> Whatever we do here, we need to have a plan to deal with the fact >>>>>>>>> that we already have a feature (in 4.6 and later) that allows you to >>>>>>>>> use >>>>>>>>> the existing volume-snapshot APIs to create a volume snapshot (for >>>>>>>>> managed >>>>>>>>> storage) that resides on a backend SAN (using a custom snapshot >>>>>>>>> strategy >>>>>>>>> and a custom data motion strategy). >>>>>>>>> >>>>>>>>> If these new APIs go in, then how should the original >>>>>>>>> implementation (present in 4.6 and later) be changed? If it is >>>>>>>>> changed, how >>>>>>>>> do we support customers who are already using the original >>>>>>>>> volume-snapshot >>>>>>>>> API to take snapshots on a backend SAN? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Mike >>>>>>>>> >>>>>>>>> On Thu, Feb 4, 2016 at 11:27 AM, Will Stevens < >>>>>>>>> wstev...@cloudops.com> wrote: >>>>>>>>> >>>>>>>>>> Will you be able to create a Template from a StorageSnapshot? If >>>>>>>>>> yes, will the template be stored in the secondary storage like normal >>>>>>>>>> templates or will that be handled somehow on the vendor side? >>>>>>>>>> >>>>>>>>>> *Will STEVENS* >>>>>>>>>> Lead Developer >>>>>>>>>> >>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts >>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 >>>>>>>>>> w cloudops.com *|* tw @CloudOps_ >>>>>>>>>> >>>>>>>>>> On Thu, Feb 4, 2016 at 1:22 PM, Syed Mushtaq < >>>>>>>>>> syed1.mush...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Will!!! >>>>>>>>>>> >>>>>>>>>>> On Thu, Feb 4, 2016 at 1:19 PM, Will Stevens < >>>>>>>>>>> wstev...@cloudops.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> I explicitly linked the Design Spec in the Jira ticket because >>>>>>>>>>>> it was not clear in the 'mention' section because it shows as a >>>>>>>>>>>> page 'you >>>>>>>>>>>> do not have permission to'. >>>>>>>>>>>> >>>>>>>>>>>> *Will STEVENS* >>>>>>>>>>>> Lead Developer >>>>>>>>>>>> >>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts >>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 >>>>>>>>>>>> w cloudops.com *|* tw @CloudOps_ >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Feb 4, 2016 at 1:02 PM, Syed Ahmed <sah...@cloudops.com >>>>>>>>>>>> > wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Design Spec: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/StorageSnapshot++API >>>>>>>>>>>>> >>>>>>>>>>>>> Jira Ticket >>>>>>>>>>>>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-9278 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> We plan to propose a new set of APIs to do snapshots on >>>>>>>>>>>>> managed storage backends like SolidFire. Snapshots on current >>>>>>>>>>>>> managed >>>>>>>>>>>>> storage stay on the >>>>>>>>>>>>> device which is contrary to what CloudStack calls snpshots. >>>>>>>>>>>>> But taking snapshots >>>>>>>>>>>>> on storage and keeping it there has its own advantages and we >>>>>>>>>>>>> would ideally like >>>>>>>>>>>>> to have both ways of doing snapshots. This proposal adds 4 new >>>>>>>>>>>>> APIs to >>>>>>>>>>>>> create snapshots on backend storage. >>>>>>>>>>>>> >>>>>>>>>>>>> What do you guys think of this feature? I would love to have >>>>>>>>>>>>> some feedback. I am working on making the design spec more >>>>>>>>>>>>> concrete but >>>>>>>>>>>>> wanted to have >>>>>>>>>>>>> a high level feedback first before starting to work on it. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> -Syed >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Mike Tutkowski* >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>> e: mike.tutkow...@solidfire.com >>>>>>>>> o: 303.746.7302 >>>>>>>>> Advancing the way the world uses the cloud >>>>>>>>> <http://solidfire.com/solution/overview/?video=play>*™* >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Tutkowski* >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>> e: mike.tutkow...@solidfire.com >>>>>> o: 303.746.7302 >>>>>> Advancing the way the world uses the cloud >>>>>> <http://solidfire.com/solution/overview/?video=play>*™* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Mike Tutkowski* >>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>> e: mike.tutkow...@solidfire.com >>>>> o: 303.746.7302 >>>>> Advancing the way the world uses the cloud >>>>> <http://solidfire.com/solution/overview/?video=play>*™* >>>>> >>>> >>>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkow...@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the cloud >>> <http://solidfire.com/solution/overview/?video=play>*™* >>> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the cloud > <http://solidfire.com/solution/overview/?video=play>*™* >
