Hi Mike, The reason behind the creation of a SAN snapshot which is exported into secondary storage, is because creating a copy of the .VHD directly would impact uptime of the VM as creating that copy take lots of time. Has oppose to a SAN snapshot that is close to instantaneous which can afterward be clone into Secondary Storage asynchronously.
I would suspect an extracted VolumeSnapshot taken from a SAN snapshot could have is SAN snapshot deleted to avoid duplica and space consumption on the Primary Storage side. I see 3 definitions in our current discussion regarding the term snapshot (these are not official terminology but by own interpretation of them): 1. *Snapshot* (AKA: Storage Snapshot / Mike's definition of a snapshot): it's a volume snapshot at the storage level, point in time of your data. it reside on the primary storage. Useful and efficient for software side incident. 2. *Cloud Snapshot *( AKA: CloudStack VolumeSnapshot/ cloud backup aws-S3 style ): Point in time copy of the Virtual Disk that reside on a different storage array then the original Volume. Facilitate data migration between clusters and, in case of primary storage incident, Volume snapshots are not impacted and can be reuse. 3. *Backup*: Archival of your Virtual-machines data that also validate data integrity, provide a storage efficient archiving method and an independent way to restore your data in case of an major infrastructure disaster. Regards, PL On Fri, Feb 5, 2016 at 1:34 PM, Mike Tutkowski <mike.tutkow...@solidfire.com > wrote: > So, let's see if I currently follow the requirements: > > * Augment volume snapshots for managed storage to conditionally export data > to NFS. The current process of taking a snapshot on the SAN is fine, but > we'd like the option to export the data to NFS, as well. > > Questions: > > Once the data has been exported to NFS, do we keep the SAN snapshot or > delete it? > > If we are deleting the SAN snapshot, then why don't we just copy the VHD > from primary to secondary the way we do today for non-managed (i.e. > traditional) storage? Why create a SAN snapshot in that scenario? Perhaps > to have the SSVM mount and perform the VHD copy to secondary storage > instead of a XenServer host? > > Thanks for the clarification. > > By the way, to me a backup is when you copy data from one storage system to > another (regardless of features, if any, to restore the data in the > future). A snapshot is a point-in-time view of the data of a volume and > it's stored on the same storage system as the volume. > > On Fri, Feb 5, 2016 at 11:09 AM, Pierre-Luc Dion <pd...@cloudops.com> > wrote: > > > That's fun to see that discussion happening. I 100% agree with Paul's > > points of view. VolumeSnapshot are not a backup, but I do consider them > as > > a safety vest against Primary Storage failure, because failure append > :-( . > > > > The current proposal around snapshots that reside on the primary storage > or > > snapshots that end in the Secondary Storage is not to address any kind > of > > backups requirement because a snapshot is not a backup, event an > extracted > > VM snapshot. > > > > The main idea, and again this is for managed storage; > > > > 1. StorageSnapshotAPI: Provide storage side snapshot capability for fast > > response time that support rollback to previous timestamp, create new > > volume and maybe create template. > > not required to be a new API if the work is already done, I think > this > > is a different behaviors than the user expectation of a volume-snapshot. > > 2. VolumeSnapshotAPI: Provide current cloudstack behavior that create an > > extraction of a volume into SecondaryStorage which can be reuse to > create a > > new volume into another Primary Storage. This type of snapshot is a slow > > job since yes it would have to copy the full volume size on the Secondary > > storage. > > > > > > PL > > > > > > > > On Fri, Feb 5, 2016 at 12:45 PM, Syed Mushtaq <syed1.mush...@gmail.com> > > wrote: > > > > > I think I share you view on the 'Ideal world'. Backup (via Volume > > > Snapshots) is a huge bottleneck in Cloudstack. This is amplified > > especially > > > when you have a object storage as your secondary storage because it > > > requires two copies (one to an NFS staging area and from there to > object > > > storage). And not to mention that all these copies are consuming > > hypervisor > > > resources. Xenserver's Dom0 is also a huge bottleneck as all the > Network > > > and I/O flow through it. So our intention of proposing the "Storage > > > Snapshots" is to give a better way of achiving snapshots while still > > > keeping the original definition of volume snpashots (ie upload to sec > > > storage). > > > > > > But as Erik pointed out volume snapshots are not backups. They don't > work > > > form multi-disk LVM volume groups and dynamic disks. I am all in for a > > > better backup solution which handles these use cases and utilizes the > > > storage's advanced features. > > > > > > > > > > > > On Fri, Feb 5, 2016 at 12:29 PM, Paul Angus <paul.an...@shapeblue.com> > > > wrote: > > > > > > > In the beginning... there were CloudStack snapshots and they were > > > actually > > > > volume snapshots not hypervisor point-in-time snapshots. > > > > Then VM snapshots were created (which are point-in-time hypervisor > > > > snapshots) and we started referring to the original snapshots as > volume > > > > snapshots. > > > > > > > > CloudStack does not offer 'backups', but many people use volume > > snapshots > > > > as backups. However you can't in-place restore volume snapshots and > if > > > you > > > > have a VM with multiple volumes, the volume snapshots must be done in > > > > series, meaning that the state across all of the volumes is unlikely > to > > > be > > > > consistent. > > > > > > > > 'Actual Backups' would enable all of the restore options which users > > > might > > > > expect as well options as to where they might be stored. In my ideal > > > world > > > > they would also be able to leverage back-end hardware (such as > > Solidfire, > > > > NetApp etc :) ) and software such as Veeam, Commvault etc to > accelerate > > > the > > > > process. > > > > > > > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> > > > > Paul Angus > > > > VP Technology , ShapeBlue > > > > d: *+44 203 617 0528 | s: +44 203 603 0540* > > > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> | m: > > > > *+44 7711 418784* <+44%207711%20418784> > > > > e: *paul.an...@shapeblue.com | t: @cloudyangus* > > > > <paul.an...@shapeblue.com%20%7C%20t:%20@cloudyangus> | w: > > > > *www.shapeblue.com* <http://www.shapeblue.com> > > > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK > > > > Shape Blue Ltd is a company incorporated in England & Wales. > ShapeBlue > > > > Services India LLP is a company incorporated in India and is operated > > > under > > > > license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a > > > > company incorporated in Brasil and is operated under license from > Shape > > > > Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The > Republic > > of > > > > South Africa and is traded under license from Shape Blue Ltd. > ShapeBlue > > > is > > > > a registered trademark. > > > > This email and any attachments to it may be confidential and are > > intended > > > > solely for the use of the individual to whom it is addressed. Any > views > > > or > > > > opinions expressed are solely those of the author and do not > > necessarily > > > > represent those of Shape Blue Ltd or related companies. If you are > not > > > the > > > > intended recipient of this email, you must neither take any action > > based > > > > upon its contents, nor copy or show it to anyone. Please contact the > > > sender > > > > if you believe you have received this email in error. > > > > > > > > > > > > -----Original Message----- > > > > From: Syed Mushtaq [mailto:syed1.mush...@gmail.com] > > > > Sent: Friday, February 5, 2016 4:58 PM > > > > To: dev@cloudstack.apache.org > > > > Subject: Re: [Propose][New Feature] Storage Snapshots > > > > > > > > Paul, > > > > > > > > When you say actual backups, how would it be different from the > Volume > > > > Snapshots that exist currently. My understanding is that Backups end > up > > > in > > > > Sec Storage whereas Snapshots are just a point-in-time state of your > > > volume > > > > which can be restored back correct? > > > > > > > > -Syed > > > > > > > > > > > > On Fri, Feb 5, 2016 at 11:23 AM, Paul Angus < > paul.an...@shapeblue.com> > > > > wrote: > > > > > > > > > Hi Syed, > > > > > > > > > > As I understand it, the SolidFire plugin will export the snapshot > to > > > > > secondary storage if the user requests a template from the snapshot > > or > > > > > wants to download the snapshot from the cloud. This is a good, > > > > > pragmatic approach and yes Mike the SolidFire storage is super > > > > > reliable and snapshots on SolidFire arrays take up next to no > space. > > > > > BUT I think that we are talking about a more general purpose API, > and > > > > > other storage systems may not be as awesome as Mike's. That's my > > > > > concern. Also, the time to transfer for say 1TB to move from > primary > > > > > to sec storage and then create a VM template out of it may be too > > long > > > > for users. > > > > > > > > > > @Mike I don’t think 'we' use the term volume snapshot for backup, > > it's > > > > > just that users want to do backups and a volume snapshot is the > only > > > > > type of snapshot that copies the disk elsewhere and can be > scheduled. > > > > > > > > > > I'm 'pondering' the implications of enabling actual backups > (through > > > > > recognised backup providers) and the user requirements around them > > > > > (particularly restoration use cases) as a separate thread of work. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> Paul Angus VP > > Technology > > > > > , ShapeBlue > > > > > d: *+44 203 617 0528 | s: +44 203 603 0540* > > > > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> | m: > > > > > *+44 7711 418784* <+44%207711%20418784> > > > > > e: *paul.an...@shapeblue.com | t: @cloudyangus* > > > > > <paul.an...@shapeblue.com%20%7C%20t:%20@cloudyangus> | w: > > > > > *www.shapeblue.com* <http://www.shapeblue.com> > > > > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK Shape Blue > Ltd > > > > > is a company incorporated in England & Wales. ShapeBlue Services > > India > > > > > LLP is a company incorporated in India and is operated under > license > > > > > from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a > company > > > > > incorporated in Brasil and is operated under license from Shape > Blue > > > > > Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic > of > > > > > South Africa and is traded under license from Shape Blue Ltd. > > > > > ShapeBlue is a registered trademark. > > > > > This email and any attachments to it may be confidential and are > > > > > intended solely for the use of the individual to whom it is > > addressed. > > > > > Any views or opinions expressed are solely those of the author and > do > > > > > not necessarily represent those of Shape Blue Ltd or related > > > > > companies. If you are not the intended recipient of this email, you > > > > > must neither take any action based upon its contents, nor copy or > > show > > > > > it to anyone. Please contact the sender if you believe you have > > > received > > > > this email in error. > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Syed Mushtaq [mailto:syed1.mush...@gmail.com] > > > > > Sent: 05 February 2016 15:31 > > > > > To: dev@cloudstack.apache.org > > > > > Subject: Re: [Propose][New Feature] Storage Snapshots > > > > > > > > > > I think the terminology confusion comes from AWS where they do EBS > > > > > snapshots backed up to S3 and CloudStack sort of followed that. And > > as > > > > > an end user who is oblivious to the internals of my provider, my > > > > > expectation would be something similar to what AWS as that is my > > > > > biggest reference point. > > > > > > > > > > To your point Mike, I agree that a Primary Storage failure on > > > > > SolidFire is unlikely, there are other motivations for us to push > > data > > > > > to secondary storage. Primary storage (atleast for us) costs > around 3 > > > > > times as much as secondary storage and snapshots on primary storage > > > > > are rarely used (especially for some of our customers who do daily > > > > backups). > > > > > > > > > > > > > > > On Fri, Feb 5, 2016 at 10:18 AM, Mike Tutkowski < > > > > > mike.tutkow...@solidfire.com> wrote: > > > > > > > > > > > Some of the weirdness is around terminology. > > > > > > > > > > > > For most systems I've worked on, a snapshot and a backup are two > > > > > > completely different things (but CloudStack has traditionally > used > > > > > > the term "volume snapshot" to mean backup). > > > > > > > > > > > > I will put in a SolidFire "plug" here and say, though, that if > your > > > > > > primary storage is running on SolidFire that it is unlikely > you'll > > > > > > encounter an issue where your primary storage goes offline (and > > > > > > you'll even maintain your performance guarantees during failure > > > > > > scenarios and upgrades, as well). That being the case, it is less > > > > > > useful to require a backup to Swift (but it's perfectly OK if > > that's > > > > > > what we want to do > > > > > here). > > > > > > > > > > > > On Fri, Feb 5, 2016 at 8:07 AM, Syed Mushtaq > > > > > > <syed1.mush...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Paul, > > > > > > > > > > > > > > I believe with the current implementation of Snapshots on > managed > > > > > > > storage > > > > > > > (SolidFire) the snapshots are never exported to the secondary > > > > storage. > > > > > > > While this solves the problem of having snapshots taking > forever > > > > > > > to get to sec storage, this leaves us with a > > > > > > huge > > > > > > > liability if our primary storage goes down. We see snapshots as > > > > > > > our recovery path because we store them in Swift which is > > reliable > > > > > > > and resilient to failures. > > > > > > > > > > > > > > With Storage snpashots our goal is to have Volume snapshots > > always > > > > > > > backed up to secondary storage and Storage Snapshots stay on > the > > > > > > > primary > > > > > > storage. > > > > > > > A provider could potentially mix both these and solve the > problem > > > > > > > that you mentioned where you want to meet user's expectation > of a > > > > > > > snapshot (ie backup to sec storage) while having an ability to > > > > > > > utilize faster sanpshots (i.e. on the device) > > > > > > > > > > > > > > Hope this clarifies things. > > > > > > > > > > > > > > Thanks, > > > > > > > -Syed > > > > > > > > > > > > > > On Fri, Feb 5, 2016 at 9:04 AM, Paul Angus > > > > > > > <paul.an...@shapeblue.com> > > > > > > > wrote: > > > > > > > > > > > > > > > HI guys, > > > > > > > > > > > > > > > > Could someone point me to the Jira bug of FS for the > > > > > > > > SAN-snapshot > > > > > > feature > > > > > > > > in 4.6 which is mentioned. > > > > > > > > > > > > > > > > From my discussions with users and operators around snapshots > > > > > > > > I'd make > > > > > > > the > > > > > > > > following observations: > > > > > > > > a. 'users' use snapshots as backups (both long-term and short > > > > > > > > term) > > > > > > with > > > > > > > > the expectation that they can use them for recovery if > > required. > > > > > > > > b. operators fall back to snapshots if something has gone > wrong > > > > > > > > with primary storage. > > > > > > > > c. users sometimes want to be able to export snapshots as > well > > > > > > > > as > > > > > > create > > > > > > > > new VMs from their snapshots > > > > > > > > d. snapshots are a currently a massive pain for operators, I > > > > > > > > know at > > > > > > > least > > > > > > > > one public cloud who have snapshots which take 2 days to > > > complete. > > > > > > > > e. snapshots (as they are) can't be used for multiple LVM > > disks. > > > > > > > > > > > > > > > > I think the process Mike has used in the SolidFire plugin > (only > > > > > > > > moving > > > > > > > the > > > > > > > > disk image to secondary storage when you absolutely have to) > is > > > > > > > > a very > > > > > > > good > > > > > > > > and pragmatic solution. I wonder what problems an operator > > might > > > > > > > experience > > > > > > > > if they have an issue with a given primary storage pool in a > > > > cluster. > > > > > > (I > > > > > > > > know that that is REALLY unlikely in the SolidFire case Mike > :) > > > > > > > > ) And > > > > > > if > > > > > > > > the transfer from primary to secondary is slow, the time to > > > > > > > > being able > > > > > > to > > > > > > > > create a template or export the volume will be slow. > > > > > > > > > > > > > > > > So for me the issue is around making sure that the end users > > > > > > expectations > > > > > > > > are met (while improving the speed/efficiency of the back > end) > > > > > > > > > > > > > > > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> Paul Angus VP > > > > > > > > Technology , ShapeBlue > > > > > > > > d: *+44 203 617 0528 | s: +44 203 603 0540* > > > > > > > > <+44%20203%20617%200528%20%7C%20s:%20+44%20203%20603%200540> > | > > m: > > > > > > > > *+44 7711 418784* <+44%207711%20418784> > > > > > > > > e: *paul.an...@shapeblue.com | t: @cloudyangus* > > > > > > > > <paul.an...@shapeblue.com%20%7C%20t:%20@cloudyangus> | w: > > > > > > > > *www.shapeblue.com* <http://www.shapeblue.com> > > > > > > > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK Shape > > Blue > > > > > > > > Ltd is a company incorporated in England & Wales. ShapeBlue > > > > > > > > Services India LLP is a company incorporated in India and is > > > > > > > > operated > > > > > > > under > > > > > > > > license from Shape Blue Ltd. Shape Blue Brasil Consultoria > Ltda > > > > > > > > is a company incorporated in Brasil and is operated under > > > > > > > > license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a > company > > > > > > > > registered by The Republic > > > > > > of > > > > > > > > South Africa and is traded under license from Shape Blue Ltd. > > > > > > > > ShapeBlue > > > > > > > is > > > > > > > > a registered trademark. > > > > > > > > This email and any attachments to it may be confidential and > > are > > > > > > intended > > > > > > > > solely for the use of the individual to whom it is addressed. > > > > > > > > Any views > > > > > > > or > > > > > > > > opinions expressed are solely those of the author and do not > > > > > > necessarily > > > > > > > > represent those of Shape Blue Ltd or related companies. If > you > > > > > > > > are not > > > > > > > the > > > > > > > > intended recipient of this email, you must neither take any > > > > > > > > action > > > > > > based > > > > > > > > upon its contents, nor copy or show it to anyone. Please > > contact > > > > > > > > the > > > > > > > sender > > > > > > > > if you believe you have received this email in error. > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Pierre-Luc Dion [mailto:pd...@cloudops.com] > > > > > > > > Sent: Friday, February 5, 2016 12:56 PM > > > > > > > > To: dev@cloudstack.apache.org > > > > > > > > Subject: Re: [Propose][New Feature] Storage Snapshots > > > > > > > > > > > > > > > > Hi Mike, > > > > > > > > > > > > > > > > The idea of introducing a new API: StorageSnapshot for > managed > > > > > > > > storage > > > > > > is > > > > > > > > because the VolumeSnapshot default, or expected, behavior is > to > > > > > > > > archive snapshots into the Secondary Storage. So a > > > > > > > > StorageSnapshot API would be > > > > > > > for > > > > > > > > snapshot that remain on the managed storage appliance. > > > > > > > > > > > > > > > > Quickly looking at the API doc and I don't see a strong > > > > > > > > requirement for volume snapshots to be moved into secondary > > > > > > > > storage. So, maybe StorageSnapshot API is not useful, but > both > > > > > > > > use cases are required. A snapshot that remain on the managed > > > > > > > > storage, and another type of > > > > > > snapshot > > > > > > > > that end up into the secondary storage. Since you've done a > lot > > > > > > > > of > > > > > > work, > > > > > > > > might easier to just add a parameter to the current snapshot > > API > > > > > > > > that > > > > > > > would > > > > > > > > trigger an extraction of the storage snapshot into the > > secondary > > > > > > storage? > > > > > > > > > > > > > > > > PL > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 9:02 PM, Mike Tutkowski < > > > > > > > > mike.tutkow...@solidfire.com > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > I think that all sounds reasonable then - thanks! > > > > > > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 6:52 PM, Syed Mushtaq < > > > > > > syed1.mush...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> You are correct Mike in terms of the requirements. One of > > our > > > > > > earlier > > > > > > > > >> iterations on this was to have an argument to the create > > > > > > > > >> snapshot > > > > > > API > > > > > > > > >> which decides whether to backup the volume to sec storage > > but > > > > > > > > >> we realized it would make management of snapshots quite > > messy > > > > > > > > >> so we proposed a new api instead. > > > > > > > > >> > > > > > > > > >> On Thu, Feb 4, 2016, 8:29 PM Mike Tutkowski > > > > > > > > >> <mike.tutkow...@solidfire.com> > > > > > > > > >> wrote: > > > > > > > > >> > > > > > > > > >>> Hi, > > > > > > > > >>> > > > > > > > > >>> Just to make sure I understand all the requirements here: > > > > > > > > >>> > > > > > > > > >>> 1) This relates only to managed storage (1:1 mapping > > between > > > > > > > > >>> a virtual disk and a backend SAN volume). > > > > > > > > >>> > > > > > > > > >>> 2) We want to take the current (introduced in 4.6) > > > > > > > > >>> functionality, which creates a snapshot on the SAN, and > > > > > > > > >>> extend it via a config option (or > > > > > > > > >>> something) to not only take the SAN snapshot, but to copy > > > > > > > > >>> the underlying VHD (XenServer only) to NFS. > > > > > > > > >>> > > > > > > > > >>> 3) The SAN snapshot is always taken. It's the backup to > NFS > > > > > > > > >>> that is optional. > > > > > > > > >>> > > > > > > > > >>> 4) Templates can be created from the snapshot that's on > the > > > > > > > > >>> SAN (already works). > > > > > > > > >>> > > > > > > > > >>> 5) CloudStack volumes can be created from the snapshot > > > > > > > > >>> that's on > > > > > > the > > > > > > > > >>> SAN (already works as long as the new CloudStack volume > > ends > > > > > > > > >>> up on the same primary storage). > > > > > > > > >>> > > > > > > > > >>> Would we have a need for a storage snapshot API then or > > > > > > > > >>> would that just be the standard volume snapshot without > the > > > > > > > > >>> backup to > > > > > NFS? > > > > > > > > >>> > > > > > > > > >>> Thanks! > > > > > > > > >>> Mike > > > > > > > > >>> > > > > > > > > >>> On Thu, Feb 4, 2016 at 5:24 PM, Syed Mushtaq > > > > > > > > >>> <syed1.mush...@gmail.com> > > > > > > > > >>> wrote: > > > > > > > > >>> > > > > > > > > >>>> Is it possible to have both functionalities (snapshot on > > > > > > > > >>>> SAN & Sec > > > > > > > > >>>> Storage) coexist? Because Ideally, we would like to have > > > both. > > > > > > > > >>>> For example, some of our customers want to implement > their > > > > > > > > >>>> own backup strategies and do encryption to their backups > > > > > > > > >>>> which is a perfect use case for Storage Snapshot while > our > > > > > > > > >>>> other customers will still keep using the standard > volume > > > > > snapshot. > > > > > > > > >>>> > > > > > > > > >>>> To keep things backward compatible, we can add a setting > > > > > > > > >>>> which > > > > > > says > > > > > > > > >>>> to not upload on secondary storage, because, after all, > > you > > > > > > > > >>>> would take a SAN snapshot first when doing a Volume > > > Snapshot. > > > > > > > > >>>> You could stop the process there and not do the upload. > > > > > > > > >>>> > > > > > > > > >>>> What do you think about this approach? > > > > > > > > >>>> > > > > > > > > >>>> Thanks, > > > > > > > > >>>> -Syed > > > > > > > > >>>> > > > > > > > > >>>> On Thu, Feb 4, 2016 at 4:25 PM, Mike Tutkowski < > > > > > > > > >>>> mike.tutkow...@solidfire.com> wrote: > > > > > > > > >>>> > > > > > > > > >>>>> So, this is just me thinking out load here, but if a > > given > > > > > > > > >>>>> CloudStack cloud doesn't actually need to provide both > > the > > > > > > ability > > > > > > > > >>>>> to take a SAN snapshot and export it to NFS (if just > > > > > > > > >>>>> taking a SAN snapshot is OK), then we might be able to > > get > > > > > > > > >>>>> away with no new > > > > > > API > > > > > > > > >>>>> calls and simply implement a new custom snapshot > strategy > > > > > > > > >>>>> and > > > > > > data > > > > > > > > >>>>> motion strategy to handle the case where the CloudStack > > > > > > > > >>>>> cloud > > > > > > does > > > > > > > > >>>>> want both a SAN snapshot and exported-to-NFS backup. > > > > > > > > >>>>> > > > > > > > > >>>>> In other words, the "default" behavior would be to use > > the > > > > > > > > >>>>> snapshot strategy and data motion strategy that we > > already > > > > > > > > >>>>> have (the one that only takes a SAN snapshot). > > > > > > > > >>>>> > > > > > > > > >>>>> If your CloudStack cloud, however, wants to take a SAN > > > > > > > > >>>>> snapshot and have the data exported to NFS, then we > could > > > > > > > > >>>>> have you manipulate a Swing config file to make use of > a > > > > > > > > >>>>> new snapshot strategy and data motion strategy that > > > > > > > > >>>>> performs both of these > > > > > > > > activities. > > > > > > > > >>>>> > > > > > > > > >>>>> This way, the old behavior is still the default for > > users, > > > > > > > > >>>>> but CloudStack admins can change this behavior via > > > > > configuration. > > > > > > > > >>>>> > > > > > > > > >>>>> Thoughts? > > > > > > > > >>>>> > > > > > > > > >>>>> On Thu, Feb 4, 2016 at 11:55 AM, Mike Tutkowski < > > > > > > > > >>>>> mike.tutkow...@solidfire.com> wrote: > > > > > > > > >>>>> > > > > > > > > >>>>>> Right...I think we will need to come up with a viable > > > > > > > > >>>>>> upgrade path or some reasonable way for them to move > > from > > > > > > > > >>>>>> the old way to the new way (and some obvious way that > > > > > > > > >>>>>> they will know they need > > > > > > to > > > > > > > > do this). > > > > > > > > >>>>>> > > > > > > > > >>>>>> On Thu, Feb 4, 2016 at 11:45 AM, Syed Mushtaq < > > > > > > > > >>>>>> syed1.mush...@gmail.com> wrote: > > > > > > > > >>>>>> > > > > > > > > >>>>>>> I'm not really sure about the upgrade path however, > > > > > > > > >>>>>>> customers who are using 4.6 and are on a managed > > storage > > > > > > > > >>>>>>> would no longer have the same functionality with > Volume > > > > > Snapshots. > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> On Thu, Feb 4, 2016 at 1:43 PM, Syed Mushtaq < > > > > > > > > >>>>>>> syed1.mush...@gmail.com> wrote: > > > > > > > > >>>>>>> > > > > > > > > >>>>>>>> So if I understand correctly, currently taking a > > Volume > > > > > > > > >>>>>>>> Snapshots of a volume on a managed storage keeps it > on > > > > > > > > >>>>>>>> the storage array. As a part of this feature, we can > > > > > > > > >>>>>>>> make sure > > > > > > that > > > > > > > > >>>>>>>> Volume Snapshots on managed storage are uploaded to > > the > > > > > > > > >>>>>>>> secondary storage. This would make the Volume > Snapshot > > > > > > > > >>>>>>>> feature behave the same regardless of the storage > > > > > > > > >>>>>>>> (managed or > > > > > > > > >>>>>>>> non-managed) And, for utilizing the efficient > backend > > > > > > > > >>>>>>>> storage > > > > > > > > capabilities, we can use the new storage snapshots API. > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>>> On Thu, Feb 4, 2016 at 1:36 PM, Mike Tutkowski < > > > > > > > > >>>>>>>> mike.tutkow...@solidfire.com> wrote: > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>>>> Hi everyone, > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>> Whatever we do here, we need to have a plan to deal > > > > > > > > >>>>>>>>> with the fact that we already have a feature (in > 4.6 > > > > > > > > >>>>>>>>> and > > > > > > > > >>>>>>>>> later) that allows you to use the existing > > > > > > > > >>>>>>>>> volume-snapshot APIs to create a volume snapshot > (for > > > > > > > > >>>>>>>>> managed > > > > > > > > >>>>>>>>> storage) that resides on a backend SAN (using a > > custom > > > > > > > > >>>>>>>>> snapshot strategy and a custom data motion > strategy). > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>> If these new APIs go in, then how should the > original > > > > > > > > >>>>>>>>> implementation (present in 4.6 and later) be > changed? > > > > > > > > >>>>>>>>> If it > > > > > > is > > > > > > > > >>>>>>>>> changed, how do we support customers who are > already > > > > > > > > >>>>>>>>> using > > > > > > the > > > > > > > > >>>>>>>>> original volume-snapshot API to take snapshots on a > > > > > > > > >>>>>>>>> backend > > > > > > > SAN? > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>> Thanks, > > > > > > > > >>>>>>>>> Mike > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>> On Thu, Feb 4, 2016 at 11:27 AM, Will Stevens < > > > > > > > > >>>>>>>>> wstev...@cloudops.com> wrote: > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>>> Will you be able to create a Template from a > > > > > > StorageSnapshot? > > > > > > > > >>>>>>>>>> If yes, will the template be stored in the > secondary > > > > > > > > >>>>>>>>>> storage like normal templates or will that be > > handled > > > > > > > > >>>>>>>>>> somehow on the > > > > > > > > vendor side? > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>>> *Will STEVENS* > > > > > > > > >>>>>>>>>> Lead Developer > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts > > > > > > > > >>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w > > > > > > > > >>>>>>>>>> cloudops.com *|* tw @CloudOps_ > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>>> On Thu, Feb 4, 2016 at 1:22 PM, Syed Mushtaq < > > > > > > > > >>>>>>>>>> syed1.mush...@gmail.com> wrote: > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>>>> Thanks Will!!! > > > > > > > > >>>>>>>>>>> > > > > > > > > >>>>>>>>>>> On Thu, Feb 4, 2016 at 1:19 PM, Will Stevens < > > > > > > > > >>>>>>>>>>> wstev...@cloudops.com> wrote: > > > > > > > > >>>>>>>>>>> > > > > > > > > >>>>>>>>>>>> I explicitly linked the Design Spec in the Jira > > > > > > > > >>>>>>>>>>>> ticket because it was not clear in the 'mention' > > > > > > > > >>>>>>>>>>>> section because it shows as a page 'you do not > > have > > > > > permission to'. > > > > > > > > >>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>> *Will STEVENS* > > > > > > > > >>>>>>>>>>>> Lead Developer > > > > > > > > >>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts > > > > > > > > >>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 > w > > > > > > > > >>>>>>>>>>>> cloudops.com *|* tw @CloudOps_ > > > > > > > > >>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>> On Thu, Feb 4, 2016 at 1:02 PM, Syed Ahmed > > > > > > > > >>>>>>>>>>>> <sah...@cloudops.com > > > > > > > > >>>>>>>>>>>> > wrote: > > > > > > > > >>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> Design Spec: > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> > > > > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Sto > > > > > > > > >>>>>>>>>>>>> rageSnapshot++API > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> Jira Ticket > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> > > https://issues.apache.org/jira/browse/CLOUDSTACK-9 > > > > > > > > >>>>>>>>>>>>> 27 > > > > > > > > >>>>>>>>>>>>> 8 > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> Hi All, > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> We plan to propose a new set of APIs to do > > > > > > > > >>>>>>>>>>>>> snapshots on managed storage backends like > > > SolidFire. > > > > > > > > >>>>>>>>>>>>> Snapshots on current managed storage stay on > the > > > > > > > > >>>>>>>>>>>>> device which is contrary to what CloudStack > calls > > > > > snpshots. > > > > > > > > >>>>>>>>>>>>> But taking snapshots on storage and keeping it > > > > > > > > >>>>>>>>>>>>> there has its own advantages > > > > > > and > > > > > > > > >>>>>>>>>>>>> we would ideally like to have both ways of > doing > > > > > > > > >>>>>>>>>>>>> snapshots. This proposal adds 4 new APIs to > > create > > > > > > > > >>>>>>>>>>>>> snapshots on backend storage. > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> What do you guys think of this feature? I would > > > > > > > > >>>>>>>>>>>>> love to have some feedback. I am working on > > making > > > > > > > > >>>>>>>>>>>>> the design > > > > > > spec > > > > > > > > >>>>>>>>>>>>> more concrete but wanted to have a high level > > > > > > > > >>>>>>>>>>>>> feedback first before starting to work on it. > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> Thanks, > > > > > > > > >>>>>>>>>>>>> -Syed > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>>> > > > > > > > > >>>>>>>>>>>> > > > > > > > > >>>>>>>>>>> > > > > > > > > >>>>>>>>>> > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>>> -- > > > > > > > > >>>>>>>>> *Mike Tutkowski* > > > > > > > > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > > > > > > > > >>>>>>>>> e: mike.tutkow...@solidfire.com > > > > > > > > >>>>>>>>> o: 303.746.7302 > > > > > > > > >>>>>>>>> Advancing the way the world uses the cloud > > > > > > > > >>>>>>>>> < > http://solidfire.com/solution/overview/?video=play > > >*™ > > > > > > > > >>>>>>>>> * > > > > > > > > >>>>>>>>> > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>>> > > > > > > > > >>>>>>> > > > > > > > > >>>>>> > > > > > > > > >>>>>> > > > > > > > > >>>>>> -- > > > > > > > > >>>>>> *Mike Tutkowski* > > > > > > > > >>>>>> *Senior CloudStack Developer, SolidFire Inc.* > > > > > > > > >>>>>> e: mike.tutkow...@solidfire.com > > > > > > > > >>>>>> o: 303.746.7302 > > > > > > > > >>>>>> Advancing the way the world uses the cloud > > > > > > > > >>>>>> <http://solidfire.com/solution/overview/?video=play > >*™* > > > > > > > > >>>>>> > > > > > > > > >>>>> > > > > > > > > >>>>> > > > > > > > > >>>>> > > > > > > > > >>>>> -- > > > > > > > > >>>>> *Mike Tutkowski* > > > > > > > > >>>>> *Senior CloudStack Developer, SolidFire Inc.* > > > > > > > > >>>>> e: mike.tutkow...@solidfire.com > > > > > > > > >>>>> o: 303.746.7302 > > > > > > > > >>>>> Advancing the way the world uses the cloud > > > > > > > > >>>>> <http://solidfire.com/solution/overview/?video=play > >*™* > > > > > > > > >>>>> > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> -- > > > > > > > > >>> *Mike Tutkowski* > > > > > > > > >>> *Senior CloudStack Developer, SolidFire Inc.* > > > > > > > > >>> e: mike.tutkow...@solidfire.com > > > > > > > > >>> o: 303.746.7302 > > > > > > > > >>> Advancing the way the world uses the cloud > > > > > > > > >>> <http://solidfire.com/solution/overview/?video=play>*™* > > > > > > > > >>> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > *Mike Tutkowski* > > > > > > > > > *Senior CloudStack Developer, SolidFire Inc.* > > > > > > > > > e: mike.tutkow...@solidfire.com > > > > > > > > > o: 303.746.7302 > > > > > > > > > Advancing the way the world uses the cloud > > > > > > > > > <http://solidfire.com/solution/overview/?video=play>*™* > > > > > > > > > > > > > > > > > Find out more about ShapeBlue and our range of CloudStack > > > > > > > > related > > > > > > > services: > > > > > > > > IaaS Cloud Design & Build > > > > > > > > <http://shapeblue.com/iaas-cloud-design-and-build//> | > > CSForge – > > > > > > > > rapid IaaS deployment framework < > http://shapeblue.com/csforge/ > > > > > > > > > > > CloudStack Consulting > > > > > > > > <http://shapeblue.com/cloudstack-consultancy/> | > > > > > > > CloudStack > > > > > > > > Software Engineering > > > > > > > > <http://shapeblue.com/cloudstack-software-engineering/> > > > > > > > > CloudStack Infrastructure Support > > > > > > > > <http://shapeblue.com/cloudstack-infrastructure-support/> | > > > > > > > > CloudStack Bootcamp Training Courses > > > > > > > > <http://shapeblue.com/cloudstack-training/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > *Mike Tutkowski* > > > > > > *Senior CloudStack Developer, SolidFire Inc.* > > > > > > e: mike.tutkow...@solidfire.com > > > > > > o: 303.746.7302 > > > > > > Advancing the way the world uses the cloud > > > > > > <http://solidfire.com/solution/overview/?video=play>*™* > > > > > > > > > > > Find out more about ShapeBlue and our range of CloudStack related > > > > services: > > > > > IaaS Cloud Design & Build > > > > > <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – > > rapid > > > > > IaaS deployment framework <http://shapeblue.com/csforge/> > CloudStack > > > > > Consulting <http://shapeblue.com/cloudstack-consultancy/> | > > CloudStack > > > > > Software Engineering > > > > > <http://shapeblue.com/cloudstack-software-engineering/> > > > > > CloudStack Infrastructure Support > > > > > <http://shapeblue.com/cloudstack-infrastructure-support/> | > > CloudStack > > > > > Bootcamp Training Courses < > http://shapeblue.com/cloudstack-training/ > > > > > > > > > > > > Find out more about ShapeBlue and our range of CloudStack related > > > services: > > > > IaaS Cloud Design & Build > > > > <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – > rapid > > > > IaaS deployment framework <http://shapeblue.com/csforge/> > > > > CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> > | > > > CloudStack > > > > Software Engineering > > > > <http://shapeblue.com/cloudstack-software-engineering/> > > > > CloudStack Infrastructure Support > > > > <http://shapeblue.com/cloudstack-infrastructure-support/> | > CloudStack > > > > Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/ > > > > > > > > > > > > > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the cloud > <http://solidfire.com/solution/overview/?video=play>*™* >
