As Will put it, the implementation for using Object store as secondary storage relies on using Staging NFS. You would also loose the functionality to do differential snapshots. All your snapshots will basically be full volume snapshots. So, if you have a lot of snapshots being taken at scheduled intervals, you might end up with wasting a lot of space on your S3. Not to mention all of this has to go through the NFS which can (an in our experience has) fill up preventing you from creating new VMs (because templates cannot be downloaded). However you do benefit from a region wide Image store. We've worked around the NFS problem by basically having multiple staging NFS stores.
I have plans to make Object stores better integrated in Cloudstack. The holy grail would be to avoid staging NFS completely (or atleast in the upload path). But for now we have to live with the limitations. -Syed On Tue, Jul 19, 2016 at 11:41 AM, Will Stevens <wstev...@cloudops.com> wrote: > We are using Swift as secondary storage. It also offers inter-zone > secondary storage. We have had to do fixes to make it work correctly, but > we have pushed those fixes upstream so they are available in 4.9 (and some > fixes earlier than that). > > Like Wido said, the staging NFS is still needed which is a real problem > with these implementations. You end up copying the data many times and you > will run into problems if you have lots of customers doing scheduled > snapshots at the same time because you can fill your NFS staging area > unless you give it a lot of space to work with. > > The object storage integrations are working, but they are not designed very > well at present. You have code for both S3 and Swift basically just > separated with conditionals and much of the logic is shared. This is very > brittle because making changes in one integration can (and historically > has) broken the other implementation. This is a relatively big problem > because most people working with either Swift or S3 will only have the > ability to test one of them, so they don't really have the ability to test > if they have broken the other. It does not help that there is Zero CI > coverage run for either the Swift or S3 integrations due to the difficulty > building and tearing down object storage environments on the fly. > > All in all, yes they should be working, but expect a bit of a bumpy road. > > Hope that helps, > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 > w cloudops.com *|* tw @CloudOps_ > > On Tue, Jul 19, 2016 at 11:26 AM, Wido den Hollander <w...@widodh.nl> > wrote: > > > > > > Op 19 juli 2016 om 14:38 schreef Nux! <n...@li.nux.ro>: > > > > > > > > > Hello, > > > > > > Could anyone summarise the state of S3 (and clones) for secondary > > storage? > > > I read at some point that it's the only secondary storage type > supported > > inter-zone, so it would appeal from a DR pov, but not sure how mature it > is > > or if there are any serious gotchas involved. > > > > > > > We have it running for our largest region (Amsterdam) with Ceph's RADOS > > Gateway as backend. We had some issues, but they seem to be fixed after > our > > latest patches. > > > > You still need the staging NFS though, so that doesn't solve anything. > > > > Wido > > > > > Thanks > > > Lucian > > > > > > -- > > > Sent from the Delta quadrant using Borg technology! > > > > > > Nux! > > > www.nux.ro > > >
