> -----Original Message----- > From: John Burwell [mailto:jburw...@basho.com] > Sent: Tuesday, July 09, 2013 1:31 PM > To: dev@cloudstack.apache.org > Cc: 'Chip Childers' > Subject: Re: Swift in 4.2 is broken, anybody wants it to be supported in 4.2? > > Edison, > > TL;DR The shorter path is to re-implement/fix the Swift driver. > > Multipart upload would need to implemented in Swift, not in CloudStack. > Therefore, such a change would need to be accepted and released by the > OpenStack project before the 4.2.0 release. We would also be stranding any > of our current users who cannot or will not upgrade their Swift instances. > > Knowing what it took to implement it in Riak CS, multi-part upload was a lot > of work to implement in a Dynamo-based system. The S3 has the following > three-phase process: > > 1. Initiate a Upload: Declare the number of parts and their size > 2. Submit each part per the definition in Step 1 (e.g. 50 parts = 50 > HTTP PUTs) > 3. Complete the multi-part upload: Declare that all parts have been > uploaded which causes the object to become available > > In contrast, Swift uses HTTP chunking to solve the same problem with one > API call. In addition to providing all of the reliability guarantees of the > S3 API, > an implementor of S3 multipart uploads will have to provide a way to > translate the 3-phase model into the single call model used by Swift.
Seems swift client can support upload files more than 5GB: http://fedoraproject.org/wiki/QA:Testcase_Swift_Upload_Large_File Ok, so we should just re-implement/fix the existing swift driver. > > Thanks, > -John > > On Jul 9, 2013, at 4:15 PM, Edison Su <edison...@citrix.com> wrote: > > > Could they(swiftstack) help us, or guide us on how to implement multi-part > upload? > > > >> -----Original Message----- > >> From: Sebastien Goasguen [mailto:run...@gmail.com] > >> Sent: Tuesday, July 09, 2013 1:07 PM > >> To: dev@cloudstack.apache.org > >> Cc: 'Chip Childers' > >> Subject: Re: Swift in 4.2 is broken, anybody wants it to be supported in > >> 4.2? > >> > >> if swift does not work anymore in 4.0 or 4.1 maybe be should inform > >> swiftstack: > >> http://swiftstack.com/cloudstack/ > >> > >> > >> On Jul 9, 2013, at 3:57 PM, John Burwell <jburw...@basho.com> wrote: > >> > >>> Edison, > >>> > >>> Swift does not support S3 multi-part uploads [1] which CloudStack > >>> must use > >> in order to store files larger than 5 GB. Therefore, using the > >> Swift's S3 compatibility layer is not a viable workaround. > >>> > >>> Thanks, > >>> -John > >>> > >>> [1]: https://wiki.openstack.org/wiki/Swift/APIFeatureComparison > >>> > >>> On Jul 9, 2013, at 2:12 PM, Edison Su <edison...@citrix.com> wrote: > >>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Chip Childers [mailto:chip.child...@sungard.com] > >>>>> Sent: Monday, July 08, 2013 1:26 PM > >>>>> To: Edison Su > >>>>> Cc: <dev@cloudstack.apache.org> > >>>>> Subject: Re: Swift in 4.2 is broken, anybody wants it to be > >>>>> supported in > >> 4.2? > >>>>> > >>>>> On Mon, Jul 08, 2013 at 05:15:19PM +0000, Edison Su wrote: > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Chip Childers [mailto:chip.child...@sungard.com] > >>>>>>> Sent: Monday, July 08, 2013 6:46 AM > >>>>>>> To: <dev@cloudstack.apache.org>; Edison Su > >>>>>>> Subject: Re: Swift in 4.2 is broken, anybody wants it to be > >>>>>>> supported in > >>>>> 4.2? > >>>>>>> > >>>>>>> On Mon, Jul 8, 2013 at 9:22 AM, David Nalley <da...@gnsa.us> > wrote: > >>>>>>>> On Wed, Jul 3, 2013 at 5:29 PM, Edison Su > >>>>>>>> <edison...@citrix.com> > >>>>> wrote: > >>>>>>>>> Due to object store refactor, Swift is broken. The reason, is > >>>>>>>>> that, we only > >>>>>>> have S3 test environment in our lab, so only S3 is tested for now. > >>>>>>>>> Before adding the feature back, I'd better ask from, the > >>>>>>>>> community, do > >>>>>>> we want to support Swift? If so, which version of Swift? This > >>>>>>> will take some efforts to support Swift, are there any > >>>>>>> volunteers can help the > >>>>> integration? > >>>>>>>> > >>>>>>>> > >>>>>>>> Whats the bug ID for this? > >>>>>>>> Unplanned/Unannounced deprecation of a feature is a blocker > IMO. > >>>>>>>> It engenders a bad relationship with our users, and strands > >>>>>>>> them on previous versions with no good migration/upgrade path. > >>>>>>>> > >>>>>>>> --David > >>>>>>>> > >>>>>>> > >>>>>>> Edison, How broken is it? Is it shorter to fix or revert the > >>>>>>> object store changes? > >>>>>> It's not working at all. Not sure, revert object store will > >>>>>> change it or not, as > >>>>> this feature is not tested by QA for a long time. > >>>>>> > >>>>>> > >>>>> > >>>>> So any idea what the effort of fixing it looks like? I mean, just > >>>>> because it > >>>> > >>>> If it's ok to use S3 api talking to swift, then there is zero > >>>> effort to support > >> Swift. > >>>> But who will make the decision? > >>>> > >>>>> wasn't tested in the last couple of releases doesn't necessarily > >>>>> mean that it wasn't working. As Sudha mentioned, it wasn't tested > >>>>> only because of a lack of change that triggered the expected need > >>>>> to perform regression testing of that feature. > >>>>> > >>>>> I believe that this was an honest mistake, but we need to figure > >>>>> out what to do. I'm -1 on us saying "we'll drop Swift support". > >>>>> If necessary, I'd say that we need to roll back the object-store > >>>>> branch merge... I don't want to see that happen though. That's > >>>>> why I'm asking > >> about effort to fix it. > >>>>> > >>>>> -chip > >>> > >
