Edison,
TL;DR The shorter path is to re-implement/fix the Swift driver.
Multipart upload would need to implemented in Swift, not in CloudStack.
Therefore, such a change would need to be accepted and released by the
OpenStack project before the 4.2.0 release. We would also be stranding any of
our current users who cannot or will not upgrade their Swift instances.
Knowing what it took to implement it in Riak CS, multi-part upload was a lot of
work to implement in a Dynamo-based system. The S3 has the following
three-phase process:
1. Initiate a Upload: Declare the number of parts and their size
2. Submit each part per the definition in Step 1 (e.g. 50 parts = 50
HTTP PUTs)
3. Complete the multi-part upload: Declare that all parts have been
uploaded which causes the object to become available
In contrast, Swift uses HTTP chunking to solve the same problem with one API
call. In addition to providing all of the reliability guarantees of the S3
API, an implementor of S3 multipart uploads will have to provide a way to
translate the 3-phase model into the single call model used by Swift.
Thanks,
-John
On Jul 9, 2013, at 4:15 PM, Edison Su <[email protected]> wrote:
> Could they(swiftstack) help us, or guide us on how to implement multi-part
> upload?
>
>> -----Original Message-----
>> From: Sebastien Goasguen [mailto:[email protected]]
>> Sent: Tuesday, July 09, 2013 1:07 PM
>> To: [email protected]
>> Cc: 'Chip Childers'
>> Subject: Re: Swift in 4.2 is broken, anybody wants it to be supported in 4.2?
>>
>> if swift does not work anymore in 4.0 or 4.1 maybe be should inform
>> swiftstack:
>> http://swiftstack.com/cloudstack/
>>
>>
>> On Jul 9, 2013, at 3:57 PM, John Burwell <[email protected]> wrote:
>>
>>> Edison,
>>>
>>> Swift does not support S3 multi-part uploads [1] which CloudStack must use
>> in order to store files larger than 5 GB. Therefore, using the Swift's S3
>> compatibility layer is not a viable workaround.
>>>
>>> Thanks,
>>> -John
>>>
>>> [1]: https://wiki.openstack.org/wiki/Swift/APIFeatureComparison
>>>
>>> On Jul 9, 2013, at 2:12 PM, Edison Su <[email protected]> wrote:
>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Chip Childers [mailto:[email protected]]
>>>>> Sent: Monday, July 08, 2013 1:26 PM
>>>>> To: Edison Su
>>>>> Cc: <[email protected]>
>>>>> Subject: Re: Swift in 4.2 is broken, anybody wants it to be supported in
>> 4.2?
>>>>>
>>>>> On Mon, Jul 08, 2013 at 05:15:19PM +0000, Edison Su wrote:
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Chip Childers [mailto:[email protected]]
>>>>>>> Sent: Monday, July 08, 2013 6:46 AM
>>>>>>> To: <[email protected]>; Edison Su
>>>>>>> Subject: Re: Swift in 4.2 is broken, anybody wants it to be
>>>>>>> supported in
>>>>> 4.2?
>>>>>>>
>>>>>>> On Mon, Jul 8, 2013 at 9:22 AM, David Nalley <[email protected]> wrote:
>>>>>>>> On Wed, Jul 3, 2013 at 5:29 PM, Edison Su <[email protected]>
>>>>> wrote:
>>>>>>>>> Due to object store refactor, Swift is broken. The reason, is
>>>>>>>>> that, we only
>>>>>>> have S3 test environment in our lab, so only S3 is tested for now.
>>>>>>>>> Before adding the feature back, I'd better ask from, the
>>>>>>>>> community, do
>>>>>>> we want to support Swift? If so, which version of Swift? This will
>>>>>>> take some efforts to support Swift, are there any volunteers can
>>>>>>> help the
>>>>> integration?
>>>>>>>>
>>>>>>>>
>>>>>>>> Whats the bug ID for this?
>>>>>>>> Unplanned/Unannounced deprecation of a feature is a blocker IMO.
>>>>>>>> It engenders a bad relationship with our users, and strands them
>>>>>>>> on previous versions with no good migration/upgrade path.
>>>>>>>>
>>>>>>>> --David
>>>>>>>>
>>>>>>>
>>>>>>> Edison, How broken is it? Is it shorter to fix or revert the
>>>>>>> object store changes?
>>>>>> It's not working at all. Not sure, revert object store will change
>>>>>> it or not, as
>>>>> this feature is not tested by QA for a long time.
>>>>>>
>>>>>>
>>>>>
>>>>> So any idea what the effort of fixing it looks like? I mean, just
>>>>> because it
>>>>
>>>> If it's ok to use S3 api talking to swift, then there is zero effort to
>>>> support
>> Swift.
>>>> But who will make the decision?
>>>>
>>>>> wasn't tested in the last couple of releases doesn't necessarily
>>>>> mean that it wasn't working. As Sudha mentioned, it wasn't tested
>>>>> only because of a lack of change that triggered the expected need to
>>>>> perform regression testing of that feature.
>>>>>
>>>>> I believe that this was an honest mistake, but we need to figure out
>>>>> what to do. I'm -1 on us saying "we'll drop Swift support". If
>>>>> necessary, I'd say that we need to roll back the object-store branch
>>>>> merge... I don't want to see that happen though. That's why I'm asking
>> about effort to fix it.
>>>>>
>>>>> -chip
>>>
>
