Darren, Since S3/Riak CS can receive the contents of a file out of order and re-assemble upon completion of transfer of all parts, the underlying driver uses the TransferManager which automatically breaks files up into multi-part uploads to increase parallelism to decrease the time required to upload to the object store. Therefore, multi-part uploads are not employed solely to upload files larger than 5 GB.
Have you tried to upload the file to Riak CS using s3cmd outside of CloudStack? Thanks, -John On Oct 31, 2013, at 1:23 PM, Darren Shepherd <darren.s.sheph...@gmail.com> wrote: > My template is less than 5gb and it still uses multipart. I actually don't > think this is a CloudStack issue but more of an issue with Riak-CS. I was > just wondering if anybody has specifically ran the setup or knows more about > Riak-CS and can tell me whats up. I've been trying off and on for the last > week to try to get this working with 4.2 but it just keeps failing. I think > I'll just try ceph rados gateway if I can't get this working. > > Darren > > > On Wed, Oct 30, 2013 at 11:17 PM, Sanjeev Neelarapu > <sanjeev.neelar...@citrix.com> wrote: > Hi, > > Cloudstack with Riak-CS works fine. If we register template more than 5GB we > use multipart upload to push the files to Riak. But this does not work for > snapshots and volumes. This will be fixed in CS4.2.1 . But for template > registration it should work fine. > > Thanks, > Sanjeev > > -----Original Message----- > From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] > Sent: Thursday, October 31, 2013 11:18 AM > To: dev@cloudstack.apache.org > Subject: ACS + Riak-CS, works? > > Has anybody gotten CloudStack and Riak-CS to work together. No matter what I > try when it goes to download and save the template, after its posted a series > of multi part uploads it ends up sending a 403. The access log looks like > > 127.0.0.1 - - [31/Oct/2013:05:41:11 +0000] "PUT > /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=14 > HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic > OpenJDK_64-Bit_Server_VM/20.0-b12" > 127.0.0.1 - - [31/Oct/2013:05:41:28 +0000] "PUT > /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=15 > HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic > OpenJDK_64-Bit_Server_VM/20.0-b12" > 127.0.0.1 - - [31/Oct/2013:05:41:42 +0000] "PUT > /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=16 > HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic > OpenJDK_64-Bit_Server_VM/20.0-b12" > 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT > /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=17 > HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic > OpenJDK_64-Bit_Server_VM/20.0-b12" > 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT > /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=18 > HTTP/1.1" 403 311 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic > OpenJDK_64-Bit_Server_VM/20.0-b12" > > And then in the error log I always get > > 2013-10-31 05:41:51.867 [error] > <0.5117.0>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user record > for s3 failed. Reason: disconnected > > I'm able to interact with Riak-CS with s3cmd just fine but its not using > multipart. I think I'm going to try to download a new s3cmd that does > multipart and see if even that will work. > > Darren >
signature.asc
Description: Message signed with OpenPGP using GPGMail
