Uploading without multipart works with s3cmd. The s3cmd I was using didn't have multipart support, so I'll download a newer one and try it tonight.
Would you happen to know what the below means? I get a 200 response for parts 1,2,3,4,etc and then randomly one part will fail with the below. riak_cs_wm_common:maybe_create_user:223 Retrieval of user record for s3 failed. Reason: disconnected Darren On Thu, Oct 31, 2013 at 10:30 AM, John Burwell <jburw...@basho.com> wrote: > Darren, > > Since S3/Riak CS can receive the contents of a file out of order and > re-assemble upon completion of transfer of all parts, the underlying driver > uses the TransferManager which automatically breaks files up into > multi-part uploads to increase parallelism to decrease the time required to > upload to the object store. Therefore, multi-part uploads are not employed > solely to upload files larger than 5 GB. > > Have you tried to upload the file to Riak CS using s3cmd outside of > CloudStack? > > Thanks, > -John > > On Oct 31, 2013, at 1:23 PM, Darren Shepherd <darren.s.sheph...@gmail.com> > wrote: > > My template is less than 5gb and it still uses multipart. I actually > don't think this is a CloudStack issue but more of an issue with Riak-CS. > I was just wondering if anybody has specifically ran the setup or knows > more about Riak-CS and can tell me whats up. I've been trying off and on > for the last week to try to get this working with 4.2 but it just keeps > failing. I think I'll just try ceph rados gateway if I can't get this > working. > > Darren > > > On Wed, Oct 30, 2013 at 11:17 PM, Sanjeev Neelarapu < > sanjeev.neelar...@citrix.com> wrote: > >> Hi, >> >> Cloudstack with Riak-CS works fine. If we register template more than 5GB >> we use multipart upload to push the files to Riak. But this does not work >> for snapshots and volumes. This will be fixed in CS4.2.1 . But for template >> registration it should work fine. >> >> Thanks, >> Sanjeev >> >> -----Original Message----- >> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >> Sent: Thursday, October 31, 2013 11:18 AM >> To: dev@cloudstack.apache.org >> Subject: ACS + Riak-CS, works? >> >> Has anybody gotten CloudStack and Riak-CS to work together. No matter >> what I try when it goes to download and save the template, after its posted >> a series of multi part uploads it ends up sending a 403. The access log >> looks like >> >> 127.0.0.1 - - [31/Oct/2013:05:41:11 +0000] "PUT >> >> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=14 >> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic >> OpenJDK_64-Bit_Server_VM/20.0-b12" >> 127.0.0.1 - - [31/Oct/2013:05:41:28 +0000] "PUT >> >> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=15 >> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic >> OpenJDK_64-Bit_Server_VM/20.0-b12" >> 127.0.0.1 - - [31/Oct/2013:05:41:42 +0000] "PUT >> >> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=16 >> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic >> OpenJDK_64-Bit_Server_VM/20.0-b12" >> 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT >> >> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=17 >> HTTP/1.1" 200 0 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic >> OpenJDK_64-Bit_Server_VM/20.0-b12" >> 127.0.0.1 - - [31/Oct/2013:05:41:51 +0000] "PUT >> >> /buckets/bucket1/objects/template%2Ftmpl%2F1%2F4%2Fcentos55-x86_64%2Feec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2/uploads/LmEFZncZTJqgTdSe2FP5bw==?partNumber=18 >> HTTP/1.1" 403 311 "" "aws-sdk-java/1.3.22 Linux/3.8.0-31-generic >> OpenJDK_64-Bit_Server_VM/20.0-b12" >> >> And then in the error log I always get >> >> 2013-10-31 05:41:51.867 [error] >> <0.5117.0>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user >> record for s3 failed. Reason: disconnected >> >> I'm able to interact with Riak-CS with s3cmd just fine but its not using >> multipart. I think I'm going to try to download a new s3cmd that does >> multipart and see if even that will work. >> >> Darren >> > > >
