Thanks, Tom and Sanjeev for verification. On 7/19/13 1:26 AM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote:
>Hi Min, > >I will have time next week to check through the bugs and verify >everything. Thanks for checking on me. > >Tom. > >On Thu, 2013-07-18 at 23:37 +0000, Min Chen wrote: >> Thanks Jessica. Tom, did you still see the issue? >> >> -min >> >> On 7/8/13 1:20 PM, "Jessica Wang" <jessica.w...@citrix.com> wrote: >> >> >Min, >> > >> >> would you please take a look at this to see if UI can disable >>decoding >> >>in displaying this download template url >> >> returned from API? >> > >> >I just changed UI to not decode the URL returned in extractTemplate, >> >extractIso API. >> > >> >Jessica >> > >> > >> >-----Original Message----- >> >From: Min Chen >> >Sent: Wednesday, July 03, 2013 5:53 PM >> >To: dev@cloudstack.apache.org; Thomas O'Dowd >> >Cc: Jessica Wang >> >Subject: Re: Query String Request Authentication(QSRA) support by S3 >> >providers >> > >> >Jessica, would you please take a look at this to see if UI can disable >> >decoding in displaying this download template url returned from API? >> > >> >Thanks >> >-min >> > >> >On 7/3/13 5:38 PM, "Min Chen" <min.c...@citrix.com> wrote: >> > >> >>By examining further what returned from extractTemplateCmd api, I >> >>realized >> >>that the URL returned from API is different from what is displayed >>from >> >>pop-up dialog from UI. Directly using the link returned from API >>(with / >> >>encoded as %2F can successfully download the template. So the issue >>may >> >>not be that bad to upgrade Amazon SDK, but a simple UI issue. That >>is, is >> >>it possible for UI not to decode URLEncoded string in this case? >> >> >> >>Thanks >> >>-min >> >> >> >>On 7/3/13 5:22 PM, "Min Chen" <min.c...@citrix.com> wrote: >> >> >> >>>Hi Tom, >> >>> >> >>> I can reproduce this issue using Cloudian, after investigation, I >> >>>realized that this is a bug in Amazon SDK we have used, based on this >> >>>thread: >> >>>>>http://stackoverflow.com/questions/15473582/amazon-s3-presigned-urls-e >>>>>sc >> >>>a >> >>>p >> >>>e >> >>>-the-slashes-in-the-key. When generatePresignedUrl is called it takes >> >>>the >> >>>entire key and escapes it, and then creates a signature using the >> >>>escaped >> >>>key. You cannot use the signature from the escaped key and combine it >> >>>with >> >>>the unescaped key in the URL. See the bug code here: >> >>> >> >>> String resourcePath = "/" + >> >>> ((bucketName != null) ? bucketName + "/" : "") + >> >>> ((key != null) ? ServiceUtils.urlEncode(key) : "") + >> >>> ((subResource != null) ? "?" + subResource : ""); >> >>> >> >>>We have two options to fix this: >> >>> 1. Either upgrade Amazon SDK to use 1.4.3 version, someone in that >> >>>thread >> >>>claimed that it is fixed in that version, but I haven't checked that. >> >>>Currently CloudStack is using 1.3.21. Not sure if this will break >> >>>CloudStack cloud_bridge. >> >>> 2. Workaround by creating customized AmazonS3Client to change the >> >>>internal implementation on this. >> >>> >> >>> Thanks >> >>> -min >> >>> >> >>> >> >>> >> >>>On 7/2/13 11:31 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote: >> >>> >> >>>>Excellent. The link is there now. Thank you Min. I verified that bug >> >>>>and >> >>>>closed it. >> >>>> >> >>>>However - now that I can finally click the download link... I ran >>into >> >>>>the issue that the link doesn't work on AWS or Cloudian. Please see >> >>>>this >> >>>>bug for details (latest 4.2 updates included in my test). >> >>>> >> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-3341 >> >>>> >> >>>>Tom. >> >>>> >> >>>>On Tue, 2013-07-02 at 22:54 +0000, Min Chen wrote: >> >>>>> Hi Tom, >> >>>>> I investigated this issue through the db dump you provided in >> >>>>> the >> >>>>>bug, >> >>>>> this is an issue with our db view template_view creation script, >>and >> >>>>>it >> >>>>> has been fixed in resolving >> >>>>> another bug >>(https://issues.apache.org/jira/browse/CLOUDSTACK-3314). >> >>>>>I >> >>>>> have verified the fix using your db dump on my local setup. Please >> >>>>>check >> >>>>> out latest 4.2 or master code to try again. >> >>>>> >> >>>>> Thanks >> >>>>> -min >> >>>>> >> >>>>> On 7/2/13 2:18 PM, "Min Chen" <min.c...@citrix.com> wrote: >> >>>>> >> >>>>> >Tom, this seems like an issue with entry stored in our DB. I will >> >>>>>take >> >>>>>a >> >>>>> >look at this bug and update you. Just to clarify, this symptom >>only >> >>>>> >happens when you register these templates to Amazon S3, not for >> >>>>>Cloudian >> >>>>> >or RiakCS S3, right? >> >>>>> > >> >>>>> >Thanks >> >>>>> >-min >> >>>>> > >> >>>>> >On 7/1/13 7:27 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote: >> >>>>> > >> >>>>> >>Yes thanks Jessica. I re-opened the bug again. I know its not a >>gui >> >>>>> >>problem per-say in that the template is not ready to show the >> >>>>>download >> >>>>> >>link. However, it never becomes ready is the actual problem. >>What >> >>>>>sets >> >>>>> >>the "isready" property to true? As far as I can see, the >>objects in >> >>>>>the >> >>>>> >>S3 stores (AWS or Cloudian) are complete and from my perspective >> >>>>>"ready" >> >>>>> >>to download/use. It sounds like a bug when registering the >> >>>>>template. >> >>>>> >> >> >>>>> >>Tom. >> >>>>> >> >> >>>>> >>On Mon, 2013-07-01 at 18:54 +0000, Jessica Wang wrote: >> >>>>> >>> Thomas, >> >>>>> >>> >> >>>>> >>> I checked the data you provided. >> >>>>> >>> >> >>>>> >>> The reason that the 2 templates("MyTiny", "AnotherTiny") have >>no >> >>>>> >>>download button is because they are not ready >> >>>>> >>> (i.e. their "isready" property is false). >> >>>>> >>> >> >>>>> >>> Download button is only available when "isready" property is >> >>>>>true. >> >>>>> >>> >> >>>>> >>> Jessica >> >>>>> >>> >> >>>>> >>> -----Original Message----- >> >>>>> >>> From: Thomas O'Dowd [mailto:tpod...@cloudian.com] >> >>>>> >>> Sent: Thursday, June 27, 2013 8:04 PM >> >>>>> >>> To: Min Chen >> >>>>> >>> Cc: dev@cloudstack.apache.org; Jessica Wang >> >>>>> >>> Subject: Re: Query String Request Authentication(QSRA) >>support by >> >>>>>S3 >> >>>>> >>>providers >> >>>>> >>> >> >>>>> >>> Hi Min/Jessica, >> >>>>> >>> >> >>>>> >>> I attached an image to that issue to show what what my >>browser is >> >>>>> >>> showing. >> >>>>> >>> >> >>>>> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-3220 >> >>>>> >>> >> >>>>> >>> Tom. >> >>>>> >>> >> >>>>> >>> On Fri, 2013-06-28 at 09:45 +0900, Thomas O'Dowd wrote: >> >>>>> >>> > Hi Min, >> >>>>> >>> > >> >>>>> >>> > Yes. I'll try it again today to check again but when I added >> >>>>>Amazon >> >>>>> >>>S3 >> >>>>> >>> > as the S3 secondary storage and uploaded a template, I was >>not >> >>>>>shown >> >>>>> >>>the >> >>>>> >>> > "download template" link. However - for Cloudian S3, I am >>shown >> >>>>>it so >> >>>>> >>> > I'm wondering why. >> >>>>> >>> > >> >>>>> >>> > Tom. >> >>>>> >>> > >> >>>>> >>> > On Fri, 2013-06-28 at 00:26 +0000, Min Chen wrote: >> >>>>> >>> > > Hi Tom, >> >>>>> >>> > > >> >>>>> >>> > > Are you saying that you cannot see a Download Template >> >>>>>button >> >>>>>from >> >>>>> >>>UI >> >>>>> >>> > > when Amazon S3 is added as secondary storage? I only >>tested >> >>>>>with >> >>>>> >>>RiakCS >> >>>>> >>> > > and Cloudian, so didn't see this issue. But I am CC >>Jessica >> >>>>>her >> >>>>>to >> >>>>> >>>confirm >> >>>>> >>> > > what special handling is done in UI to enable/disable a >> >>>>>button >> >>>>>from >> >>>>> >>>UI. >> >>>>> >>> > > >> >>>>> >>> > > Thanks >> >>>>> >>> > > -min >> >>>>> >>> > > >> >>>>> >>> > > On 6/27/13 5:23 PM, "Thomas O'Dowd" <tpod...@cloudian.com> >> >>>>>wrote: >> >>>>> >>> > > >> >>>>> >>> > > >Hi Min, >> >>>>> >>> > > > >> >>>>> >>> > > >Can you check this bug? I'm trying to test this feature >>for >> >>>>>Amazon >> >>>>> >>>but >> >>>>> >>> > > >having no luck getting the Download template link/button >>to >> >>>>> >>>appear. >> >>>>> >>> > > > >> >>>>> >>> > > >https://issues.apache.org/jira/browse/CLOUDSTACK-3220 >> >>>>> >>> > > > >> >>>>> >>> > > >Thanks, >> >>>>> >>> > > > >> >>>>> >>> > > >Tom. >> >>>>> >>> > > > >> >>>>> >>> > > >On Fri, 2013-06-21 at 17:21 +0000, Min Chen wrote: >> >>>>> >>> > > >> John, >> >>>>> >>> > > >> >> >>>>> >>> > > >> For S3, the api call createEntityExtractUrl is done on >> >>>>> >>>management >> >>>>> >>> > > >>server >> >>>>> >>> > > >> side; while for NFS secondary storage, if the >> >>>>>implementation >> >>>>>of >> >>>>> >>> > > >> createEntityExtractUrl will involve some code be >>executed >> >>>>>in >> >>>>> >>>ssvm to >> >>>>> >>> > > >>copy >> >>>>> >>> > > >> template from the install location to a public >>accessible >> >>>>>web >> >>>>> >>>server >> >>>>> >>> > > >> location. >> >>>>> >>> > > >> I don't quite understand some of your comments below. >> >>>>>This >> >>>>>API >> >>>>> >>>is not >> >>>>> >>> > > >> used to write any information to S3 bucket/directory. >>This >> >>>>>is >> >>>>> >>>used for >> >>>>> >>> > > >> object already existed on S3, and we just provide a URL >> >>>>>for >> >>>>>user >> >>>>> >>>to >> >>>>> >>> > > >> download a template from S3, just like how Amazon >>provided >> >>>>>user >> >>>>> >>>a way to >> >>>>> >>> > > >> user to extract a S3 object through >>generatePresignedUrl. >> >>>>>We >> >>>>>can >> >>>>> >>>discuss >> >>>>> >>> > > >> more on this on collaboration conference. >> >>>>> >>> > > >> >> >>>>> >>> > > >> Thanks >> >>>>> >>> > > >> -min >> >>>>> >>> > > >> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >> >>>>> >>> > > >> On 6/21/13 7:25 AM, "John Burwell" <jburw...@basho.com> >> >>>>>wrote: >> >>>>> >>> > > >> >> >>>>> >>> > > >> >Min, >> >>>>> >>> > > >> > >> >>>>> >>> > > >> >(I apologize for my belated reply -- I lost track of >>this >> >>>>>draft >> >>>>> >>>in the >> >>>>> >>> > > >> >chaos of the last couple of days.) >> >>>>> >>> > > >> > >> >>>>> >>> > > >> >Upon further review, I think I feel into the confusion >> >>>>>between >> >>>>> >>> > > >>management >> >>>>> >>> > > >> >server and ssvm. This code is executing on the >> >>>>>management >> >>>>> >>>server side, >> >>>>> >>> > > >> >correct? Based on my "corrected" understanding is >> >>>>>correct, >> >>>>>I >> >>>>> >>>would >> >>>>> >>> > > >>like >> >>>>> >>> > > >> >to amend my thoughts. Namely, I would like to see the >> >>>>>driver >> >>>>> >>> > > >>operations >> >>>>> >>> > > >> >pushed out to the SSVM where we can use the stream. >>As I >> >>>>>think >> >>>>> >>>about >> >>>>> >>> > > >>it, >> >>>>> >>> > > >> >the management server should not need to interact with >> >>>>>the >> >>>>> >>>driver. >> >>>>> >>> > > >> >Simply yard up the DataStore attributes + details map >>and >> >>>>>other >> >>>>> >>>extract >> >>>>> >>> > > >> >parameters, and send them to the SSVM. Using this >> >>>>>information, >> >>>>> >>>the S3 >> >>>>> >>> > > >> >driver could open a stream to write the template out >>to >> >>>>>the >> >>>>> >>> > > >> >bucket/directory. I recognize it changes the protocol >> >>>>>between >> >>>>> >>>the >> >>>>> >>> > > >> >management server and SSVM, but it simply both sides >>of >> >>>>>the >> >>>>> >>>operation >> >>>>> >>> > > >>by >> >>>>> >>> > > >> >allowing the DataStore information to be treated >>opaquely >> >>>>>until >> >>>>> >>>it is >> >>>>> >>> > > >> >consumed by the driver to execute the write >>operation. I >> >>>>>also >> >>>>> >>> > > >>recognize >> >>>>> >>> > > >> >that we may a little late in the cycle to address it >>for >> >>>>>4.2, >> >>>>> >>>and it >> >>>>> >>> > > >>may >> >>>>> >>> > > >> >need to be part of the 4.3 enhancements. >> >>>>> >>> > > >> > >> >>>>> >>> > > >> >Thanks, >> >>>>> >>> > > >> >-John >> >>>>> >>> > > >> > >> >>>>> >>> > > >> >On Jun 18, 2013, at 3:55 PM, Min Chen >> >>>>><min.c...@citrix.com> >> >>>>> >>>wrote: >> >>>>> >>> > > >> > >> >>>>> >>> > > >> >> John, >> >>>>> >>> > > >> >> In that case, how do we keep backward >>compatibility of >> >>>>> >>> > > >>extractTemplate >> >>>>> >>> > > >> >> api, which requires a URL in the response? >> >>>>> >>> > > >> >> >> >>>>> >>> > > >> >> Thanks >> >>>>> >>> > > >> >> -min >> >>>>> >>> > > >> >> >> >>>>> >>> > > >> >> On 6/18/13 11:53 AM, "John Burwell" >> >>>>><jburw...@basho.com> >> >>>>> >>>wrote: >> >>>>> >>> > > >> >> >> >>>>> >>> > > >> >>> Min, >> >>>>> >>> > > >> >>> >> >>>>> >>> > > >> >>> Looking through the code, I think we can simplify >> >>>>>driver >> >>>>> >>>operation >> >>>>> >>> > > >>and >> >>>>> >>> > > >> >>> increase robustness by changing >> >>>>> >>> > > >> >>>ImageStoreDriver#createEntityExtractUrl() >> >>>>> >>> > > >> >>> : String to ImageStoreDriver#readEntity(Š) : >> >>>>>InputStream. >> >>>>> >>>My first >> >>>>> >>> > > >> >>> concern with the current implementation is that it >> >>>>> >>>circumvents any >> >>>>> >>> > > >> >>> connection pooling/resource management underlying >> >>>>>client >> >>>>> >>>libraries >> >>>>> >>> > > >> >>> provide. I/O streams provide a higher-level >> >>>>>abstraction >> >>>>> >>>that allows >> >>>>> >>> > > >> >>> drivers to provide the orchestration components >>with >> >>>>>actual >> >>>>> >>> > > >>resources >> >>>>> >>> > > >> >>> rather String references. Second, the current >> >>>>>interface >> >>>>> >>>seems to >> >>>>> >>> > > >> >>>appears >> >>>>> >>> > > >> >>> to assume that an http/https URL will be returned. >> >>>>>With >> >>>>>I/O >> >>>>> >>> > > >>streams, >> >>>>> >>> > > >> >>>we >> >>>>> >>> > > >> >>> can support any client library capable of using the >> >>>>>standard >> >>>>> >>>I/O >> >>>>> >>> > > >> >>> framework -- enabling us to support other protocols >> >>>>>for >> >>>>> >>>downloading >> >>>>> >>> > > >> >>> templates in the future (e.g. RBD, local >>filesystem, >> >>>>>NBD, >> >>>>> >>>etc). >> >>>>> >>> > > >> >>> >> >>>>> >>> > > >> >>> Thanks, >> >>>>> >>> > > >> >>> -John >> >>>>> >>> > > >> >>> >> >>>>> >>> > > >> >>> On Jun 18, 2013, at 1:11 PM, Min Chen >> >>>>><min.c...@citrix.com> >> >>>>> >>>wrote: >> >>>>> >>> > > >> >>> >> >>>>> >>> > > >> >>>> A new version of using generatePresignedUrl in >> >>>>> >>> > > >>S3ImageStoreDriverImpl >> >>>>> >>> > > >> >>>>is >> >>>>> >>> > > >> >>>> checked into object_store. >> >>>>> >>> > > >> >>>> >> >>>>> >>> > > >> >>>> THanks >> >>>>> >>> > > >> >>>> -min >> >>>>> >>> > > >> >>>> >> >>>>> >>> > > >> >>>> On 6/18/13 8:29 AM, "Min Chen" >><min.c...@citrix.com> >> >>>>>wrote: >> >>>>> >>> > > >> >>>> >> >>>>> >>> > > >> >>>>> Yes, current code is in >> >>>>> >>> > > >> >>>>>S3ImageStoreDriverImpl.createEntityExtractUrl, >> >>>>> >>> > > >> >>>>> which has a security issue mentioned in >> >>>>>CLOUDSTACK-3030. I >> >>>>> >>>am >> >>>>> >>> > > >>going >> >>>>> >>> > > >> >>>>>to >> >>>>> >>> > > >> >>>>> change it to use generatePresignedUrl api from >>AWS >> >>>>>S3 >> >>>>>api. >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Thanks >> >>>>> >>> > > >> >>>>> -min >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> From: John Burwell >> >>>>> >>><jburw...@basho.com<mailto:jburw...@basho.com>> >> >>>>> >>> > > >> >>>>> Date: Tuesday, June 18, 2013 8:07 AM >> >>>>> >>> > > >> >>>>> To: Min Chen >> >>>>> >>><min.c...@citrix.com<mailto:min.c...@citrix.com>> >> >>>>> >>> > > >> >>>>> Cc: Thomas O'Dowd >> >>>>> >>> > > >> >>>>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>, >> >>>>> >>> > > >> >>>>> >> >>>>> >>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >> >>>>> >>> > > >> >>>>> >> >>>>> >>><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >> >>>>> >>> > > >> >>>>> Subject: Re: Query String Request >> >>>>>Authentication(QSRA) >> >>>>> >>>support by >> >>>>> >>> > > >>S3 >> >>>>> >>> > > >> >>>>> providers >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Min, >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Is the code checked into the object_store branch? >> >>>>>If >> >>>>>so, >> >>>>> >>>which >> >>>>> >>> > > >>lines >> >>>>> >>> > > >> >>>>> in >> >>>>> >>> > > >> >>>>> S3TemplateDownloader? >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Thanks, >> >>>>> >>> > > >> >>>>> -John >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> On Jun 18, 2013, at 12:39 AM, Min Chen >> >>>>> >>> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> >> >>>>>wrote: >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Hi John, >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> This is regarding extractTemplate api, where for >> >>>>> >>>extractable >> >>>>> >>> > > >> >>>>>template, >> >>>>> >>> > > >> >>>>> users can click "Download Template" button from >>UI >> >>>>>to >> >>>>>get >> >>>>> >>>a http >> >>>>> >>> > > >>url >> >>>>> >>> > > >> >>>>>to >> >>>>> >>> > > >> >>>>> download the template already stored at S3 >>without >> >>>>> >>>providing S3 >> >>>>> >>> > > >> >>>>> credentials. In 4.1, we don't have this issue, >>since >> >>>>>the >> >>>>> >>>URL >> >>>>> >>> > > >>returned >> >>>>> >>> > > >> >>>>> is >> >>>>> >>> > > >> >>>>> the public web server location hosted in ssvm, >>and >> >>>>>in >> >>>>>4.2, >> >>>>> >>>we are >> >>>>> >>> > > >> >>>>> returning URL pointing to s3 object. Without >>setting >> >>>>>ACL >> >>>>> >>>to the S3 >> >>>>> >>> > > >> >>>>> object, user cannot directly click the URL >>returned >> >>>>>from >> >>>>> >>> > > >> >>>>> extractTemplate >> >>>>> >>> > > >> >>>>> api to download the template without providing >> >>>>> >>>credentials. By >> >>>>> >>> > > >> >>>>>reading >> >>>>> >>> > > >> >>>>> the AWS SDK doc today, I ran across the following >> >>>>>API >> >>>>>that >> >>>>> >>>I may >> >>>>> >>> > > >>be >> >>>>> >>> > > >> >>>>> able >> >>>>> >>> > > >> >>>>> to use for this purpose: >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>URL<http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL.h >>>>>>>>>>>>>>>>>tm >> >>>>>>>>>>>>>>>l >> >>>>>>>>>>>>>>>? >> >>>>>>>>>>>>>>>i >> >>>>>>>>>>>>>>>s- >> >>>>> >>>>>>>>>>e >> >>>>> >>>>>>>>>>xt >> >>>>> >>> > > >>>>>>>er >> >>>>> >>> > > >> >>>>>na >> >>>>> >>> > > >> >>>>> l= >> >>>>> >>> > > >> >>>>> true> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>generatePresignedUrl<http://docs.aws.amazon.com/AWSJavaSDK >>>>>>>>>>>>>>>>>/l >> >>>>>>>>>>>>>>>a >> >>>>>>>>>>>>>>>t >> >>>>>>>>>>>>>>>e >> >>>>>>>>>>>>>>>st >> >>>>> >>>>>>>>>>/ >> >>>>> >>>>>>>>>>ja >> >>>>> >>> > > >>>>>>>va >> >>>>> >>> > > >> >>>>>do >> >>>>> >>> > > >> >>>>> c/ >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>com/amazonaws/services/s3/AmazonS3Client.html#generatePres >>>>>>>>>>>>>>>>>ig >> >>>>>>>>>>>>>>>n >> >>>>>>>>>>>>>>>e >> >>>>>>>>>>>>>>>d >> >>>>>>>>>>>>>>>Ur >> >>>>> >>>>>>>>>>l >> >>>>> >>>>>>>>>>%2 >> >>>>> >>> > > >>>>>>>8j >> >>>>> >>> > > >> >>>>>av >> >>>>> >>> > > >> >>>>> a. >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>lang.String,%20java.lang.String,%20java.util.Date,%20com.a >>>>>>>>>>>>>>>>>ma >> >>>>>>>>>>>>>>>z >> >>>>>>>>>>>>>>>o >> >>>>>>>>>>>>>>>n >> >>>>>>>>>>>>>>>aw >> >>>>> >>>>>>>>>>s >> >>>>> >>>>>>>>>>.H >> >>>>> >>> > > >>>>>>>tt >> >>>>> >>> > > >> >>>>>pM >> >>>>> >>> > > >> >>>>> et >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>hod%29>(String<http://java.sun.com/j2se/1.5.0/docs/api/jav >>>>>>>>>>>>>>>>>a/ >> >>>>>>>>>>>>>>>l >> >>>>>>>>>>>>>>>a >> >>>>>>>>>>>>>>>n >> >>>>>>>>>>>>>>>g/ >> >>>>> >>>>>>>>>>S >> >>>>> >>>>>>>>>>tr >> >>>>> >>> > > >>>>>>>in >> >>>>> >>> > > >> >>>>>g. >> >>>>> >>> > > >> >>>>> ht >> >>>>> >>> > > >> >>>>> ml?is-external=true> bucketName, >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/S >>>>>>>>>>>>>>>>>tr >> >>>>>>>>>>>>>>>i >> >>>>>>>>>>>>>>>n >> >>>>>>>>>>>>>>>g >> >>>>>>>>>>>>>>>.h >> >>>>> >>>>>>>>>>t >> >>>>> >>>>>>>>>>ml >> >>>>> >>> > > >>>>>>>?i >> >>>>> >>> > > >> >>>>>s- >> >>>>> >>> > > >> >>>>> ex >> >>>>> >>> > > >> >>>>> ternal=true> key, >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>Date<http://java.sun.com/j2se/1.5.0/docs/api/java/util/Dat >>>>>>>>>>>>>>>>>e. >> >>>>>>>>>>>>>>>h >> >>>>>>>>>>>>>>>t >> >>>>>>>>>>>>>>>m >> >>>>>>>>>>>>>>>l? >> >>>>> >>>>>>>>>>i >> >>>>> >>>>>>>>>>s- >> >>>>> >>> > > >>>>>>>ex >> >>>>> >>> > > >> >>>>>te >> >>>>> >>> > > >> >>>>> rn >> >>>>> >>> > > >> >>>>> al=true> expiration, >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >> >>>>> >>> > > >> >>>>> >> >>>>>>>>>>>>>>>>>HttpMethod<http://docs.aws.amazon.com/AWSJavaSDK/latest/ja >>>>>>>>>>>>>>>>>va >> >>>>>>>>>>>>>>>d >> >>>>>>>>>>>>>>>o >> >>>>>>>>>>>>>>>c >> >>>>>>>>>>>>>>>/c >> >>>>> >>>>>>>>>>o >> >>>>> >>>>>>>>>>m/ >> >>>>> >>> > > >>>>>>>am >> >>>>> >>> > > >> >>>>>az >> >>>>> >>> > > >> >>>>> on >> >>>>> >>> > > >> >>>>> aws/HttpMethod.html> method) >> >>>>> >>> > > >> >>>>> Returns a pre-signed URL for accessing an >> >>>>>Amazon >> >>>>> >>>S3 >> >>>>> >>> > > >>resource. >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> This is along the same line as QSRA mentioned by >> >>>>>Tom, >> >>>>>by >> >>>>> >>>wrapped >> >>>>> >>> > > >>in >> >>>>> >>> > > >> >>>>> AmazonS3Client for easy consumption. By using >>this >> >>>>>method, >> >>>>> >>>I think >> >>>>> >>> > > >> >>>>> that I >> >>>>> >>> > > >> >>>>> don't need to change ACL of S3 object to open a >> >>>>>security >> >>>>> >>>hole. >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Thanks >> >>>>> >>> > > >> >>>>> -min >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> From: John Burwell >> >>>>> >>><jburw...@basho.com<mailto:jburw...@basho.com>> >> >>>>> >>> > > >> >>>>> Date: Monday, June 17, 2013 7:38 PM >> >>>>> >>> > > >> >>>>> To: Min Chen >> >>>>> >>><min.c...@citrix.com<mailto:min.c...@citrix.com>> >> >>>>> >>> > > >> >>>>> Cc: Thomas O'Dowd >> >>>>> >>> > > >> >>>>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>, >> >>>>> >>> > > >> >>>>> >> >>>>> >>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >> >>>>> >>> > > >> >>>>> >> >>>>> >>><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >> >>>>> >>> > > >> >>>>> Subject: Re: Query String Request >> >>>>>Authentication(QSRA) >> >>>>> >>>support by >> >>>>> >>> > > >>S3 >> >>>>> >>> > > >> >>>>> providers >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Min, >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Why are we mucking with ACLs at all? The best >> >>>>>security >> >>>>> >>>practice >> >>>>> >>> > > >> >>>>>would >> >>>>> >>> > > >> >>>>> be >> >>>>> >>> > > >> >>>>> to create a bucket for CloudStack's use and >>assign >> >>>>>it >> >>>>>a >> >>>>> >>>dedicated >> >>>>> >>> > > >> >>>>> access >> >>>>> >>> > > >> >>>>> key and secret key pair with read/write access >>only >> >>>>>to >> >>>>> >>>that >> >>>>> >>> > > >>bucket. >> >>>>> >>> > > >> >>>>> Requiring an administrative account to an object >> >>>>>store >> >>>>> >>>opens an >> >>>>> >>> > > >> >>>>> unnecessarily large attack surface. Therefore, >>as >> >>>>> >>>implemented in >> >>>>> >>> > > >> >>>>>4.1, >> >>>>> >>> > > >> >>>>> we >> >>>>> >>> > > >> >>>>> should defer bucket creation, ACL assignment, and >> >>>>> >>>credential >> >>>>> >>> > > >>creation >> >>>>> >>> > > >> >>>>> to >> >>>>> >>> > > >> >>>>> the administrator/operator. >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Thanks, >> >>>>> >>> > > >> >>>>> -John >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> On Jun 17, 2013, at 1:15 PM, Min Chen >> >>>>> >>> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> >> >>>>>wrote: >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Tom filed a very good bug for ACL setting change >>on >> >>>>>S3 >> >>>>> >>>object when >> >>>>> >>> > > >> >>>>> users >> >>>>> >>> > > >> >>>>> issue extractTemplate API >> >>>>> >>> > > >> >>>>> >> >>>>>(https://issues.apache.org/jira/browse/CLOUDSTACK-3030), >> >>>>> >>>and his >> >>>>> >>> > > >> >>>>> recommendation of using Query String Request >> >>>>> >>>Authentication (QSRA) >> >>>>> >>> > > >> >>>>> alternative sounds like a right approach to fix >>this >> >>>>>bug. >> >>>>> >>>Before >> >>>>> >>> > > >> >>>>> implementing it, I would like to confirm if QSRA >> >>>>>should be >> >>>>> >>> > > >>supported >> >>>>> >>> > > >> >>>>>by >> >>>>> >>> > > >> >>>>> all S3 providers if they claim that they are AWS >>s3 >> >>>>> >>>compatible. If >> >>>>> >>> > > >> >>>>>so, >> >>>>> >>> > > >> >>>>> we >> >>>>> >>> > > >> >>>>> will make this assumption in our code. Based on >>Tom, >> >>>>> >>>Cloudian is >> >>>>> >>> > > >> >>>>> supporting it. How about RiakCS, John? >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> Thanks >> >>>>> >>> > > >> >>>>> -min >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>>> >> >>>>> >>> > > >> >>>> >> >>>>> >>> > > >> >>> >> >>>>> >>> > > >> >> >> >>>>> >>> > > >> > >> >>>>> >>> > > >> >> >>>>> >>> > > > >> >>>>> >>> > > >-- >> >>>>> >>> > > >Cloudian KK - http://www.cloudian.com/get-started.html >> >>>>> >>> > > >Fancy 100TB of full featured S3 Storage? >> >>>>> >>> > > >Checkout the Cloudian(R) Community Edition! >> >>>>> >>> > > > >> >>>>> >>> > > >> >>>>> >>> > >> >>>>> >>> >> >>>>> >> >> >>>>> >>-- >> >>>>> >>Cloudian KK - http://www.cloudian.com/get-started.html >> >>>>> >>Fancy 100TB of full featured S3 Storage? >> >>>>> >>Checkout the Cloudian(R) Community Edition! >> >>>>> >> >> >>>>> > >> >>>>> >> >>>> >> >>>>-- >> >>>>Cloudian KK - http://www.cloudian.com/get-started.html >> >>>>Fancy 100TB of full featured S3 Storage? >> >>>>Checkout the Cloudian(R) Community Edition! >> >>>> >> >>> >> >> >> > >> > >-- >Cloudian KK - http://www.cloudian.com/get-started.html >Fancy 100TB of full featured S3 Storage? >Checkout the Cloudian(R) Community Edition! >
