Hi Tom,
I can reproduce this issue using Cloudian, after investigation, I
realized that this is a bug in Amazon SDK we have used, based on this
thread:
http://stackoverflow.com/questions/15473582/amazon-s3-presigned-urls-escape
-the-slashes-in-the-key. When generatePresignedUrl is called it takes the
entire key and escapes it, and then creates a signature using the escaped
key. You cannot use the signature from the escaped key and combine it with
the unescaped key in the URL. See the bug code here:
String resourcePath = "/" +
((bucketName != null) ? bucketName + "/" : "") +
((key != null) ? ServiceUtils.urlEncode(key) : "") +
((subResource != null) ? "?" + subResource : "");
We have two options to fix this:
1. Either upgrade Amazon SDK to use 1.4.3 version, someone in that
thread
claimed that it is fixed in that version, but I haven't checked that.
Currently CloudStack is using 1.3.21. Not sure if this will break
CloudStack cloud_bridge.
2. Workaround by creating customized AmazonS3Client to change the
internal implementation on this.
Thanks
-min
On 7/2/13 11:31 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote:
>Excellent. The link is there now. Thank you Min. I verified that bug and
>closed it.
>
>However - now that I can finally click the download link... I ran into
>the issue that the link doesn't work on AWS or Cloudian. Please see this
>bug for details (latest 4.2 updates included in my test).
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-3341
>
>Tom.
>
>On Tue, 2013-07-02 at 22:54 +0000, Min Chen wrote:
>> Hi Tom,
>> I investigated this issue through the db dump you provided in the bug,
>> this is an issue with our db view template_view creation script, and it
>> has been fixed in resolving
>> another bug (https://issues.apache.org/jira/browse/CLOUDSTACK-3314). I
>> have verified the fix using your db dump on my local setup. Please check
>> out latest 4.2 or master code to try again.
>>
>> Thanks
>> -min
>>
>> On 7/2/13 2:18 PM, "Min Chen" <min.c...@citrix.com> wrote:
>>
>> >Tom, this seems like an issue with entry stored in our DB. I will take
>>a
>> >look at this bug and update you. Just to clarify, this symptom only
>> >happens when you register these templates to Amazon S3, not for
>>Cloudian
>> >or RiakCS S3, right?
>> >
>> >Thanks
>> >-min
>> >
>> >On 7/1/13 7:27 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote:
>> >
>> >>Yes thanks Jessica. I re-opened the bug again. I know its not a gui
>> >>problem per-say in that the template is not ready to show the download
>> >>link. However, it never becomes ready is the actual problem. What sets
>> >>the "isready" property to true? As far as I can see, the objects in
>>the
>> >>S3 stores (AWS or Cloudian) are complete and from my perspective
>>"ready"
>> >>to download/use. It sounds like a bug when registering the template.
>> >>
>> >>Tom.
>> >>
>> >>On Mon, 2013-07-01 at 18:54 +0000, Jessica Wang wrote:
>> >>> Thomas,
>> >>>
>> >>> I checked the data you provided.
>> >>>
>> >>> The reason that the 2 templates("MyTiny", "AnotherTiny") have no
>> >>>download button is because they are not ready
>> >>> (i.e. their "isready" property is false).
>> >>>
>> >>> Download button is only available when "isready" property is true.
>> >>>
>> >>> Jessica
>> >>>
>> >>> -----Original Message-----
>> >>> From: Thomas O'Dowd [mailto:tpod...@cloudian.com]
>> >>> Sent: Thursday, June 27, 2013 8:04 PM
>> >>> To: Min Chen
>> >>> Cc: dev@cloudstack.apache.org; Jessica Wang
>> >>> Subject: Re: Query String Request Authentication(QSRA) support by S3
>> >>>providers
>> >>>
>> >>> Hi Min/Jessica,
>> >>>
>> >>> I attached an image to that issue to show what what my browser is
>> >>> showing.
>> >>>
>> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-3220
>> >>>
>> >>> Tom.
>> >>>
>> >>> On Fri, 2013-06-28 at 09:45 +0900, Thomas O'Dowd wrote:
>> >>> > Hi Min,
>> >>> >
>> >>> > Yes. I'll try it again today to check again but when I added
>>Amazon
>> >>>S3
>> >>> > as the S3 secondary storage and uploaded a template, I was not
>>shown
>> >>>the
>> >>> > "download template" link. However - for Cloudian S3, I am shown
>>it so
>> >>> > I'm wondering why.
>> >>> >
>> >>> > Tom.
>> >>> >
>> >>> > On Fri, 2013-06-28 at 00:26 +0000, Min Chen wrote:
>> >>> > > Hi Tom,
>> >>> > >
>> >>> > > Are you saying that you cannot see a Download Template button
>>from
>> >>>UI
>> >>> > > when Amazon S3 is added as secondary storage? I only tested with
>> >>>RiakCS
>> >>> > > and Cloudian, so didn't see this issue. But I am CC Jessica her
>>to
>> >>>confirm
>> >>> > > what special handling is done in UI to enable/disable a button
>>from
>> >>>UI.
>> >>> > >
>> >>> > > Thanks
>> >>> > > -min
>> >>> > >
>> >>> > > On 6/27/13 5:23 PM, "Thomas O'Dowd" <tpod...@cloudian.com>
>>wrote:
>> >>> > >
>> >>> > > >Hi Min,
>> >>> > > >
>> >>> > > >Can you check this bug? I'm trying to test this feature for
>>Amazon
>> >>>but
>> >>> > > >having no luck getting the Download template link/button to
>> >>>appear.
>> >>> > > >
>> >>> > > >https://issues.apache.org/jira/browse/CLOUDSTACK-3220
>> >>> > > >
>> >>> > > >Thanks,
>> >>> > > >
>> >>> > > >Tom.
>> >>> > > >
>> >>> > > >On Fri, 2013-06-21 at 17:21 +0000, Min Chen wrote:
>> >>> > > >> John,
>> >>> > > >>
>> >>> > > >> For S3, the api call createEntityExtractUrl is done on
>> >>>management
>> >>> > > >>server
>> >>> > > >> side; while for NFS secondary storage, if the implementation
>>of
>> >>> > > >> createEntityExtractUrl will involve some code be executed in
>> >>>ssvm to
>> >>> > > >>copy
>> >>> > > >> template from the install location to a public accessible web
>> >>>server
>> >>> > > >> location.
>> >>> > > >> I don't quite understand some of your comments below. This
>>API
>> >>>is not
>> >>> > > >> used to write any information to S3 bucket/directory. This is
>> >>>used for
>> >>> > > >> object already existed on S3, and we just provide a URL for
>>user
>> >>>to
>> >>> > > >> download a template from S3, just like how Amazon provided
>>user
>> >>>a way to
>> >>> > > >> user to extract a S3 object through generatePresignedUrl. We
>>can
>> >>>discuss
>> >>> > > >> more on this on collaboration conference.
>> >>> > > >>
>> >>> > > >> Thanks
>> >>> > > >> -min
>> >>> > > >>
>> >>> > > >>
>> >>> > > >>
>> >>> > > >> On 6/21/13 7:25 AM, "John Burwell" <jburw...@basho.com>
>>wrote:
>> >>> > > >>
>> >>> > > >> >Min,
>> >>> > > >> >
>> >>> > > >> >(I apologize for my belated reply -- I lost track of this
>>draft
>> >>>in the
>> >>> > > >> >chaos of the last couple of days.)
>> >>> > > >> >
>> >>> > > >> >Upon further review, I think I feel into the confusion
>>between
>> >>> > > >>management
>> >>> > > >> >server and ssvm. This code is executing on the management
>> >>>server side,
>> >>> > > >> >correct? Based on my "corrected" understanding is correct,
>>I
>> >>>would
>> >>> > > >>like
>> >>> > > >> >to amend my thoughts. Namely, I would like to see the
>>driver
>> >>> > > >>operations
>> >>> > > >> >pushed out to the SSVM where we can use the stream. As I
>>think
>> >>>about
>> >>> > > >>it,
>> >>> > > >> >the management server should not need to interact with the
>> >>>driver.
>> >>> > > >> >Simply yard up the DataStore attributes + details map and
>>other
>> >>>extract
>> >>> > > >> >parameters, and send them to the SSVM. Using this
>>information,
>> >>>the S3
>> >>> > > >> >driver could open a stream to write the template out to the
>> >>> > > >> >bucket/directory. I recognize it changes the protocol
>>between
>> >>>the
>> >>> > > >> >management server and SSVM, but it simply both sides of the
>> >>>operation
>> >>> > > >>by
>> >>> > > >> >allowing the DataStore information to be treated opaquely
>>until
>> >>>it is
>> >>> > > >> >consumed by the driver to execute the write operation. I
>>also
>> >>> > > >>recognize
>> >>> > > >> >that we may a little late in the cycle to address it for
>>4.2,
>> >>>and it
>> >>> > > >>may
>> >>> > > >> >need to be part of the 4.3 enhancements.
>> >>> > > >> >
>> >>> > > >> >Thanks,
>> >>> > > >> >-John
>> >>> > > >> >
>> >>> > > >> >On Jun 18, 2013, at 3:55 PM, Min Chen <min.c...@citrix.com>
>> >>>wrote:
>> >>> > > >> >
>> >>> > > >> >> John,
>> >>> > > >> >> In that case, how do we keep backward compatibility of
>> >>> > > >>extractTemplate
>> >>> > > >> >> api, which requires a URL in the response?
>> >>> > > >> >>
>> >>> > > >> >> Thanks
>> >>> > > >> >> -min
>> >>> > > >> >>
>> >>> > > >> >> On 6/18/13 11:53 AM, "John Burwell" <jburw...@basho.com>
>> >>>wrote:
>> >>> > > >> >>
>> >>> > > >> >>> Min,
>> >>> > > >> >>>
>> >>> > > >> >>> Looking through the code, I think we can simplify driver
>> >>>operation
>> >>> > > >>and
>> >>> > > >> >>> increase robustness by changing
>> >>> > > >> >>>ImageStoreDriver#createEntityExtractUrl()
>> >>> > > >> >>> : String to ImageStoreDriver#readEntity(Š) : InputStream.
>> >>>My first
>> >>> > > >> >>> concern with the current implementation is that it
>> >>>circumvents any
>> >>> > > >> >>> connection pooling/resource management underlying client
>> >>>libraries
>> >>> > > >> >>> provide. I/O streams provide a higher-level abstraction
>> >>>that allows
>> >>> > > >> >>> drivers to provide the orchestration components with
>>actual
>> >>> > > >>resources
>> >>> > > >> >>> rather String references. Second, the current interface
>> >>>seems to
>> >>> > > >> >>>appears
>> >>> > > >> >>> to assume that an http/https URL will be returned. With
>>I/O
>> >>> > > >>streams,
>> >>> > > >> >>>we
>> >>> > > >> >>> can support any client library capable of using the
>>standard
>> >>>I/O
>> >>> > > >> >>> framework -- enabling us to support other protocols for
>> >>>downloading
>> >>> > > >> >>> templates in the future (e.g. RBD, local filesystem, NBD,
>> >>>etc).
>> >>> > > >> >>>
>> >>> > > >> >>> Thanks,
>> >>> > > >> >>> -John
>> >>> > > >> >>>
>> >>> > > >> >>> On Jun 18, 2013, at 1:11 PM, Min Chen
>><min.c...@citrix.com>
>> >>>wrote:
>> >>> > > >> >>>
>> >>> > > >> >>>> A new version of using generatePresignedUrl in
>> >>> > > >>S3ImageStoreDriverImpl
>> >>> > > >> >>>>is
>> >>> > > >> >>>> checked into object_store.
>> >>> > > >> >>>>
>> >>> > > >> >>>> THanks
>> >>> > > >> >>>> -min
>> >>> > > >> >>>>
>> >>> > > >> >>>> On 6/18/13 8:29 AM, "Min Chen" <min.c...@citrix.com>
>>wrote:
>> >>> > > >> >>>>
>> >>> > > >> >>>>> Yes, current code is in
>> >>> > > >> >>>>>S3ImageStoreDriverImpl.createEntityExtractUrl,
>> >>> > > >> >>>>> which has a security issue mentioned in
>>CLOUDSTACK-3030. I
>> >>>am
>> >>> > > >>going
>> >>> > > >> >>>>>to
>> >>> > > >> >>>>> change it to use generatePresignedUrl api from AWS S3
>>api.
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Thanks
>> >>> > > >> >>>>> -min
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> From: John Burwell
>> >>><jburw...@basho.com<mailto:jburw...@basho.com>>
>> >>> > > >> >>>>> Date: Tuesday, June 18, 2013 8:07 AM
>> >>> > > >> >>>>> To: Min Chen
>> >>><min.c...@citrix.com<mailto:min.c...@citrix.com>>
>> >>> > > >> >>>>> Cc: Thomas O'Dowd
>> >>> > > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>,
>> >>> > > >> >>>>>
>> >>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
>> >>> > > >> >>>>>
>> >>><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
>> >>> > > >> >>>>> Subject: Re: Query String Request Authentication(QSRA)
>> >>>support by
>> >>> > > >>S3
>> >>> > > >> >>>>> providers
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Min,
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Is the code checked into the object_store branch? If
>>so,
>> >>>which
>> >>> > > >>lines
>> >>> > > >> >>>>> in
>> >>> > > >> >>>>> S3TemplateDownloader?
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Thanks,
>> >>> > > >> >>>>> -John
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> On Jun 18, 2013, at 12:39 AM, Min Chen
>> >>> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>>
>>wrote:
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Hi John,
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> This is regarding extractTemplate api, where for
>> >>>extractable
>> >>> > > >> >>>>>template,
>> >>> > > >> >>>>> users can click "Download Template" button from UI to
>>get
>> >>>a http
>> >>> > > >>url
>> >>> > > >> >>>>>to
>> >>> > > >> >>>>> download the template already stored at S3 without
>> >>>providing S3
>> >>> > > >> >>>>> credentials. In 4.1, we don't have this issue, since
>>the
>> >>>URL
>> >>> > > >>returned
>> >>> > > >> >>>>> is
>> >>> > > >> >>>>> the public web server location hosted in ssvm, and in
>>4.2,
>> >>>we are
>> >>> > > >> >>>>> returning URL pointing to s3 object. Without setting
>>ACL
>> >>>to the S3
>> >>> > > >> >>>>> object, user cannot directly click the URL returned
>>from
>> >>> > > >> >>>>> extractTemplate
>> >>> > > >> >>>>> api to download the template without providing
>> >>>credentials. By
>> >>> > > >> >>>>>reading
>> >>> > > >> >>>>> the AWS SDK doc today, I ran across the following API
>>that
>> >>>I may
>> >>> > > >>be
>> >>> > > >> >>>>> able
>> >>> > > >> >>>>> to use for this purpose:
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>URL<http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL.html?i
>>>>>>>>>>>>s-
>> >>>>>>>>>>e
>> >>>>>>>>>>xt
>> >>> > > >>>>>>>er
>> >>> > > >> >>>>>na
>> >>> > > >> >>>>> l=
>> >>> > > >> >>>>> true>
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>generatePresignedUrl<http://docs.aws.amazon.com/AWSJavaSDK/late
>>>>>>>>>>>>st
>> >>>>>>>>>>/
>> >>>>>>>>>>ja
>> >>> > > >>>>>>>va
>> >>> > > >> >>>>>do
>> >>> > > >> >>>>> c/
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>com/amazonaws/services/s3/AmazonS3Client.html#generatePresigned
>>>>>>>>>>>>Ur
>> >>>>>>>>>>l
>> >>>>>>>>>>%2
>> >>> > > >>>>>>>8j
>> >>> > > >> >>>>>av
>> >>> > > >> >>>>> a.
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>lang.String,%20java.lang.String,%20java.util.Date,%20com.amazon
>>>>>>>>>>>>aw
>> >>>>>>>>>>s
>> >>>>>>>>>>.H
>> >>> > > >>>>>>>tt
>> >>> > > >> >>>>>pM
>> >>> > > >> >>>>> et
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>hod%29>(String<http://java.sun.com/j2se/1.5.0/docs/api/java/lan
>>>>>>>>>>>>g/
>> >>>>>>>>>>S
>> >>>>>>>>>>tr
>> >>> > > >>>>>>>in
>> >>> > > >> >>>>>g.
>> >>> > > >> >>>>> ht
>> >>> > > >> >>>>> ml?is-external=true> bucketName,
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String
>>>>>>>>>>>>.h
>> >>>>>>>>>>t
>> >>>>>>>>>>ml
>> >>> > > >>>>>>>?i
>> >>> > > >> >>>>>s-
>> >>> > > >> >>>>> ex
>> >>> > > >> >>>>> ternal=true> key,
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>Date<http://java.sun.com/j2se/1.5.0/docs/api/java/util/Date.htm
>>>>>>>>>>>>l?
>> >>>>>>>>>>i
>> >>>>>>>>>>s-
>> >>> > > >>>>>>>ex
>> >>> > > >> >>>>>te
>> >>> > > >> >>>>> rn
>> >>> > > >> >>>>> al=true> expiration,
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >>
>> >>> > >
>>
>>>>>>>>>>>>HttpMethod<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc
>>>>>>>>>>>>/c
>> >>>>>>>>>>o
>> >>>>>>>>>>m/
>> >>> > > >>>>>>>am
>> >>> > > >> >>>>>az
>> >>> > > >> >>>>> on
>> >>> > > >> >>>>> aws/HttpMethod.html> method)
>> >>> > > >> >>>>> Returns a pre-signed URL for accessing an
>>Amazon
>> >>>S3
>> >>> > > >>resource.
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> This is along the same line as QSRA mentioned by Tom,
>>by
>> >>>wrapped
>> >>> > > >>in
>> >>> > > >> >>>>> AmazonS3Client for easy consumption. By using this
>>method,
>> >>>I think
>> >>> > > >> >>>>> that I
>> >>> > > >> >>>>> don't need to change ACL of S3 object to open a
>>security
>> >>>hole.
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Thanks
>> >>> > > >> >>>>> -min
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> From: John Burwell
>> >>><jburw...@basho.com<mailto:jburw...@basho.com>>
>> >>> > > >> >>>>> Date: Monday, June 17, 2013 7:38 PM
>> >>> > > >> >>>>> To: Min Chen
>> >>><min.c...@citrix.com<mailto:min.c...@citrix.com>>
>> >>> > > >> >>>>> Cc: Thomas O'Dowd
>> >>> > > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>,
>> >>> > > >> >>>>>
>> >>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
>> >>> > > >> >>>>>
>> >>><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
>> >>> > > >> >>>>> Subject: Re: Query String Request Authentication(QSRA)
>> >>>support by
>> >>> > > >>S3
>> >>> > > >> >>>>> providers
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Min,
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Why are we mucking with ACLs at all? The best security
>> >>>practice
>> >>> > > >> >>>>>would
>> >>> > > >> >>>>> be
>> >>> > > >> >>>>> to create a bucket for CloudStack's use and assign it a
>> >>>dedicated
>> >>> > > >> >>>>> access
>> >>> > > >> >>>>> key and secret key pair with read/write access only to
>> >>>that
>> >>> > > >>bucket.
>> >>> > > >> >>>>> Requiring an administrative account to an object store
>> >>>opens an
>> >>> > > >> >>>>> unnecessarily large attack surface. Therefore, as
>> >>>implemented in
>> >>> > > >> >>>>>4.1,
>> >>> > > >> >>>>> we
>> >>> > > >> >>>>> should defer bucket creation, ACL assignment, and
>> >>>credential
>> >>> > > >>creation
>> >>> > > >> >>>>> to
>> >>> > > >> >>>>> the administrator/operator.
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Thanks,
>> >>> > > >> >>>>> -John
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> On Jun 17, 2013, at 1:15 PM, Min Chen
>> >>> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>>
>>wrote:
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Tom filed a very good bug for ACL setting change on S3
>> >>>object when
>> >>> > > >> >>>>> users
>> >>> > > >> >>>>> issue extractTemplate API
>> >>> > > >> >>>>>
>>(https://issues.apache.org/jira/browse/CLOUDSTACK-3030),
>> >>>and his
>> >>> > > >> >>>>> recommendation of using Query String Request
>> >>>Authentication (QSRA)
>> >>> > > >> >>>>> alternative sounds like a right approach to fix this
>>bug.
>> >>>Before
>> >>> > > >> >>>>> implementing it, I would like to confirm if QSRA
>>should be
>> >>> > > >>supported
>> >>> > > >> >>>>>by
>> >>> > > >> >>>>> all S3 providers if they claim that they are AWS s3
>> >>>compatible. If
>> >>> > > >> >>>>>so,
>> >>> > > >> >>>>> we
>> >>> > > >> >>>>> will make this assumption in our code. Based on Tom,
>> >>>Cloudian is
>> >>> > > >> >>>>> supporting it. How about RiakCS, John?
>> >>> > > >> >>>>>
>> >>> > > >> >>>>> Thanks
>> >>> > > >> >>>>> -min
>> >>> > > >> >>>>>
>> >>> > > >> >>>>>
>> >>> > > >> >>>>
>> >>> > > >> >>>
>> >>> > > >> >>
>> >>> > > >> >
>> >>> > > >>
>> >>> > > >
>> >>> > > >--
>> >>> > > >Cloudian KK - http://www.cloudian.com/get-started.html
>> >>> > > >Fancy 100TB of full featured S3 Storage?
>> >>> > > >Checkout the Cloudian(R) Community Edition!
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>> >>--
>> >>Cloudian KK - http://www.cloudian.com/get-started.html
>> >>Fancy 100TB of full featured S3 Storage?
>> >>Checkout the Cloudian(R) Community Edition!
>> >>
>> >
>>
>
>--
>Cloudian KK - http://www.cloudian.com/get-started.html
>Fancy 100TB of full featured S3 Storage?
>Checkout the Cloudian(R) Community Edition!
>
