There is no bucket created on my NFS mount. This is not an error specific to S3 browser tool.Even if I use this url in the IE browser, it gives 400-Bad request error.
When I give url as http://torvm-cloudstack-mgmt.sigmasys.net:7080/awsapi/rest/AmazonS3/test2342 It gives error like : <Error> <Code>NoSuchBucket</Code> <Message>The specified bucket does not exist</Message> <BucketName>test2342</BucketName> <RequestId>1DEADBEEF9</RequestId> <HostId>abCdeFgHiJ1k2LmN3op4q56r7st89</HostId> </Error> This is a normal behavior. Same behavior I should get when I don't pass any name of bucket. >>>>>>>>>>>>>1.....Please check if this is a bug in CS4. Now, when I create a bucket using the S3 AWS API on cloudstack I get error as - OutOfStorageException: Unable to create /s3storage/test123 for bucket test123 Stacktrace is- 2013-03-26 07:13:47,796 INFO [bridge.service.S3RestServlet] (catalina-exec-int-22:null) Request header x-amz-date:Tue, 26 Mar 2013 11:13:42 GMT 2013-03-26 07:13:47,796 INFO [bridge.service.S3RestServlet] (catalina-exec-int-22:null) Request header host:torvm-cloudstack-mgmt.sigmasys.net:7080 2013-03-26 07:13:47,796 INFO [bridge.service.S3RestServlet] (catalina-exec-int-22:null) Request header content-length:0 2013-03-26 07:13:47,796 INFO [bridge.service.S3RestServlet] (catalina-exec-int-22:null) Request header connection:Keep-Alive 2013-03-26 07:13:47,796 INFO [bridge.service.S3RestServlet] (catalina-exec-int-22:null) - End of request - 2013-03-26 07:13:47,800 DEBUG [bridge.service.UserContext] (catalina-exec-int-22:null) initializing a new [anonymous] UserContext! 2013-03-26 07:13:47,815 ERROR [bridge.service.S3RestServlet] (catalina-exec-int-22:null) Unexpected exception Unable to create /s3storage/test123 for bucket test123 com.cloud.bridge.service.exception.OutOfStorageException: Unable to create /s3storage/test123 for bucket test123 at com.cloud.bridge.io.S3FileSystemBucketAdapter.createContainer(S3FileSystemBucketAdapter.java:55) at com.cloud.bridge.service.core.s3.S3Engine.allocBucketStorageHost(S3Engine.java:1466) at com.cloud.bridge.service.core.s3.S3Engine.handleRequest(S3Engine.java:210) at com.cloud.bridge.service.controller.s3.S3BucketAction.executePutBucket(S3BucketAction.java:837) at com.cloud.bridge.service.controller.s3.S3BucketAction.execute(S3BucketAction.java:159) at com.cloud.bridge.service.S3RestServlet.processRequest(S3RestServlet.java:181) at com.cloud.bridge.service.S3RestServlet.doPut(S3RestServlet.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:640) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) What could be the reason? Thanks and Regards. Asmita -----Original Message----- From: Sebastien Goasguen [mailto:run...@gmail.com] Sent: 26 March 2013 PM 04:33 To: Asmita Vagyani Subject: Re: CS4 AWS S3 support On Mar 26, 2013, at 6:56 AM, Asmita Vagyani <asmita.vagy...@sigma-systems.com> wrote: > Hi Sebastien, > > Thanks , after giving the FQDN instead of IP address, it works. > Now, atleast the S3 browser is able to invoke the cloudstack S3 AWS REST API. > > I saw in the documentation > https://cwiki.apache.org/CLOUDSTACK/s3-api-in-cloudstack.html > It says to List All Buckets, simply make a call to : > REST GET request, http://myhost:8080/awsapi/rest/AmazonS3 > > However, when I invoke > http://torvm-cloudstack-mgmt.sigmasys.net:7080/awsapi/rest/AmazonS3 > I get the 400 Bad request error on browser. > I get the exception at backend when I checked awsapi.log: > I have never used the S3 browser but it looks like you may have some existing buckets in there that cause the error. > 2013-03-26 06:49:32,080 INFO [bridge.service.S3RestServlet] > (catalina-exec-int-20:null) Request servletPath: /rest/AmazonS3 > 2013-03-26 06:49:32,080 INFO [bridge.service.S3RestServlet] > (catalina-exec-int-20:null) Request header user-agent:S3 Browser3-8-1 > 2013-03-26 06:49:32,081 INFO [bridge.service.S3RestServlet] > (catalina-exec-int-20:null) Request header authorization:AWS > Qh69lnbtbr0H68i_Q7im7eZNhoFYUyHS8eDU9nyL2k3OP0gBGN0gNHfHWeKQo1EZq2r0MM > ZK-23U9_wF6PMojg:RWAiwif28JxZ/le9mDkQskfPAd8= > 2013-03-26 06:49:32,081 INFO [bridge.service.S3RestServlet] > (catalina-exec-int-20:null) Request header x-amz-date:Tue, 26 Mar 2013 > 10:49:26 GMT > 2013-03-26 06:49:32,081 INFO [bridge.service.S3RestServlet] > (catalina-exec-int-20:null) Request header > host:torvm-cloudstack-mgmt.sigmasys.net:7080 > 2013-03-26 06:49:32,081 INFO [bridge.service.S3RestServlet] > (catalina-exec-int-20:null) - End of request - > 2013-03-26 06:49:32,093 DEBUG [bridge.service.UserContext] > (catalina-exec-int-20:null) initializing a new [anonymous] UserContext! > 2013-03-26 06:49:32,093 ERROR [bridge.service.S3RestServlet] > (catalina-exec-int-20:null) Unexpected exception is not between 3 and > 255 characters long > com.cloud.bridge.service.exception.InvalidBucketName: is not between 3 and > 255 characters long > at > com.cloud.bridge.service.core.s3.S3Engine.verifyBucketName(S3Engine.java:1809) > at > com.cloud.bridge.service.S3RestServlet.routeRequest(S3RestServlet.java:440) > at > com.cloud.bridge.service.S3RestServlet.processRequest(S3RestServlet.java:179) > at com.cloud.bridge.service.S3RestServlet.doGet(S3RestServlet.java:84) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) > at > org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) > at > org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721) > at > org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > > Thanks and Regards. > > Asmita > > -----Original Message----- > From: Sebastien Goasguen [mailto:run...@gmail.com] > Sent: 26 March 2013 PM 04:06 > To: Asmita Vagyani > Subject: Re: CS4 AWS S3 support > > > On Mar 26, 2013, at 6:25 AM, Asmita Vagyani > <asmita.vagy...@sigma-systems.com> wrote: > >> Hi >> Where do I register the DNS entry I mean do you mean a local DNS entry or >> what? > > Local entry would do, as long as your "s3 browser" knows how to route > <server.com> to the IP of the endpoint. > >> >> Thanks and Regards. >> >> Asmita >> >> -----Original Message----- >> From: Sebastien Goasguen [mailto:run...@gmail.com] >> Sent: 26 March 2013 PM 03:34 >> To: Asmita Vagyani >> Subject: Re: CS4 AWS S3 support >> >> >> On Mar 26, 2013, at 4:40 AM, Asmita Vagyani >> <asmita.vagy...@sigma-systems.com> wrote: >> >>> Hi all, >>> >>> I tried to use a couple of freewares/tools like CyberDuck, DragonDisk, etc >>> which interfaces the Cloudstack S3 APIs. >>> I am stuck at one common point in all these tools. >>> All tools say they support the alternative S3 compatible services, which in >>> our case is our Cloudstack AWS S3. >>> However, it asks me endpoint of the S3 compatible service. >>> When I mention the endpoint as >>> http://<cloudstack-server-IP:port>/awsapi/rest/AmazonS3 it doesnot accept >>> it. >>> It wants the server name as server.com. >>> >> >> Asmita, I woud do two things: >> >> 1-Try to use boto like I explained in the slides, that should show you if >> the S3 service is working properly or not. >> 2-If the S3 tools that you are using do not take an IP, then you will need >> to register a dns entry. >> >> note that the S3 support is really a tech preview. >> >> -sebastien >> >>> Is there any way I can get the S3 webservice endpoint the way these tools >>> want. >>> Is this because Cloudstack is not consistent with the EC2 endpoint. >>> If yes, when are we planning to make it consistent. >>> >>> >>> Thanks and Regards. >>> >>> Asmita >>> >>> -----Original Message----- >>> From: Rajesh Battala [mailto:rajesh.batt...@citrix.com] >>> Sent: 25 March 2013 PM 09:40 >>> To: users@cloudstack.apache.org; 'cloudstack-us...@incubator.apache.org'; >>> 'run...@gmail.com' >>> Subject: RE: CS4 AWS S3 support >>> >>> Am not sure whether s3 browser allows user to configure the settings to >>> access the cloudstack s3. >>> If you can configure the settings the way we configure for boto then we can >>> use s3 browser. >>> >>> Thanks >>> Rajesh Battala >>> >>>> -----Original Message----- >>>> From: Asmita Vagyani [mailto:asmita.vagy...@sigma-systems.com] >>>> Sent: Monday, March 25, 2013 7:09 PM >>>> To: 'users@cloudstack.apache.org'; >>>> 'cloudstack-us...@incubator.apache.org'; >>>> 'run...@gmail.com' >>>> Subject: RE: CS4 AWS S3 support >>>> >>>> Are there any other clients, non-python based. >>>> Can I integrate this with S3 browser, such that the S3 browser >>>> shows me buckets created on the NFS drive mounted using >>>> cloud-bridge.properties? >>>> User can just upload and download files in S3 browser and >>>> ultimately on the NFS drive buckets. >>>> >>>> >>>> Thanks and Regards. >>>> >>>> Asmita >>>> >>>> -----Original Message----- >>>> From: Rajesh Battala [mailto:rajesh.batt...@citrix.com] >>>> Sent: 25 March 2013 PM 04:42 >>>> To: users@cloudstack.apache.org; >>>> 'cloudstack-us...@incubator.apache.org'; >>>> 'run...@gmail.com' >>>> Subject: RE: CS4 AWS S3 support >>>> >>>> Asmita, >>>> You can follow this link. >>>> https://cwiki.apache.org/CLOUDSTACK/s3-api-in- >>>> cloudstack.html >>>> >>>> You can use boto client which is python based to send s3 requests to CS. >>>> >>>> Thanks >>>> Rajesh Battala >>>> >>>>> -----Original Message----- >>>>> From: Asmita Vagyani [mailto:asmita.vagy...@sigma-systems.com] >>>>> Sent: Monday, March 25, 2013 4:13 PM >>>>> To: 'users@cloudstack.apache.org'; >>>>> 'cloudstack-us...@incubator.apache.org'; >>>>> 'run...@gmail.com' >>>>> Subject: RE: CS4 AWS S3 support >>>>> >>>>> Hi, >>>>> >>>>> I can see the entry of keys in the usercredential table in CS4 DB. >>>>> When I access http://<server-ip>/awsapi link I get the servlet exception : >>>>> >>>>> java.lang.RuntimeException: javax.servlet.ServletException: >>>>> Servlet execution threw an exception >>>>> >>>>> com.cloud.bridge.service.EC2MainServlet.doGetOrPost(EC2MainServlet >>>>> .java:104) >>>>> >>>>> com.cloud.bridge.service.EC2MainServlet.doGet(EC2MainServlet.java: >>>>> 7 >>>>> 3) >>>>> >>>>> How can I run S3 requests? any links to documentation would be helpful. >>>>> >>>>> Thanks and Regards. >>>>> >>>>> Asmita >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Rajesh Battala [mailto:rajesh.batt...@citrix.com] >>>>> Sent: 25 March 2013 PM 03:35 >>>>> To: users@cloudstack.apache.org; >>>>> 'cloudstack-us...@incubator.apache.org'; >>>>> 'run...@gmail.com' >>>>> Subject: RE: CS4 AWS S3 support >>>>> >>>>> Hi Asmita, >>>>> >>>>> In the cloudbridge db, usercredentials table has the >>>>> username/password values present? >>>>> Generally, for an existing account, generate the apikey/secrect >>>>> key and register them in awsapi. >>>>> >>>>> Were you able to run s3 requests.? Is awsap is running at 7080 port? >>>>> >>>>> Thanks >>>>> Rajesh Battala >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Asmita Vagyani [mailto:asmita.vagy...@sigma-systems.com] >>>>>> Sent: Monday, March 25, 2013 3:24 PM >>>>>> To: 'cloudstack-us...@incubator.apache.org'; 'run...@gmail.com' >>>>>> Subject: CS4 AWS S3 support >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I am trying to do following to have an implementation of S3 in >>>>>> our >>>> application. >>>>>> Followed steps given in: >>>>>> http://www.slideshare.net/sebastiengoasguen/cloudstack-s3 >>>>>> >>>>>> >>>>>> 1) I have enabled S3 support and EC2 support flag in the CS4 global >>>>> variables. >>>>>> >>>>>> 2) I have mounted the NFS store on the CS4 management server (I am >>>> using >>>>>> the NFS drive to store the buckets/files uploaded by user on this >>>>>> drive) >>>>>> >>>>>> 3) I have configured the cloud-bridge.properties appropriately >>>>>> >>>>>> host=http://localhost:7080/awsapi >>>>>> >>>>>> storage.root=/s3storage >>>>>> >>>>>> storage.multipartDir=__multipart__uploads__ >>>>>> >>>>>> bucket.dns=false >>>>>> >>>>>> serviceEndpoint=localhost:7080 >>>>>> >>>>>> >>>>>> >>>>>> 4) I made up a user account into CS4 client console, generated the >>>>>> API key >>>>>> and the Secret key. >>>>>> >>>>>> 5) I have registered the user by using cloudstack-aws-api-register >>>>>> (refer - >>>>>> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.0.1- >>>>>> incubating/html/Installation_Guide/aws-ec2-user-setup.html ) >>>>>> >>>>>> 6) I have downloaded the S3 browser. >>>>>> >>>>>> 7) I am trying to make an account with the same user API key and >>>>>> Secret >>>> key >>>>>> which we create on Cloudstack. >>>>>> >>>>>> While creating an account it says - InvalidAccesskeyId >>>>>> >>>>>> Any hints if I am missing some step here? >>>>>> >>>>>> Thanks and Regards. >>>>>> >>>>>> Asmita Patil Vagyani. >>>>>> >>>>>> Tech Lead, Sigma Software Solutions, Pune, >>>>>> >>>>>> T: +91-20-25661992 / 93 / 94. Extn. : 286 >>>>> >>>>> >>>> >>>> >>> >>> >>> >> >> >> > > >
