Re: [Openstack] Fwd: Question about cloudfiles API
On 5/30/2012 4:02 PM, Luis Gervaso wrote: Hi Shawn, I discovered in your code: 1) properties.put(identity.endp oint.publicURL, http:/testb.REDACTED.com:8080/v2.0 http://testb.REDACTED.com:8080/v2.0); here you are using only one slash http:/. So it's not valid URL. 2) actually testb.REDACTED.com:8080/v2.0 http://testb.REDACTED.com:8080/v2.0 is your identity endpoint? Keystone usually listen on 5000 or 35357 I fixed the URL in my code. It's still failing, though now it is failing in a different way. I am not using keystone, I am using the plain swift proxy server. Keystone is not part of the multiserver howto: http://swift.openstack.org/howto_installmultinode.html Looking at the docs for keystone, I cannot see how to make it fault tolerant. The entire point of trying swift out is fault tolerance, so if there is no documentation on fault tolerance for keystone, I can't use it even if I can figure out how to run it. How do I use the openstack java sdk with the proxy server that I've already got running? Is there a different java API that I can use instead? I've already determined that the rackspace cloudfiles API is slow for storing documents. I am working on a test for retrieving documents. Thanks, Shawn ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Question about cloudfiles API
Hi Shawn, I understand your needs. I'll try to add support for them in the RC3. It'll be out next week. Luis On Mon, Jun 4, 2012 at 8:49 PM, Shawn Heisey launch...@elyograg.org wrote: On 5/30/2012 4:02 PM, Luis Gervaso wrote: Hi Shawn, I discovered in your code: 1) properties.put(identity.endp oint.publicURL, http:/testb.REDACTED.com:8080/v2.0); here you are using only one slash http:/ http:/. So it's not valid URL. 2) actually testb.REDACTED.com:8080/v2.0 is your identity endpoint? Keystone usually listen on 5000 or 35357 I fixed the URL in my code. It's still failing, though now it is failing in a different way. I am not using keystone, I am using the plain swift proxy server. Keystone is not part of the multiserver howto: http://swift.openstack.org/howto_installmultinode.html Looking at the docs for keystone, I cannot see how to make it fault tolerant. The entire point of trying swift out is fault tolerance, so if there is no documentation on fault tolerance for keystone, I can't use it even if I can figure out how to run it. How do I use the openstack java sdk with the proxy server that I've already got running? Is there a different java API that I can use instead? I've already determined that the rackspace cloudfiles API is slow for storing documents. I am working on a test for retrieving documents. Thanks, Shawn ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ luis.gerv...@gmail.comwoorea.es ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Question about cloudfiles API
On 5/29/2012 5:51 PM, Luis Gervaso wrote:
Good point,
I'll prepare a benchmark
Until RC3 you can build from source (2 steps)
I installed maven ... that required a huge number of dependencies. I
was able to get the source for the openstack java sdk compiled using the
openjdk on ubuntu. Taking that jarfile to my test client (sun java6 on
centos 6), working my way through dependencies, I could not get it to
actually work, so I must be doing something wrong. A sniffer trace at
the proxy server shows nothing at all, and below is what it spits out,
slightly redacted. What other information do you need?
May 30, 2012 10:57:49 AM org.glassfish.jersey.filter.LoggingFilter log
INFO: 1 * LoggingFilter - Request received on thread main
1 POST http:/testb.REDACTED.com:8080/auth/v1.0/tokens
1 Accept: application/json
1 Content-Type: application/json
{auth:{passwordCredentials:{username:system:root,password:testpass},credentials:{username:system:root,password:testpass}}}
InvocationException{message=null, response=null}
at
org.glassfish.jersey.client.JerseyClient$1.failure(JerseyClient.java:251)
at
org.glassfish.jersey.process.internal.ResponseProcessor.notifyCallback(ResponseProcessor.java:334)
at
org.glassfish.jersey.process.internal.ResponseProcessor.setResult(ResponseProcessor.java:321)
at
org.glassfish.jersey.process.internal.ResponseProcessor.access$400(ResponseProcessor.java:86)
at
org.glassfish.jersey.process.internal.ResponseProcessor$1.run(ResponseProcessor.java:225)
at
org.glassfish.jersey.process.internal.ResponseProcessor.runInScope(ResponseProcessor.java:262)
at
org.glassfish.jersey.process.internal.ResponseProcessor.run(ResponseProcessor.java:198)
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
at
com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
at
com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
at
com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
at
org.glassfish.jersey.process.internal.RequestInvoker$2.runInScope(RequestInvoker.java:215)
at
org.glassfish.jersey.process.internal.RequestInvoker$2.run(RequestInvoker.java:188)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
at
com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:44)
at
com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:41)
at
org.glassfish.jersey.process.internal.RequestInvoker.apply(RequestInvoker.java:221)
at
org.glassfish.jersey.client.JerseyClient.submit(JerseyClient.java:240)
at
org.glassfish.jersey.client.JerseyInvocation.submit(JerseyInvocation.java:589)
at
org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:544)
at
org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:335)
at
org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:248)
at
org.openstack.api.identity.resources.TokensResource.post(TokensResource.java:21)
at
org.openstack.client.OpenStackClient.authenticate(OpenStackClient.java:76)
at com.REDACTED.swiftest.Main.main(Main.java:58)
Caused by: java.lang.NullPointerException
at sun.net.www.ParseUtil.toURI(ParseUtil.java:261)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:787)
at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:718)
at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:896)
at
org.glassfish.jersey.client.HttpUrlConnector$1$1.getOutputStream(HttpUrlConnector.java:140)
at
org.glassfish.jersey.internal.util.CommittingOutputStream.commitWrite(CommittingOutputStream.java:150)
at
org.glassfish.jersey.internal.util.CommittingOutputStream.write(CommittingOutputStream.java:120)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at
Re: [Openstack] Fwd: Question about cloudfiles API
On 5/29/2012 5:51 PM, Luis Gervaso wrote: Good point, I'll prepare a benchmark Until RC3 you can build from source (2 steps) #git clone #mvn clean install As I mentioned in my last email, I am getting an exception, see that email for full exception details. This is the code in my little application that's failing. Originally I had /auth/v1.0 (from the swift howto) instead of /v2.0, but I can see in the openstack api docs that /v2.0 is a correct API URL, and the openstack sdk examples have /v2.0. String username = system:root; String password = testpass; Properties properties = System.getProperties(); properties.put(verbose, true); // properties.put(auth.credentials, passwordCredentials); properties.put(auth.username, username); properties.put(auth.password, password); // properties.put(auth.tenantName, demo); properties.put(identity.endpoint.publicURL, http:/testb.REDACTED.com:8080/v2.0); OpenStackClient openstack = OpenStackClient .authenticate(properties); Here are the jarfiles that I have placed in the classpath. Some of these I was able to download, some of them had to come from my .m2 maven directory: asm-all-repackaged-2.0.3.jar auto-depends-2.0.3.jar commons-cli-1.2.jar commons-codec-1.6.jar commons-io-2.2.jar commons-lang-2.6.jar commons-logging-1.1.1.jar guava-12.0.jar hk2-api-2.0.3.jar jackson-core-asl-1.9.2.jar jackson-jaxrs-1.9.2.jar jackson-mapper-asl-1.9.2.jar jackson-xc-1.9.2.jar javax.inject-2.0.3.jar javax.ws.rs-api-2.0-m05.jar jersey-client-2.0-m04.jar jersey-common-2.0-m04.jar jersey-media-json-2.0-m04.jar mailapi.jar openstack-java-sdk-1.0-RC2.jar osgi-resource-locator-1.0.1.jar Thanks, Shawn ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Question about cloudfiles API
Hi Shawn,
I discovered in your code:
1)
properties.put(identity.endp
oint.publicURL,
http:/testb.REDACTED.com:8080/v2.0);
here you are using only one slash http:/. So it's not valid URL.
2) actually testb.REDACTED.com:8080/v2.0 is your identity endpoint?
Keystone usually listen on 5000 or 35357
At the end of this mail you have a complete log from OpenStack Java SDK
talking Swift running on Folsom. Actually it's tested on HPCloud (Diablo)
and DevStack (stable/essex) as well.
3) Thanks, you discovered a bug. The code was trying to authenticate with
duplicate credentials. It was working but I have fixed! Already pushed on
github
Cheers!
TEST LOGS
May 30, 2012 11:50:38 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 1 * LoggingFilter - Request received on thread main
1 POST http://192.168.1.43:5000/v2.0/tokens
1 Accept: application/json
1 Content-Type: application/json
{auth:{tenantName:admin,passwordCredentials:{username:admin,password:secret0}}}
May 30, 2012 11:50:39 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 2 * LoggingFilter - Response received on thread main
2 200
2 Vary: X-Auth-Token
2 Date: Wed, 30 May 2012 21:50:39 GMT
2 Content-Length: 2541
2 Content-Type: application/json
2 Connection: keep-alive
{access: {token: {expires: 2012-05-31T21:50:39Z, id:
cedec9abd4c249479181c2c6d9c0ad85, tenant: {enabled: true, id:
1e9c9a2fe2454950a900202b1e82b54f, name: admin, description: null}},
serviceCatalog: [{endpoints: [{adminURL:
http://192.168.1.43:8774/v2/1e9c9a2fe2454950a900202b1e82b54f;, region:
RegionOne, publicURL:
http://192.168.1.43:8774/v2/1e9c9a2fe2454950a900202b1e82b54f;,
internalURL: http://192.168.1.43:8774/v2/1e9c9a2fe2454950a900202b1e82b54f}],
endpoints_links: [], type: compute, name: Compute Service},
{endpoints: [{adminURL: http://192.168.1.43:8080;, region:
RegionOne, publicURL: http://192.168.1.43:8080;, internalURL:
http://192.168.1.43:8080}], endpoints_links: [], type: s3, name:
S3 Service}, {endpoints: [{adminURL: http://192.168.1.43:9292;,
region: RegionOne, publicURL: http://192.168.1.43:9292;,
internalURL: http://192.168.1.43:9292}], endpoints_links: [], type:
image, name: Image Service}, {endpoints: [{adminURL:
http://192.168.1.43:8776/v1/1e9c9a2fe2454950a900202b1e82b54f;, region:
RegionOne, publicURL:
http://192.168.1.43:8776/v1/1e9c9a2fe2454950a900202b1e82b54f;,
internalURL: http://192.168.1.43:8776/v1/1e9c9a2fe2454950a900202b1e82b54f}],
endpoints_links: [], type: volume, name: Volume Service},
{endpoints: [{adminURL: http://192.168.1.43:8773/services/Admin;,
region: RegionOne, publicURL: http://192.168.1.43:8773/services/Cloud;,
internalURL: http://192.168.1.43:8773/services/Cloud}],
endpoints_links: [], type: ec2, name: EC2 Service}, {endpoints:
[{adminURL: http://192.168.1.43:8080/;, region: RegionOne,
publicURL:
http://192.168.1.43:8080/v1/AUTH_1e9c9a2fe2454950a900202b1e82b54f;,
internalURL:
http://192.168.1.43:8080/v1/AUTH_1e9c9a2fe2454950a900202b1e82b54f}],
endpoints_links: [], type: object-store, name: Swift Service},
{endpoints: [{adminURL: http://192.168.1.43:35357/v2.0;, region:
RegionOne, publicURL: http://192.168.1.43:5000/v2.0;, internalURL:
http://192.168.1.43:5000/v2.0}], endpoints_links: [], type:
identity, name: Identity Service}], user: {username: admin,
roles_links: [], id: e357f1b90efb4196af7d375ff7af21c6, roles:
[{id: 5f9c2353722741c58bc3c09dfeb32759, name: KeystoneAdmin},
{id: bf2e2b36928e42eb8ed977ba8b470e77, name: admin}, {id:
b6b1b22f0fc640dab1958d8eb977625b, name: KeystoneServiceAdmin}],
name: admin}}}
May 30, 2012 11:50:39 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 1 * LoggingFilter - Request received on thread main
1 PUT
http://192.168.1.43:8080/v1/AUTH_1e9c9a2fe2454950a900202b1e82b54f/test-container
1 Accept: application/json
1 X-Auth-Token: cedec9abd4c249479181c2c6d9c0ad85
May 30, 2012 11:50:39 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 2 * LoggingFilter - Response received on thread main
2 201
2 Date: Wed, 30 May 2012 21:50:39 GMT
2 Content-Length: 18
2 Content-Type: text/html; charset=UTF-8
2 Connection: keep-alive
201 Created
May 30, 2012 11:50:39 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * LoggingFilter - Request received on thread main
3 GET http://192.168.1.43:8080/v1/AUTH_1e9c9a2fe2454950a900202b1e82b54f
3 Accept: application/json
3 X-Auth-Token: cedec9abd4c249479181c2c6d9c0ad85
May 30, 2012 11:50:39 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 4 * LoggingFilter - Response received on thread main
4 200
4 X-Account-Object-Count: 0
4 X-Account-Container-Count: 1
4 Date: Wed, 30 May 2012 21:50:39 GMT
4 Content-Length: 47
4 X-Timestamp: 1338414639.80726
4 X-Account-Bytes-Used: 0
4 Accept-Ranges: bytes
4 Content-Type: application/json; charset=utf-8
4 Connection: keep-alive
[{name:test-container,count:0,bytes:0}]
May 30, 2012 11:50:39 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 5 * LoggingFilter -
Re: [Openstack] Fwd: Question about cloudfiles API
On 5/24/2012 7:25 PM, Luis Gervaso wrote: Hi Shawn, You can try with OpenStack Java SDK https://github.com/woorea/openstack-java-sdk https://github.com/woorea/openstack-java-sdk/wiki/Swift-Tutorial Is there a jar I can download and a list of dependencies? That appears to be a source archive, I'm looking for a binary release. Is this API faster than the rackspace cloudfiles API? I am getting terrible performance numbers with cloudfiles. This is in bytes per second, both the proxy and storage networks are gigabit: totalCount: 10024 Max rate: 16290 Median rate: 2859 99th percentile rate: 9126 95th percentile rate: 6673 75th percentile rate: 4266 A similar program using the GridFS API in MongoDB several weeks ago, running on the same server testbed: totalCount: 9050 Max rate: 121353 Median rate: 42377 99th percentile rate: 102912 95th percentile rate: 91223 75th percentile rate: 67423 Thanks, Shawn ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Fwd: Question about cloudfiles API
-- Forwarded message --
From: Luis Gervaso l...@woorea.es
Date: Fri, May 25, 2012 at 3:24 AM
Subject: Re: [Openstack] Question about cloudfiles API
To: Shawn Heisey launch...@elyograg.org
Hi Shawn,
You can try with OpenStack Java SDK
https://github.com/woorea/openstack-java-sdk
https://github.com/woorea/openstack-java-sdk/wiki/Swift-Tutorial
Regards
On Thu, May 24, 2012 at 10:03 PM, Shawn Heisey launch...@elyograg.orgwrote:
This question is probably more appropriate for the Swift mailing list, but
I could not figure out how to subscribe to that list, so it's going here.
I'm OK with moving it there, if someone can tell me how to get subscribed,
or if I'm in completely the wrong place, let me know.
I am attempting to evaluate Swift for our environment. I have set up a
Swift cluster using the ubuntu multi-server HOWTO and I can use the
commandline utilities to upload and download files. Now I need to do a
test using the Java API. I downloaded java-cloudfiles and I can't seem to
make it work. It fails at the login() step.
FilesClient client = new FilesClient(username, password,
authUrl, null, 1);
if (client.login())
{
javax.net.ssl.**SSLPeerUnverifiedException: peer not authenticated
at com.sun.net.ssl.internal.ssl.**SSLSessionImpl.**
getPeerCertificates(**SSLSessionImpl.java:352)
at org.apache.http.conn.ssl.**AbstractVerifier.verify(**
AbstractVerifier.java:128)
at org.apache.http.conn.ssl.**SSLSocketFactory.**connectSocket(**
SSLSocketFactory.java:397)
at org.apache.http.impl.conn.**DefaultClientConnectionOperato**
r.openConnection(**DefaultClientConnectionOperato**r.java:148)
at org.apache.http.impl.conn.**AbstractPoolEntry.open(**
AbstractPoolEntry.java:150)
at org.apache.http.impl.conn.**AbstractPooledConnAdapter.**open(**
AbstractPooledConnAdapter.**java:121)
at org.apache.http.impl.client.**DefaultRequestDirector.**
tryConnect(**DefaultRequestDirector.java:**575)
at org.apache.http.impl.client.**DefaultRequestDirector.**execute(*
*DefaultRequestDirector.java:**425)
at org.apache.http.impl.client.**AbstractHttpClient.execute(**
AbstractHttpClient.java:820)
at org.apache.http.impl.client.**AbstractHttpClient.execute(**
AbstractHttpClient.java:754)
at org.apache.http.impl.client.**AbstractHttpClient.execute(**
AbstractHttpClient.java:732)
at com.rackspacecloud.client.**cloudfiles.FilesClient.login(**
FilesClient.java:278)
at com.REDACTED.swiftest.Main.**main(Main.java:50)
This all works with the curl command using the same auth URL. I've got
the default user/password set up from the HOWTO.
Initially I suspected that the problem was due to the self-signed
certificate, but watching syslog on the primary proxy server, I don't see
any requests logged, but I do see a conversation happen on port 8080 with
tcpdump. How can I troubleshoot this?
Thanks,
Shawn
__**_
Mailing list:
https://launchpad.net/~**openstackhttps://launchpad.net/%7Eopenstack
Post to : openstack@lists.launchpad.net
Unsubscribe :
https://launchpad.net/~**openstackhttps://launchpad.net/%7Eopenstack
More help :
https://help.launchpad.net/**ListHelphttps://help.launchpad.net/ListHelp
--
---
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO CTO
mobile: (+34) 627983344
luis@ luis.gerv...@gmail.comwoorea.es
--
---
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO CTO
mobile: (+34) 627983344
luis@ luis.gerv...@gmail.comwoorea.es
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
