[
https://issues.apache.org/jira/browse/JCLOUDS-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13766554#comment-13766554
]
Erik Paulsson edited comment on JCLOUDS-278 at 9/13/13 3:38 PM:
----------------------------------------------------------------
Andrew, I did some more testing after your comment.
Yes, the response is in JSON format and needs to be parsed by JClouds to pull
out the proper fields, BUT I don't believe any *URL encoding OR decoding*
should ever be done.
The below JSON returned from Swift is from a container listing holding objects
with the following names:
AND_-_100%_TVX_EXT/test_update.txt
NassauGrouper.jpg
NiceDolphin.jpg
NiceLobster.jpg
SnooksKing.jpg
blankπ × { } © 佈 б ' ' —.txt
JSON:
{code}
[
{
'bytes': 86,
'last_modified': '2013-09-13T14:11:27.655280',
'hash': '74aa9ef5568a88a927de0e056e3db160',
'name': 'AND_-_100%_TVX_EXT/test_update.txt',
'content_type': 'application/unknown'
},
{
'bytes': 245725,
'last_modified': '2013-09-11T18:37:05.306840',
'hash': '56a8b9fb7d100e8a3b6a7394d882b3fb',
'name': 'NassauGrouper.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 308532,
'last_modified': '2013-09-11T18:37:19.984820',
'hash': '466f114d85a15b1b2ced3fa3dc2f99fd',
'name': 'NiceDolphin.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 226173,
'last_modified': '2013-09-11T18:37:21.744060',
'hash': '8610e201d9f625f7541396b03d989a00',
'name': 'NiceLobster.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 333704,
'last_modified': '2013-09-11T18:37:22.581460',
'hash': '7dd34b31503fb350acd011300505949e',
'name': 'SnooksKing.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 62,
'last_modified': '2013-09-11T18:37:05.283650',
'hash': '78a4ec0f9aa3eb9c2cb858a160b50bbd',
'name': u"blank\u03c0 \xd7 { } \xa9 \u4f48 \u0431 ' ' \u2014.txt",
'content_type': 'application/unknown'
}
]
{code}
Notice that no part of the JSON response is URL encoded. However, the last
file name is UTF-8 encoded which is totally fine and requires no special
handling by JClouds.
"blankπ × { } © 佈 б ' ' —.txt" is returned as:
u"blank\u03c0 \xd7 { } \xa9 \u4f48 \u0431 ' ' \u2014.txt"
Why would URIBuilder ever be used when parsing a JSON response body?
Andrew, if I understand your suggestion correctly you are saying to add a call
to Strings2.urlEncode just so that the call to Strings2.urlDecode doesn't blow
up when a '%' symbol is encountered. If this is the case then I believe the
best solution is to remove all URL encoded handling of Strings when handling
JSON response data.
Not trying to be argumentative... I am the newbie here. Just trying to
understand why JClouds is doing any URI/URL handling of JSON response body data.
Please take a second look at this.
was (Author: paulsson):
Andrew, I did some more testing after your comment.
Yes, the response is in JSON format and needs to be parsed by JClouds to pull
out the proper fields, BUT I don't believe any *strong*URL encoding OR
decoding*strong* should ever be done.
The below JSON returned from Swift is from a container listing holding objects
with the following names:
AND_-_100%_TVX_EXT/test_update.txt
NassauGrouper.jpg
NiceDolphin.jpg
NiceLobster.jpg
SnooksKing.jpg
blankπ × { } © 佈 б ' ' —.txt
JSON:
{code}
[
{
'bytes': 86,
'last_modified': '2013-09-13T14:11:27.655280',
'hash': '74aa9ef5568a88a927de0e056e3db160',
'name': 'AND_-_100%_TVX_EXT/test_update.txt',
'content_type': 'application/unknown'
},
{
'bytes': 245725,
'last_modified': '2013-09-11T18:37:05.306840',
'hash': '56a8b9fb7d100e8a3b6a7394d882b3fb',
'name': 'NassauGrouper.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 308532,
'last_modified': '2013-09-11T18:37:19.984820',
'hash': '466f114d85a15b1b2ced3fa3dc2f99fd',
'name': 'NiceDolphin.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 226173,
'last_modified': '2013-09-11T18:37:21.744060',
'hash': '8610e201d9f625f7541396b03d989a00',
'name': 'NiceLobster.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 333704,
'last_modified': '2013-09-11T18:37:22.581460',
'hash': '7dd34b31503fb350acd011300505949e',
'name': 'SnooksKing.jpg',
'content_type': 'application/unknown'
},
{
'bytes': 62,
'last_modified': '2013-09-11T18:37:05.283650',
'hash': '78a4ec0f9aa3eb9c2cb858a160b50bbd',
'name': u"blank\u03c0 \xd7 { } \xa9 \u4f48 \u0431 ' ' \u2014.txt",
'content_type': 'application/unknown'
}
]
{code}
Notice that no part of the JSON response is URL encoded. However, the last
file name is UTF-8 encoded which is totally fine and requires no special
handling by JClouds.
"blankπ × { } © 佈 б ' ' —.txt" is returned as:
u"blank\u03c0 \xd7 { } \xa9 \u4f48 \u0431 ' ' \u2014.txt"
Why would URIBuilder ever be used when parsing a JSON response body?
Andrew, if I understand your suggestion correctly you are saying to add a call
to Strings2.urlEncode just so that the call to Strings2.urlDecode doesn't blow
up when a '%' symbol is encountered. If this is the case then I believe the
best solution is to remove all URL encoded handling of Strings when handling
JSON response data.
Not trying to be argumentative... I am the newbie here. Just trying to
understand why JClouds is doing any URI/URL handling of JSON response body data.
Please take a second look at this.
> SwiftClient parse exception on '%' symbols
> ------------------------------------------
>
> Key: JCLOUDS-278
> URL: https://issues.apache.org/jira/browse/JCLOUDS-278
> Project: jclouds
> Issue Type: Bug
> Affects Versions: 1.5.10, 1.6.2
> Environment: Tested on:
> org.jclouds:jclouds-allblobstore:1.5.5
> AND
> org.apache.jclouds:jclouds-allblobstore:1.6.2-incubating
> Reporter: Erik Paulsson
>
> I've tested this using org.jclouds:jclouds-allblobstore v1.5.5 and
> 1.6.2-incubating. I get a parse exception using the SwiftClient when object
> names contain a '%' symbol.
> For example an object whose name is 'AND_-_100%_TVX/test.txt' will get stored
> fine, but when I attempt to list the contents of the container that the
> object is in I get the following exception:
> Sep 10, 2013 10:26:40 AM org.jclouds.logging.jdk.JDKLogger logError
> SEVERE: Error parsing input
> java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in
> escape (%) pattern - For input string: "_T"
> at java.net.URLDecoder.decode(URLDecoder.java:192)
> at org.jclouds.util.Strings2.urlDecode(Strings2.java:97)
> at org.jclouds.http.Uris$UriBuilder.appendPath(Uris.java:157)
> at
> org.jclouds.openstack.swift.functions.ParseObjectInfoListFromJsonResponse$3.apply(ParseObjectInfoListFromJsonResponse.java:82)
> at
> org.jclouds.openstack.swift.functions.ParseObjectInfoListFromJsonResponse$3.apply(ParseObjectInfoListFromJsonResponse.java:80)
> at com.google.common.collect.Iterators$9.transform(Iterators.java:893)
> at
> com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
> at com.google.common.collect.Sets.newTreeSet(Sets.java:345)
> at
> org.jclouds.openstack.swift.functions.ParseObjectInfoListFromJsonResponse.apply(ParseObjectInfoListFromJsonResponse.java:79)
> at
> org.jclouds.openstack.swift.functions.ParseObjectInfoListFromJsonResponse.apply(ParseObjectInfoListFromJsonResponse.java:54)
> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:66)
> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:46)
> at com.google.common.util.concurrent.Futures$3.apply(Futures.java:376)
> at
> com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:518)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> This may affect more than just the SwiftClient API since it looks like the
> bug may be in:
> org.jclouds.util.Strings2.urlDecode
> org.jclouds.http.Uris$UriBuilder.appendPath
> Here is an example of a simple method that generates the exception:
> private PageSet<ObjectInfo> listObjects(String containerName,
> String prefix,
> int maxResults,
> String marker) {
> ListContainerOptions containerOptions =
> ListContainerOptions.Builder.maxResults(maxResults);
> if(marker != null) containerOptions.afterMarker(marker);
> if(prefix != null) containerOptions.withPrefix(prefix);
> return swiftClient.listObjects(containerName, containerOptions);
> }
> Calling this method with the container name of the container containing the
> object whose name contains a '%' symbol and a maxResults of 1000 and null for
> both prefix and marker will result in the above exception.
> Let's just say my container name is "percent-container" and I have an object
> in this container with the name "AND_-_100%_TVX/test.txt"
> So this is what the method call to the above method might look like:
> listObjects("percent-container", null, 1000, null);
> Even if I URL encode the object name before storing it I still get the same
> exception when retrieving a list of contents for the space.
> I can run a cURL command straight to the Swift storage provider to get the
> object and it returns fine.
> Am I doing anything wrong? Do I need to handle '%' symbols in a special way?
> The docs swift docs don't express any character restrictions on object
> names:
> http://docs.openstack.org/trunk/openstack-object-storage/admin/content/containers-and-objects.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
