Hi Sengor:
Thanks for the replies.
I manually patched vcloud.py by appending ".decide('utf-8')" in two
the places needing it, and that worked (... and makes sense ... I had
to the same decode() step in my non-libcloud httplib2/ReST case for
python-3).
So let me move onto the subsequent parsing exceptions I mentioned...
which is raised during processing of the HTTP XML Response
(indicating malformed). Tracking it down, this happens in
vcloud.py, here:
=================
class VCloudConnection(ConnectionUserAndKey):
<snip>
def _get_auth_token(self):
...
<snip>
body = ET.XML(resp.read()) <---
<snip>
...
<snip>
=================
Through various print() and type() statements, I see that "resp.read()", which
when expanded translates to "http.client.HTTPResponse.read()", returns a
bytes string. However, I'm wondering if this may also need to be similarly
".decode('utf-8')", like this:
body = ET.XML(resp.read().decode('utf-8'))
since the Python-3 documentation for "xml.etree.ElementTree.XML()"
(which is what ET.XML() expands to), indicates that it is expecting a string
(not a bytes encoded string).
So I tried the modification above. Here is the before and after:
resp.read() [ BEFORE .decode() inserted ]
<class 'bytes'> <--- types is 'bytes', and the following is the actual bytes
string...
b'b\'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\\n <Org
href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
type="application/vnd.vmware.vcloud.org+xml"
name="[email protected]"/>\\n</OrgL'
resp.read().decode('utf-8') [ AFTER .decode() inserted ]
<class 'str'> <--- types is 'str', and the following is the actual string...
b'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\n <Org
href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
type="application/vnd.vmware.vcloud.org+xml" name="[email protected]"/>\n</OrgL
Looking carefully at both the bytes string (un-decoded) and the string
(decoded) above -- forgetting about whether we
decode() it or not for the moment -- both sequences appear to be malformed...
They both have an extra "b'"
pre-pending it, and both are truncated at the end (i.e. the enclosing "</OrgL"
... doesn't end correctly). Note: Just
in case this was a print() artefact, I also wrote the output to a file (same
truncation issue).
This above issue is only true for python-3. Python 2, (decode()ed or not) emits
output correctly as:
<OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns:xsd="http://www.w3.org/2001/XMLSchema";> <Org
href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
type="application/vnd.vmware.vcloud.org+xml"
name="[email protected]"/></OrgList>
So there appears to be two problems here:
(1) missing ".decode('utf-8')" method call.
(2) the output of "resp.read()", with or without the ".decode('utf-8')"
part, appears to be malformed.
To make matters more interesting, note that the python-3 case curl(1) output
shows a result that is not truncated,
but still, however, has the pre-pending "b'" (so it's still malformed). And
here is that curl(1) output
(remembering that the above were from print() debug statements of mine, not
from curl(1)):
b'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\n <Org
href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
type="application/vnd.vmware.vcloud.org+xml"
name="[email protected]"/>\n</OrgList>'
As I look more for the cause, any helpful feedback is appreciated.
Again,... the libcloud API is v0.10.1 with Python v3.2.3
Thank you,
Noel
>________________________________
> From: Sengor <[email protected]>
>To: [email protected]; Noel Milton Vega
><[email protected]>
>Sent: Thursday, June 21, 2012 10:21 PM
>Subject: Re: Terremark authentication example using libcloud ...
>
>Hi Noel,
>
>Replies in line below:
>
>On 22 June 2012 09:30, Noel Milton Vega <[email protected]> wrote:
>> (1) I was surprised that that base64 line worked at all (even in Python 2)
>> since I always thought use of "b" only worked for string literals
>> (not generated strings).
>
>Patch has been applied to trunk which now works for both Python 2 & 3.
>For more info see: https://issues.apache.org/jira/browse/LIBCLOUD-204
>
>
>> (2) Not an issue for this specific thread, but even when I hard-coded
>> my auth string (just to get past the Python-3 related auth issue), sadly
>> another unrelated (xml parsing) issue arose almost immediately.
>> Just as a heads up, it was this:
>>
>>
>> Traceback (most recent call last):
>> File "/usr/lib64/python3.2/xml/etree/ElementTree.py", line 1668, in feed
>> self._parser.Parse(data, 0)
>> xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1,
>> column 1
>>
>> (So I never got my list_images() output.)
>>
>> (3) Similar to the last point, python-2 gets much further along, but also
>> eventually experiences an parse-related exception too (as it's
>> iterating through the list the of images too I believe). Again, as a
>> heads-up fyi:
>>
>>
>> File "/usr/lib/python2.7/site-packages/libcloud/common/base.py", line 76,
>> in __init__
>> raise Exception(self.parse_error())
>> Exception: <Element '{http://www.w3.org/1999/xhtml}html'; at 0x118dd10>
>
>Hard to say what the issue could be for these two. Could you post the
>debugging output for these two (don't forget to remove the private
>contents such as base64 encoded auth):
>http://libcloud.apache.org/docs/debugging.html
>
>Note I've only used the vCloud module under Python 2.x at the moment
>and don't have access to Terremark. We did slightly change the VDC
>object model when vCloud v1.5 support has been introduced. If this is
>related to what you see it'd be good to fix it up, otherwise it's
>something else getting in the way.
>
>--
>sengork
>
>
>
