** Changed in: python-glanceclient
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1180377
Title:
provisioning failed if the image have a non-ascii property and using
qpid
Status in OpenStack Compute (Nova):
Fix Released
Status in Python client library for Glance:
Fix Released
Bug description:
If user set a non-ascii character to the key of an image property and
using qpid backend, the instance provisioning will be failed.
Use case:
1. Set a non-ascii character to the key of an image property:
"glance image-update --property '中文'='中文'
5a7e6de5-15b9-49d2-977e-eda0a221c0b9"
2. Launch a instance with that image:
"nova boot --image cirros-0.3.0-x86_64 --flavor m1.tiny --nic
net-id=881cbe80-3876-4f4c-b8c8-7b23cedc14f4 lzy-1"
3. Then the provisioning failed and nova-api service catch the exception and
write to log:
File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line
1052, in process
self.send(snd, msg)
File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line
1263, in send
body = enc(msg.content)
File "/usr/lib/python2.6/site-packages/qpid/messaging/message.py", line 28,
in encode
sc.write_primitive(type, x)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in
write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in
write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(),
m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in
_write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in
write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in
write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(),
m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in
_write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in
write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in
write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(),
m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in
_write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in
write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in
write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(),
m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in
_write_map_elem
sc.write_primitive(type, v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in
write_primitive
getattr(self, "write_%s" % type.NAME)(v)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in
write_map
sc.write(string.joinfields(map(self._write_map_elem, m.keys(),
m.values()), ""))
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 248, in
_write_map_elem
sc.write_str8(k)
File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 185, in
write_str8
self.write_vbin8(s.encode("utf8"))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0:
ordinal not in range(128)
Analysis:
In nova-api service, the provisioning request will be handled in
"nova/compute/api.py' file "_validate_and_provision_instance" method
("create_instance" method called), the image and its properties will be
primitive and make a request spec and pass to nova-scheduler by rpc calls via
message queue. And nova-api call glance api v1.0 ("get" method) to get image's
infromation, under current glanceclient implementation, those non-ascii
character in the image property part will be encoded by utf-8, so the root
cause is that qpid (with qpid_python-0.18 lib) can not handle those utf-8
string.
Solution:
There are two choices to resolve this issue:
1. Resolve in glanceclient. glance-api server return utf-8 property
(key/value pair) to glanceclient, so we need convert the them back to unicode
from utf-8 ("_image_meta_from_headers" method in "image.py" file), this will
cause client.get() method return unicode property string (key/value pair) and
nova-api will get those uncode strings and pass them to nova-scheduler via rpc.
this unicode encoding logic just like image property update logic
("_image_meta_to_headers" method), update method always pass unicode image
property to glance-api server but not utf-8, so this change will keep the
encoding consistency.
And this fix also need change 2 points in nova-api code
("nova/compute/api.py"), they are call str() on the image properties, it should
be unicode() after above change.
2. Resolve in nova-api. This way probably is not good then above one since
the area affected is greater the it. in this way, we need convert utf-8 string
to unicode then to put it into queue, so the change/fix in "to_primitive"
method of "jsonutils.py" file. for now, there are two problems:
a. it not take care key within a dict object, that caused utf-8 encoded key
string (in this case, it's a key of an image property) will not be primitive.
b. need ask to_primitive take care utf-8 string, IMO, need convert it to
unicode but not pass the utf-8 (basestring object) directly. So this fix will
affect more area then above glanceclient fixing way, this fix has commonality,
even user use rabbitmq backend, any utf-8 encoded key within a dict will be
convert include glance properties and any other dict objects which want to
primitive and put into queue.
IMO, I prefer use #1 way to address this issue.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1180377/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
