Bug#750120: python-apt: Some locales use no-breaking space for LC_NUMBER, so return unicode for SizeToStr
On Sun, Jun 01, 2014 at 09:18:45PM +0200, Odin Hørthe Omdal wrote: Package: python-apt Version: 0.9.3.5 Severity: normal Tags: upstream l10n patch Thanks for your bugreport and your patch! this is on Ubuntu, but this was the closest I could find to the python-apt upstream :) This is breaking the distribution upgrade on my girlfriends computer (since she is using nn_NO.UTF8 as her language). Ubuntu's upgrading python script is not expecting that the str returned will have undecodeable chars. This could also be fixed at Ubuntu's level, but looking at it, I thought it hit closer to home for apt_pkg.size_to_str() to return a ready decoded unicode string to all its users. [..] Indeed, thanks for this fix! Unfortunately as it is right now it breaks the API of size_to_str(), i.e. there are probably apps who expect a type str. So we either need to add a size_to_unicode() or a kwarg like as_unicode or add a warning in the documentation for the py2 users. Cheers, Michael -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#750120: python-apt: Some locales use no-breaking space for LC_NUMBER, so return unicode for SizeToStr
On Fri, Jun 06, 2014 at 09:03:11AM +0200, Michael Vogt wrote: On Sun, Jun 01, 2014 at 09:18:45PM +0200, Odin Hørthe Omdal wrote: Package: python-apt Version: 0.9.3.5 Severity: normal Tags: upstream l10n patch Thanks for your bugreport and your patch! this is on Ubuntu, but this was the closest I could find to the python-apt upstream :) This is breaking the distribution upgrade on my girlfriends computer (since she is using nn_NO.UTF8 as her language). Ubuntu's upgrading python script is not expecting that the str returned will have undecodeable chars. This could also be fixed at Ubuntu's level, but looking at it, I thought it hit closer to home for apt_pkg.size_to_str() to return a ready decoded unicode string to all its users. [..] Indeed, thanks for this fix! Unfortunately as it is right now it breaks the API of size_to_str(), i.e. there are probably apps who expect a type str. So we either need to add a size_to_unicode() or a kwarg like as_unicode or add a warning in the documentation for the py2 users. We should just leave it as it is. Python 3 is unicode. Python 2 has been str all the time, there's no point changing this and breaking things or introducing new APIs that are (a) pointless (b) for a deprecated language. I do not believe that returning str causes a problem itself. There may be some applications that have problems with it. Printing it to a console or a file or displaying it in a window works just fine because there *cannot* be any decoding/encoding problems, since all is bytes. Unless of course, the applications tries to print to an encoded stream, but this means the app uses io.open() which is a minority that should deal itself with the problems this causes. I believe it's more likely that the distribution upgrade script is using an io.open() file instead and has a problem encoding things here. This can be fixed easily be changing it to use an unencoded bytes stream. -- Julian Andres Klode - Debian Developer, Ubuntu Member See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/. Please do not top-post if possible. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#750120: python-apt: Some locales use no-breaking space for LC_NUMBER, so return unicode for SizeToStr
Package: python-apt Version: 0.9.3.5 Severity: normal Tags: upstream l10n patch Dear Maintainer, this is on Ubuntu, but this was the closest I could find to the python-apt upstream :) This is breaking the distribution upgrade on my girlfriends computer (since she is using nn_NO.UTF8 as her language). Ubuntu's upgrading python script is not expecting that the str returned will have undecodeable chars. This could also be fixed at Ubuntu's level, but looking at it, I thought it hit closer to home for apt_pkg.size_to_str() to return a ready decoded unicode string to all its users. -- System Information: Debian Release: jessie/sid APT prefers trusty-updates APT policy: (500, 'trusty-updates'), (500, 'trusty-security'), (500, 'trusty'), (100, 'trusty-backports') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.13.0-27-generic (SMP w/4 CPU cores) Locale: LANG=nn_NO.UTF-8, LC_CTYPE=nn_NO.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages python-apt depends on: ii libapt-inst1.5 1.0.1ubuntu2 ii libapt-pkg4.12 1.0.1ubuntu2 ii libc6 2.19-0ubuntu6 ii libgcc11:4.9-20140406-0ubuntu1 ii libstdc++6 4.8.2-19ubuntu1 ii python 2.7.5-5ubuntu3 ii python-apt-common 0.9.3.5 pn python:any none Versions of packages python-apt recommends: ii iso-codes3.52-1 ii lsb-release 4.1+Debian11ubuntu6 ii xz-utils 5.1.1alpha+20120614-2ubuntu2 Versions of packages python-apt suggests: pn python-apt-dbg none pn python-apt-doc none ii python-gtk2 2.24.0-3ubuntu3 pn python-vte none -- no debconf information From 1f482cbb9ebb2338cbe288613fc1554682a58637 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Odin=20H=C3=B8rthe=20Omdal?= odi...@opera.com Date: Sun, 1 Jun 2014 21:05:23 +0200 Subject: [PATCH] Some locales use no-breaking space for text, so return unicode --- python/string.cc | 3 ++- tests/test_size_to_str.py | 10 +- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/python/string.cc b/python/string.cc index 9b02c93..006f102 100644 --- a/python/string.cc +++ b/python/string.cc @@ -86,7 +86,8 @@ PyObject *StrSizeToStr(PyObject *Self,PyObject *Args) // Check for OverflowErrors or other exceptions during conversion. if (PyErr_Occurred()) return 0; - return CppPyString(SizeToStr(value)); + std::string Str = SizeToStr(value) + return PyUnicode_FromStringAndSize(Str.c_str(),Str.length) } PyObject *StrQuoteString(PyObject *Self,PyObject *Args) diff --git a/tests/test_size_to_str.py b/tests/test_size_to_str.py index ba1e45b..c7b844d 100644 --- a/tests/test_size_to_str.py +++ b/tests/test_size_to_str.py @@ -91,13 +91,21 @@ class SizeToStrTestCase(unittest.TestCase): class RegressionTestCase(unittest.TestCase): -Regression test for LP: #1030278 +Regression tests def test_no_overflow_error(self): # LP: #1030278 produces an overflow error in size_to_str() with a big # value under Python 3. self.assertEqual(apt_pkg.size_to_str(2147483648), '2147 E') +def test_expect_unicode(self): +import locale +old_locale = locale.getlocale() +locale.setlocale(locale.LC_ALL, 'nn_NO.utf8') +try: +self.assertEqual(apt_pkg.size_to_str(100), u'1\xa k') +finally: +locale.setlocale(locale.LC_ALL, old_locale) if __name__ == __main__: unittest.main() -- 1.9.1
