[issue3098] sys.sizeof test fails with wide unicode

Robert Schuppenies <[EMAIL PROTECTED]> added the comment: I followed Marc's advise and checked-in a corrected test. Besides, I opened a new issue to address the fallback solution for UCS4 and UCS2 (see issue3130). -- resolution: -> fixed status: open -> closed

[issue3098] sys.sizeof test fails with wide unicode

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: On 2008-06-13 21:54, Marc-Andre Lemburg wrote: > BTW: Here's another trick you can use: > > print 'sizeof(Py_UNICODE) =', len(u'\0'.encode('unicode-internal')) > > (for Py2.x) ... and for Py3.x: print(len(u'\0'.encode('unicode-internal

[issue3098] sys.sizeof test fails with wide unicode

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: On 2008-06-13 22:32, Antoine Pitrou wrote: > Antoine Pitrou <[EMAIL PROTECTED]> added the comment: > > Le vendredi 13 juin 2008 à 20:18 +, Marc-Andre Lemburg a écrit : >> AFAIK, only Crays have this problem, but apart from that: I'd c

[issue3098] sys.sizeof test fails with wide unicode

Georg Brandl <[EMAIL PROTECTED]> added the comment: Looks good to me. ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mailing lis

[issue3098] sys.sizeof test fails with wide unicode

Robert Schuppenies <[EMAIL PROTECTED]> added the comment: Correct is good, so here is a patch which exposes the size of Py_UNICODE via _testcapi. Added file: http://bugs.python.org/file10635/Py_UNICODE_SIZEOF.patch ___ Python tracker <[EMAIL PROTECTED]>

[issue3098] sys.sizeof test fails with wide unicode

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Le dimanche 15 juin 2008 à 13:18 +, Robert Schuppenies a écrit : > If I understand configure correctly, PY_UNICODE_TYPE is only set when > a type matching the size of $unicode_size is found. And this is set to > either 2 or 4. Buf if PY_U

[issue3098] sys.sizeof test fails with wide unicode

Robert Schuppenies <[EMAIL PROTECTED]> added the comment: If I understand configure correctly, PY_UNICODE_TYPE is only set when a type matching the size of $unicode_size is found. And this is set to either 2 or 4. Thus, sizeof(Py_UNICODE) should always return 2 or 4. If you agree, I would suggest

[issue3098] sys.sizeof test fails with wide unicode

Robert Schuppenies <[EMAIL PROTECTED]> added the comment: Found it, thanks. Wrong use of grep :| ___ Python tracker <[EMAIL PROTECTED]> ___ ___ P

[issue3098] sys.sizeof test fails with wide unicode

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: I believe Py_UNICODE_TYPE is set be configure in pyconfig.h. ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3098] sys.sizeof test fails with wide unicode

Robert Schuppenies <[EMAIL PROTECTED]> added the comment: I think you're right that sizeof(Py_UNICODE) is the correct value to use. But could you please explain to me how PY_UNICODE_TYPE is set, I cannot find it. Also, len(u'\0'.encode('unicode-internal')) does not work for Py3.0. Any suggestion

[issue3098] sys.sizeof test fails with wide unicode

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Le vendredi 13 juin 2008 à 20:18 +, Marc-Andre Lemburg a écrit : > AFAIK, only Crays have this problem, but apart from that: I'd consider > it a bug if sizeof(Py_UCS4) != 4. Perhaps a #error can be added to that effect? Something like (un

[issue3098] sys.sizeof test fails with wide unicode

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: On 2008-06-13 21:56, Antoine Pitrou wrote: > Antoine Pitrou <[EMAIL PROTECTED]> added the comment: > > Hmm, so it seems that in some UCS4 builds, sizeof(Py_UNICODE) could end > up being more than 4 if the native int type is itself larger

[issue3098] sys.sizeof test fails with wide unicode

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Hmm, so it seems that in some UCS4 builds, sizeof(Py_UNICODE) could end up being more than 4 if the native int type is itself larger than 32 bits; although the latter is probably quite rare (64-bit platforms are usually either LP64 or LLP64).

[issue3098] sys.sizeof test fails with wide unicode

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: BTW: Here's another trick you can use: print 'sizeof(Py_UNICODE) =', len(u'\0'.encode('unicode-internal')) (for Py2.x) ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3098] sys.sizeof test fails with wide unicode

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: It's actually very easy: Py_UNICODE is a 2-byte value for UCS-2 builds and 4 byte value for UCS-4 builds of Python. print ((sys.maxunicode < 66000) and 'UCS2' or 'UCS4') tells you which one you have. Note that you should *not* use the

[issue3098] sys.sizeof test fails with wide unicode

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: Personally, I prefer the one with _testcapi.Py_UNICODE_SIZE because it is safe against future changes, but wait for someone else's opinion. ___ Python tracker <[EMAIL PROTECTED]>

[issue3098] sys.sizeof test fails with wide unicode

Robert Schuppenies <[EMAIL PROTECTED]> added the comment: sys.maxunicode is well defined to be either 0x for UCS-2 or 0x10 for UCS-4 (see PyUnicode_GetMax). Py_UNICODE_SIZE is set in pyconfig.h to be either 2 or 4 during configuration. When >= 4, Py_UNICODE_WIDE is set which again influe

[issue3098] sys.sizeof test fails with wide unicode

Changes by Robert Schuppenies <[EMAIL PROTECTED]>: -- keywords: +patch Added file: http://bugs.python.org/file10623/maxunicode.patch ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3098] sys.sizeof test fails with wide unicode

Georg Brandl <[EMAIL PROTECTED]> added the comment: It is true that sys.maxunicode reflects whether the build is using UCS-2 or UCS-4; however, the size of Py_UNICODE is not fixed by that, look at unicodeobject.h. (Though I don't think we have platforms that actually *do* use sizes other than 2

[issue3098] sys.sizeof test fails with wide unicode

Antoine Pitrou <[EMAIL PROTECTED]> added the comment: Do you really need to expose Py_UNICODE_SIZE? There is already sys.maxunicode, unless I'm missing something. -- nosy: +pitrou ___ Python tracker <[EMAIL PROTECTED]>

[issue3098] sys.sizeof test fails with wide unicode

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: I'm sure there wasn't any a few months ago. -- nosy: +amaury.forgeotdarc ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3098] sys.sizeof test fails with wide unicode

Robert Schuppenies <[EMAIL PROTECTED]> added the comment: Are they any buildbots running with the "--enable-unicode=ucs4" option? Just curious. ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3098] sys.sizeof test fails with wide unicode

Benjamin Peterson <[EMAIL PROTECTED]> added the comment: It was recommended by Georg that you expose Py_UNICODE_SIZE in the _testcapi, since the size is not consistent across all platforms. ___ Python tracker <[EMAIL PROTECTED]>

[issue3098] sys.sizeof test fails with wide unicode

New submission from Benjamin Peterson <[EMAIL PROTECTED]>: test test_sys failed -- Traceback (most recent call last): File "/temp/python/trunk/Lib/test/test_sys.py", line 549, in test_specialtypes size2=basicsize + sys.getsizeof(str(s))) File "/temp/python/trunk/Lib/test/test_sys.py", lin