[issue7080] locale.strxfrm raises MemoryError
egreen egr...@operamail.com added the comment: The tests in TestEnUSCollation I added don't work for all encodings (e.g. Asian ones, or ASCII). Now checked for encodings which are known to work. Found and fixed a bug (result not returned) in getpreferredencoding in Lib/locale.py. This test is skipped on Mac, since r30377. I've tested it on 10.3 (PPC), and it passed, so I enabled it again. I did, however, find that FreeBSD and Mac OS X have known bugs in wcscoll and wcsxfrm: http://www.freebsd.org/cgi/man.cgi?query=wcsxfrm (cf. patch) -- Added file: http://bugs.python.org/file15167/strxfrm_fixes_and_collation_tests_2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
Antoine Pitrou pit...@free.fr added the comment: Nice work, thanks! One detail: when giving you credit, who should I mention? just egreen? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
Derk Drukker egr...@operamail.com added the comment: I've entered my real name. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
Antoine Pitrou pit...@free.fr added the comment: Ok, thanks. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
Antoine Pitrou pit...@free.fr added the comment: The patch is now committed to py3k (r75539) and 3.1 (r75541), closing. -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
egreen egr...@operamail.com added the comment: I've added the tests. I found that on Windows, strxfrm has been unavailable in py3k since r61306, because of an omission in PC/pyconfig.h. (cf. patch) In determining whether I should first test for the existence of strcoll (or strxfrm) in the locale module, as was done in TestMiscellaneous, I found this is no longer needed in py3k since r61339. (cf. patch) (In python2 such checks are still necessary.) -- Added file: http://bugs.python.org/file15158/strxfrm_fixes_and_collation_tests.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
egreen egr...@operamail.com added the comment: Could someone triage this? And review my simple fix? I think it should be a release blocker. It seems to me to be a problem on all 64-bit non-Windows platforms, or any platform if using UCS-4. The 4 high-order bytes of an 8-byte Py_ssize_t variable are not initialized. Thanks. -- components: +Extension Modules -Library (Lib) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
Antoine Pitrou pit...@free.fr added the comment: Nice catch. I wonder why not all of our modules are PY_SSIZE_T_CLEAN in py3k. This function is not actually tested, could you add a test to the regression suite? -- nosy: +pitrou priority: - high ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
egreen egr...@operamail.com added the comment: All right. The function strcoll also isn't tested (except for issue #3303). I'll look into that one, too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7080] locale.strxfrm raises MemoryError
New submission from egreen egr...@operamail.com: The strxfrm function in the locale module can potentially raise a MemoryError. The failing malloc is in Modules/_localemodule.c, line 291. This is because the variable n0 of type Py_ssize_t is passed to PyArg_ParseTuple, which expects an int when PY_SSIZE_T_CLEAN is not defined. Patch attached, which also fixes an unrelated memory leak. -- components: Library (Lib) files: strxfrm_MemoryError.patch keywords: patch messages: 93728 nosy: egreen, loewis severity: normal status: open title: locale.strxfrm raises MemoryError type: crash versions: Python 3.1, Python 3.2 Added file: http://bugs.python.org/file15078/strxfrm_MemoryError.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7080 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
