[issue45372] Unwarranted "certificate has expired" when urlopen-ing R3 sites
Alexandre added the comment: If anyone is having this issue too, you can find more informations at https://community.letsencrypt.org/t/potential-problem-with-r3-intermediates-on-windows-servers/157164, it looks like Windows keeps using expired certificates until rebooted. A solution is to remove the expired R3 intermediary from the store, forcing Window to use the newer. -- nosy: +u36959 ___ Python tracker <https://bugs.python.org/issue45372> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42707] Python uses ANSI CP for stdio on Windows console instead of using console or OEM CP
New submission from Alexandre : Hello, first of all, I hope this was not already discussed (I searched the bugs but it might have been discussed elsewhere) and it's really a bug. I've been struggling to understand today why a simple file redirection couldn't work properly today (encoding issues) and I think I finally understand the whole thing. There's an IO codepage set on Windows consoles (`chcp` for cmd, `[Console]::InputEncoding; [Console]::OutputEncoding` for PowerShell ; chcp will not work on Powershell while it displays it set the CP), 850 for my locale. When there's no redirection / piping, PyWindowsConsoleIO take cares of the encoding (utf-8 is seems), but when there's redirection or piping, encoding falls back to ANSI CP (from config_get_locale_encoding). This behavior seems to be incorrect / breaking things, an example: * testcp.py (file encoded as utf-8) ``` #!/usr/bin/env python3 # -*- coding: utf-8 print('é') ``` * using cmd: ``` # Test condition L:\Cop>chcp Page de codes active : 850 # We're fine here L:\Cop>py testcp.py é L:\Cop>py -c "import sys; print(sys.stdout.encoding)" utf-8 # Now with piping L:\Cop>py -c "import sys; print(sys.stdout.encoding)" | more cp1252 L:\Cop>py testcp.py | more Ú L:\Cop>py testcp.py > lol && type lol Ú # If we adjust cmd CP, it's fine too: L:\Cop>chcp 1252 Page de codes active : 1252 L:\Cop>py testcp.py | more é ``` * with pwsh: ``` PS L:\Cop> ([Console]::InputEncoding, [Console]::OutputEncoding) | select CodePage CodePage 850 850 # Fine without redirection PS L:\Cop> py .\testcp.py é # Here, write-host expect cp850 PS L:\Cop> py .\testcp.py | write-output Ú # Same with Out-file (used by ">") PS L:\Cop> py .\testcp.py > lol; Get-Content lol Ú # PS L:\Cop> py .\testcp.py | more ├Ü ``` By reading some sources today to solve my issue, I found many solutions: * in PS `[Console]::OutputEncoding = [Text.Utf8Encoding]::new($false); $env:PYTHONIOENCODING="utf8"` or `[Console]::OutputEncoding = [Text.Encoding]::GetEncoding(1252)` * in CMD `chcp 65001 && set PYTHONIOENCODING=utf8` (but this seems to break more) or `chcp 1252` But reading (and trusting) https://serverfault.com/questions/80635/how-can-i-manually-determine-the-codepage-and-locale-of-the-current-os (https://docs.microsoft.com/en-us/windows/win32/intl/locale-idefault-constants), I understand Python should be using reading the current CP (from GetConsoleOutputCP, like https://github.com/python/cpython/blob/3.9/Python/fileutils.c:) or using the default OEM CP, and not assuming ANSI CP for stdio : > * the OEM code page for use by legacy console applications, > * the ANSI code page for use by legacy GUI applications. The init path I could trace : > https://github.com/python/cpython/blob/3.9/Python/pylifecycle.c > init_sys_streams >> create_stdio >> (https://github.com/python/cpython/blob/3.9/Python/pylifecycle.c#L1774) >>> open.raw : >>> https://github.com/python/cpython/blob/3.9/Modules/_io/_iomodule.c#L374 >>>> https://github.com/python/cpython/blob/3.9/Modules/_io/winconsoleio.c >> fallback to ini_sys_stream encoding > https://github.com/python/cpython/blob/3.9/Python/initconfig.c > config_init_stdio_encoding > config_get_locale_encoding > GetACP() Some test with GetConsoleCP: ``` L:\Cop>py -c "import os; print(os.device_encoding(0), os.device_encoding(1))" | more cp850 None L:\Cop>type nul | py -c "import os; print(os.device_encoding(0), os.device_encoding(1))" None cp850 L:\Cop>type nul | py -c "import ctypes; print(ctypes.windll.kernel32.GetConsoleCP(), ctypes.windll.kernel32.GetConsoleOutputCP())" 850 850 L:\Cop>py -c "import ctypes; print(ctypes.windll.kernel32.GetConsoleCP(), ctypes.windll.kernel32.GetConsoleOutputCP())" | more 850 850 ``` Some links / documentations, if useful: * https://serverfault.com/questions/80635/how-can-i-manually-determine-the-codepage-and-locale-of-the-current-os * https://docs.microsoft.com/en-us/windows/win32/intl/locale-idefault-constants * https://docs.microsoft.com/en-us/windows/win32/api/winnls/nf-winnls-getoemcp * https://docs.microsoft.com/en-us/windows/win32/api/winnls/nf-winnls-getacp * https://docs.microsoft.com/en-us/windows/console/getconsoleoutputcp * https://stackoverflow.com/questions/56944301/why-does-powershell-redirection-change-the-formatting-of-the-text-content * https://stackoverflow.com/questions/19122755/output-echo-a-variable-to-a-text-file * https://stackoverflow.com/questions/40098771/changing-powershells-default-output-encoding-to-utf-8 * Maybe related: https://github.com/PowerShell/PowerShell/issues/10907 * https://stackoverflow.com/questions/57131654/using-utf-8-encoding-chcp-65001-in-comma
[issue27635] pickle documentation says that unpickling may not call __new__
Change by Alexandre Vassalotti : -- stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue27635> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31852] Crashes with lines of the form "async \"
New submission from Alexandre Hamelin : Hi. Python 3.6.2 crashes when interpreting lines with the text "async \" (future keyword 'async' and ending with a backslash). Tested in a docker environment (debian jessie). (see github.com/0xquad/docker-python36 if needed) Examples: $ docker run -ti --rm python36 root@4c09392f83c8:/# python3.6 Python 3.6.2 (default, Aug 4 2017, 14:35:04) [GCC 6.4.0 20170724] on linux Type "help", "copyright", "credits" or "license" for more information. >>> async \ ... File "", line 1 \ufffd\ufffdF\ufffd\ufffd ^ SyntaxError: invalid syntax >>> async \ Segmentation fault root@4c09392f83c8:/# Also, - file: test.py #/usr/bin/python3.6 async \ - $ ./test.py Segmentation fault $ Haven't taken the time to produce a backtrace or investigate with latest the dev versions or any further. Let me know if I can assist in any way. -- components: Interpreter Core messages: 304835 nosy: Alexandre Hamelin priority: normal severity: normal status: open title: Crashes with lines of the form "async \" type: crash versions: Python 3.6 ___ Python tracker <https://bugs.python.org/issue31852> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31852] Crashes with lines of the form "async \"
Alexandre Hamelin added the comment: Awesome work, thanks to you! Would it also be the case for 'await' ? -- ___ Python tracker <https://bugs.python.org/issue31852> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46977] tempfile.TemporaryDirectory fails removing dir in some edge cases related to symlinks
New submission from Alexandre Feblot : ```python #!/usr/bin/env python3 import os import tempfile def createUnremovableDir(workdir): print(workdir) os.mkdir(f'{workdir}/mydir') os.symlink('/bin/bash', f'{workdir}/mydir/mylink') # Symlink to a root owned file os.chmod(f'{workdir}/mydir', 0o555) with tempfile.TemporaryDirectory() as workdir: createUnremovableDir(workdir) ``` Fails because `tempfile.TemporaryDirectory._rmtree` tries to execute os.chmod(path, 0o700) on the symlink, which by default tries to change the root owned file symlink target instead of the symlink itself: ``` /tmp/tmp1_dy42ef Traceback (most recent call last): File "/usr/lib/python3.9/shutil.py", line 682, in _rmtree_safe_fd os.unlink(entry.name, dir_fd=topfd) PermissionError: [Errno 13] Permission denied: 'mylink' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "~/tempfile_demo_bug.py", line 13, in createUnremovableDir(workdir) File "/usr/lib/python3.9/tempfile.py", line 969, in __exit__ self.cleanup() File "/usr/lib/python3.9/tempfile.py", line 973, in cleanup self._rmtree(self.name) File "/usr/lib/python3.9/tempfile.py", line 955, in _rmtree _rmtree(name, onerror=onerror) File "/usr/lib/python3.9/shutil.py", line 727, in rmtree _rmtree_safe_fd(fd, path, onerror) File "/usr/lib/python3.9/shutil.py", line 664, in _rmtree_safe_fd _rmtree_safe_fd(dirfd, fullname, onerror) File "/usr/lib/python3.9/shutil.py", line 684, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/usr/lib/python3.9/tempfile.py", line 941, in onerror resetperms(path) File "/usr/lib/python3.9/tempfile.py", line 936, in resetperms _os.chmod(path, 0o700) PermissionError: [Errno 1] Operation not permitted: '/tmp/tmp1_dy42ef/mydir/mylink' ``` and leaves: ``` (.venv python 3.9.9) $ find /tmp/tmp1_dy42ef -ls 148228 4 drwx-- 3 myuser myuser 4096 Mar 10 16:54 /tmp/tmp1_dy42ef 148229 4 drwx-- 2 myuser myuser 4096 Mar 10 16:54 /tmp/tmp1_dy42ef/mydir 148230 0 lrwxrwxrwx 1 myuser myuser9 Mar 10 16:54 /tmp/tmp1_dy42ef/mydir/mylink -> /bin/bash ``` This fixes it: ``` python #!/usr/bin/env python3 import os import tempfile def createUnremovableDir(workdir): print(workdir) os.mkdir(f'{workdir}/mydir') os.symlink('/bin/bash', f'{workdir}/mydir/mylink') # Symlink to a root owned file os.chmod(f'{workdir}/mydir', 0o555) def _rmtree(cls, name, ignore_errors=False): def onerror(func, path, exc_info): if issubclass(exc_info[0], PermissionError): def resetperms(path): try: if os.chflags in os.supports_follow_symlinks: # This is the patch os.chflags(path, 0, follow_symlinks=False) # This is the patch elif not os.path.islink(path): # This is the patch os.chflags(path, 0) except AttributeError: pass if os.chmod in os.supports_follow_symlinks:# This is the patch os.chmod(path, 0o700, follow_symlinks=False) # This is the patch elif not os.path.islink(path): # This is the patch os.chmod(path, 0o700) try: if path != name: resetperms(os.path.dirname(path)) resetperms(path) try: os.unlink(path) # PermissionError is raised on FreeBSD for directories except (IsADirectoryError, PermissionError): cls._rmtree(path, ignore_errors=ignore_errors) except FileNotFoundError: pass elif issubclass(exc_info[0], FileNotFoundError): pass else: if not ignore_errors: raise shutil.rmtree(name, onerror=onerror) # Monkey patch the class method tempfile.TemporaryDirectory._rmtree from types import MethodType import shutil tempfile.TemporaryDirectory._rmtree = MethodType(_rmtree, tempfile.TemporaryDirectory) with tempfile.TemporaryDirectory() as workdir: createUnremovableDir(workdir) -- components: Library (Lib) messages: 414871 nosy: afeblot priority: normal severity: normal status: open title: tempfile.TemporaryDirectory fails removing dir in some edge cases related to symlinks versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue46977> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46977] tempfile.TemporaryDirectory fails removing dir in some edge cases related to symlinks
Change by Alexandre Feblot : -- type: -> crash ___ Python tracker <https://bugs.python.org/issue46977> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1032] Improve the hackish runtime_library_dirs support for gcc
New submission from Alexandre Vassalotti: In distutils.unixccompiler, there is a hack to passing correctly the -R option to gcc (and a few other compilers). However, there is a small bug that causes gcc to not be detected correctly if the compiler name does not start with "gcc" (e.g., "i486-linux-gnu-gcc", or "ccache gcc"). -- components: Distutils messages: 55317 nosy: alexandre.vassalotti severity: minor status: open title: Improve the hackish runtime_library_dirs support for gcc type: compile error __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1032> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1032] Improve the hackish runtime_library_dirs support for gcc
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1032> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1029] py3k: io.StringIO.getvalue() returns \r\n
Alexandre Vassalotti added the comment: As far as I know, StringIO should not do any string transformations. >From PEP-3116 "New I/O", last paragraph of the section "Text I/O": > Another implementation, StringIO, creates a file-like TextIO > implementation without an underlying Buffered I/O object. [...] It > does not support encodings or newline translations; you always read > back exactly the characters you wrote. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1029> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1029] py3k: io.StringIO.getvalue() returns \r\n
Alexandre Vassalotti added the comment: > That's why the current behaviour is not correct: When I write('\n'), > getvalue() currently returns '\r\n'. Oh, I missed your example in your initial message. So yes, I agree that StringIO shouldn't translate newlines like that. I attached a patch that should fix the bug. However, I would favor removing the "newline" keyword argument, instead. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1029> __Index: Lib/io.py === --- Lib/io.py (revision 57506) +++ Lib/io.py (working copy) @@ -1367,6 +1367,7 @@ initial_value = str(initial_value) self.write(initial_value) self.seek(0) +self._writetranslate = False def getvalue(self): self.flush() ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1032] Improve the hackish runtime_library_dirs support for gcc
Alexandre Vassalotti added the comment: > The patch is incorrect since find returns -1 on failure. Oops, that is right. I attached the corrected patch. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1032> __Index: Lib/distutils/unixccompiler.py === --- Lib/distutils/unixccompiler.py (revision 57274) +++ Lib/distutils/unixccompiler.py (working copy) @@ -282,7 +282,7 @@ return "+s -L" + dir elif sys.platform[:7] == "irix646" or sys.platform[:6] == "osf1V5": return ["-rpath", dir] -elif compiler[:3] == "gcc" or compiler[:3] == "g++": +elif "gcc" in compiler or "g++" in compiler: return "-Wl,-R" + dir else: return "-R" + dir ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1247] PEP 3137 patch (repr, names, parser)
Alexandre Vassalotti added the comment: I am not sure how the parser work, so I can't comment if your change to it is correct. It would probably a better idea to keep it for later, when the semantic differences between str8 and bytes objects will be resolved. I think changing b'' to buffer(b''), in the tests, is not a good idea. These tests will need to be changed again when bytes() will be changed to str8(). Your changes to PyString and PyBytes look good to me. As I said on the mailing list, I think the for-loops in bytes_repr() should be changed to: while (*quote_prefix) *p++ = *quote_prefix++; I attached updated patches with the above change. I also replaced bytes() for buffer() in PyBytes's docstrings. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1247> __Index: Objects/bytesobject.c === --- Objects/bytesobject.c (revision 58376) +++ Objects/bytesobject.c (working copy) @@ -395,7 +395,7 @@ if (i < 0) i += Py_Size(self); if (i < 0 || i >= Py_Size(self)) { -PyErr_SetString(PyExc_IndexError, "bytes index out of range"); +PyErr_SetString(PyExc_IndexError, "buffer index out of range"); return NULL; } return PyInt_FromLong((unsigned char)(self->ob_bytes[i])); @@ -414,7 +414,7 @@ i += PyBytes_GET_SIZE(self); if (i < 0 || i >= Py_Size(self)) { -PyErr_SetString(PyExc_IndexError, "bytes index out of range"); +PyErr_SetString(PyExc_IndexError, "buffer index out of range"); return NULL; } return PyInt_FromLong((unsigned char)(self->ob_bytes[i])); @@ -451,7 +451,7 @@ } } else { -PyErr_SetString(PyExc_TypeError, "bytes indices must be integers"); +PyErr_SetString(PyExc_TypeError, "buffer indices must be integers"); return NULL; } } @@ -551,7 +551,7 @@ i += Py_Size(self); if (i < 0 || i >= Py_Size(self)) { -PyErr_SetString(PyExc_IndexError, "bytes index out of range"); +PyErr_SetString(PyExc_IndexError, "buffer index out of range"); return -1; } @@ -587,7 +587,7 @@ i += PyBytes_GET_SIZE(self); if (i < 0 || i >= Py_Size(self)) { -PyErr_SetString(PyExc_IndexError, "bytes index out of range"); +PyErr_SetString(PyExc_IndexError, "buffer index out of range"); return -1; } @@ -619,7 +619,7 @@ } } else { -PyErr_SetString(PyExc_TypeError, "bytes indices must be integer"); +PyErr_SetString(PyExc_TypeError, "buffer indices must be integer"); return -1; } @@ -889,11 +889,14 @@ bytes_repr(PyBytesObject *self) { static const char *hexdigits = "0123456789abcdef"; -size_t newsize = 3 + 4 * Py_Size(self); +const char *quote_prefix = "buffer(b'"; +const char *quote_postfix = "')"; +/* 9 prefix + 2 postfix */ +size_t newsize = 11 + 4 * Py_Size(self); PyObject *v; -if (newsize > PY_SSIZE_T_MAX || newsize / 4 != Py_Size(self)) { +if (newsize > PY_SSIZE_T_MAX || (newsize-11) / 4 != Py_Size(self)) { PyErr_SetString(PyExc_OverflowError, -"bytes object is too large to make repr"); +"buffer object is too large to make repr"); return NULL; } v = PyUnicode_FromUnicode(NULL, newsize); @@ -904,17 +907,17 @@ register Py_ssize_t i; register Py_UNICODE c; register Py_UNICODE *p; -int quote = '\''; p = PyUnicode_AS_UNICODE(v); -*p++ = 'b'; -*p++ = quote; +while (*quote_prefix) +*p++ = *quote_prefix++; + for (i = 0; i < Py_Size(self); i++) { /* There's at least enough room for a hex escape and a closing quote. */ assert(newsize - (p - PyUnicode_AS_UNICODE(v)) >= 5); c = self->ob_bytes[i]; -if (c == quote || c == '\\') +if (c == '\'' || c == '\\') *p++ = '\\', *p++ = c; else if (c == '\t') *p++ = '\\', *p++ = 't'; @@ -934,7 +937,9 @@ *p++ = c; } assert(newsize - (p - PyUnicode_AS_UNICODE(v)) >= 1); -*p++ = quote; +while (*quote_postfix) { + *p++ = *quote_postfix++; +} *p = '\0'; if (PyUnicode_Resize(&v, (p - PyUnicode_AS_UNICODE(v)))
[issue1247] PEP 3137 patch (repr, names, parser)
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1247> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1247] PEP 3137 patch (repr, names, parser)
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1247> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1260] PEP 3137: Remove the buffer API from PyUnicode
New submission from Alexandre Vassalotti: This patch removes the buffer API from PyUnicode, as specified by PEP 3137. All the unit tests passes. However, I believe there is a few function calls to PyArg API that could become problematic if they are given a PyUnicode object: % egrep -R --include='*.c' 'PyArg.*Parse.*"[^;:]*[cwt].*"' py3k/ I haven't checked these function calls yet. So, it would probably be a good idea to wait I do so before committing this patch. -- files: unicode_rm_buf_api.patch messages: 56336 nosy: alexandre.vassalotti severity: normal status: open title: PEP 3137: Remove the buffer API from PyUnicode versions: Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1260> __Index: Modules/_sre.c === --- Modules/_sre.c (revision 58376) +++ Modules/_sre.c (working copy) @@ -1674,6 +1674,15 @@ void* ptr; Py_buffer view; +/* Unicode objects do not support the buffer API. So, get the data + directly instead. */ +if (PyUnicode_Check(string)) { +ptr = (void *)PyUnicode_AS_DATA(string); +*p_length = PyUnicode_GET_SIZE(string); +*p_charsize = sizeof(Py_UNICODE); +return ptr; +} + /* get pointer to string buffer */ view.len = -1; buffer = Py_Type(string)->tp_as_buffer; Index: Objects/unicodeobject.c === --- Objects/unicodeobject.c (revision 58376) +++ Objects/unicodeobject.c (working copy) @@ -8113,19 +8113,6 @@ }; -static int -unicode_buffer_getbuffer(PyUnicodeObject *self, Py_buffer *view, int flags) -{ - -if (flags & PyBUF_CHARACTER) { -PyErr_SetString(PyExc_SystemError, "can't use str as char buffer"); -return -1; -} -return PyBuffer_FillInfo(view, (void *)self->str, - PyUnicode_GET_DATA_SIZE(self), 1, flags); -} - - /* Helpers for PyUnicode_Format() */ static PyObject * @@ -8819,11 +8806,6 @@ return NULL; } -static PyBufferProcs unicode_as_buffer = { -(getbufferproc) unicode_buffer_getbuffer, -NULL, -}; - static PyObject * unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds); @@ -8907,7 +8889,7 @@ (reprfunc) unicode_str, /* tp_str */ PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ -&unicode_as_buffer, /* tp_as_buffer */ +0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_UNICODE_SUBCLASS, /* tp_flags */ unicode_doc, /* tp_doc */ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: I found a few problems in your patch. In PyCode_New() the type check for the filename argument is incorrect: --- Objects/codeobject.c(revision 58412) +++ Objects/codeobject.c(working copy) @@ -59,7 +59,7 @@ freevars == NULL || !PyTuple_Check(freevars) || cellvars == NULL || !PyTuple_Check(cellvars) || name == NULL || (!PyString_Check(name) && !PyUnicode_Check(name)) || - filename == NULL || !PyString_Check(filename) || + filename == NULL || (!PyString_Check(name) && !PyUnicode_Check(name)) || lnotab == NULL || !PyString_Check(lnotab) || !PyObject_CheckReadBuffer(code)) { PyErr_BadInternalCall(); @@ -260,6 +267,8 @@ ourcellvars = PyTuple_New(0); if (ourcellvars == NULL) goto cleanup; +filename = PyUnicode_DecodeFSDefault(PyString_AS_STRING(filename), + 0, NULL); The following is unnecessary and will cause a reference leak: @@ -260,6 +267,8 @@ ourcellvars = PyTuple_New(0); if (ourcellvars == NULL) goto cleanup; +filename = PyUnicode_DecodeFSDefault(PyString_AS_STRING(filename), + 0, NULL); co = (PyObject *)PyCode_New(argcount, kwonlyargcount, nlocals, stacksize, flags, I think the interface of PyUnicode_DecodeFSDefault() could be improved a bit. The function doesn't use the last argument 'errors', so why is there? I am not sure if it is useful to keep second argument, 'length', either. So, I believe the function prototype should be changed to: PyObject *PyUnicode_Decode_FSDefault(const char *s); Another thing that I am not sure about is whether it is correct to consider ISO-8859-15 the same thing as Latin-1. Overall, the patch looks good to me and doesn't cause any test to fail. I attached an updated patch with the above issues fixed. Thank you, Christian, for the patch. :) -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: Guido wrote: > Why copy the default encoding before mangling it? With a little > extra care you will only have to copy it once. Also, consider not > mangling at all, but assuming the encoding comes in a canonical form > -- several other functions assume that, e.g. PyUnicode_Decode() and > PyUnicode_AsEncodedString(). It is impossible guarantee that Py_FileSystemDefaultEncoding is normalized, since its value can be set using nl_langinfo(CODESET) during the bootstrapping process. PyUnicode_Decode() and other decoding/encoding functions use the codec module, which is not available during the early bootstrapping process, to normalize the encoding name. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1260] PEP 3137: Remove the buffer API from PyUnicode
Alexandre Vassalotti added the comment: There was a problem with one of the call of PyArg_ParseTuple in the OS/2 version of listdir() in the posix module. I also clarified the error message of the 't' format unit. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1260> __Index: Python/getargs.c === --- Python/getargs.c (revision 58422) +++ Python/getargs.c (working copy) @@ -1250,7 +1250,7 @@ arg, msgbuf, bufsize); if (pb == NULL || pb->bf_getbuffer == NULL) return converterr( -"string or read-only character buffer", +"bytes or read-only character buffer", arg, msgbuf, bufsize); if ((*pb->bf_getbuffer)(arg, &view, PyBUF_CHARACTER) != 0) Index: Objects/unicodeobject.c === --- Objects/unicodeobject.c (revision 58422) +++ Objects/unicodeobject.c (working copy) @@ -8113,19 +8113,6 @@ }; -static int -unicode_buffer_getbuffer(PyUnicodeObject *self, Py_buffer *view, int flags) -{ - -if (flags & PyBUF_CHARACTER) { -PyErr_SetString(PyExc_SystemError, "can't use str as char buffer"); -return -1; -} -return PyBuffer_FillInfo(view, (void *)self->str, - PyUnicode_GET_DATA_SIZE(self), 1, flags); -} - - /* Helpers for PyUnicode_Format() */ static PyObject * @@ -8819,11 +8806,6 @@ return NULL; } -static PyBufferProcs unicode_as_buffer = { -(getbufferproc) unicode_buffer_getbuffer, -NULL, -}; - static PyObject * unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds); @@ -8907,7 +8889,7 @@ (reprfunc) unicode_str, /* tp_str */ PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ -&unicode_as_buffer, /* tp_as_buffer */ +0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_UNICODE_SUBCLASS, /* tp_flags */ unicode_doc, /* tp_doc */ Index: Modules/_sre.c === --- Modules/_sre.c (revision 58422) +++ Modules/_sre.c (working copy) @@ -1674,6 +1674,15 @@ void* ptr; Py_buffer view; +/* Unicode objects do not support the buffer API. So, get the data + directly instead. */ +if (PyUnicode_Check(string)) { +ptr = (void *)PyUnicode_AS_DATA(string); +*p_length = PyUnicode_GET_SIZE(string); +*p_charsize = sizeof(Py_UNICODE); +return ptr; +} + /* get pointer to string buffer */ view.len = -1; buffer = Py_Type(string)->tp_as_buffer; Index: Modules/posixmodule.c === --- Modules/posixmodule.c (revision 58422) +++ Modules/posixmodule.c (working copy) @@ -2135,7 +2135,8 @@ FILEFINDBUF3 ep; APIRET rc; -if (!PyArg_ParseTuple(args, "t#:listdir", &name, &len)) +if (!PyArg_ParseTuple(args, "et#:listdir", + Py_FileSystemDefaultEncoding, &name, &len)) return NULL; if (len >= MAX_PATH) { PyErr_SetString(PyExc_ValueError, "path too long"); ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: Guido wrote: > I figured out the problem -- it came from marshalled old code objects. > If you throw away all .pyc files the problem goes away. You can also > get rid of the similar checks for the 'name' argument -- this should > just be a PyUnicode too. A systematic approach to invalidating all the > .pyc files is updating the magic number in import.c. Done. I had to remove a few another PyString instances in pyexpat.c and _ctypes.c. So, here (hopefully) the final version of the patch. The changes from the last version are: - Correct a typo in of the comments in PyUnicode_DecodeFSDefault - Specified in the API doc of PyUnicode_DecodeFSDefault that the function take a null-terminated string. - Bumped the magic number in import.c - Fix PyCode_New calls in _ctypes and pyexpat module. - Remove the PyString type check on 'filename' and 'name' in PyCode_New. - Remove the unneeded string coercion code from PyCode_New. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __Index: Python/ceval.c === --- Python/ceval.c (revision 58422) +++ Python/ceval.c (working copy) @@ -767,7 +767,7 @@ lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL; #endif #if defined(Py_DEBUG) || defined(LLTRACE) - filename = PyString_AsString(co->co_filename); + filename = PyUnicode_AsString(co->co_filename); #endif why = WHY_NOT; Index: Python/traceback.c === --- Python/traceback.c (revision 58422) +++ Python/traceback.c (working copy) @@ -229,10 +229,10 @@ while (tb != NULL && err == 0) { if (depth <= limit) { err = tb_displayline(f, - PyString_AsString( + PyUnicode_AsString( tb->tb_frame->f_code->co_filename), tb->tb_lineno, - PyString_AsString(tb->tb_frame->f_code->co_name)); + PyUnicode_AsString(tb->tb_frame->f_code->co_name)); } depth--; tb = tb->tb_next; Index: Python/pythonrun.c === --- Python/pythonrun.c (revision 58422) +++ Python/pythonrun.c (working copy) @@ -867,7 +867,8 @@ return -1; d = PyModule_GetDict(m); if (PyDict_GetItemString(d, "__file__") == NULL) { - PyObject *f = PyString_FromString(filename); + PyObject *f; + f = PyUnicode_DecodeFSDefault(filename); if (f == NULL) return -1; if (PyDict_SetItemString(d, "__file__", f) < 0) { Index: Python/import.c === --- Python/import.c (revision 58422) +++ Python/import.c (working copy) @@ -74,10 +74,11 @@ 3040 (added signature annotations) 3050 (print becomes a function) 3060 (PEP 3115 metaclass syntax) - 3070 (PEP 3109 raise changes) + 3070 (PEP 3109 raise changes) + 3080 (PEP 3137 make __file__ and __name__ unicode) . */ -#define MAGIC (3070 | ((long)'\r'<<16) | ((long)'\n'<<24)) +#define MAGIC (3080 | ((long)'\r'<<16) | ((long)'\n'<<24)) /* Magic word as global; note that _PyImport_Init() can change the value of this global to accommodate for alterations of how the @@ -652,7 +653,7 @@ /* Remember the filename as the __file__ attribute */ v = NULL; if (pathname != NULL) { - v = PyString_FromString(pathname); + v = PyUnicode_DecodeFSDefault(pathname); if (v == NULL) PyErr_Clear(); } @@ -983,7 +984,7 @@ PySys_WriteStderr("import %s # directory %s\n", name, pathname); d = PyModule_GetDict(m); - file = PyString_FromString(pathname); + file = PyUnicode_DecodeFSDefault(pathname); if (file == NULL) goto error; path = Py_BuildValue("[O]", file); Index: Python/compile.c === --- Python/compile.c (revision 58422) +++ Python/compile.c (working copy) @@ -4001,7 +4001,7 @@ freevars = dict_keys_inorder(c->u->u_freevars, PyTuple_Size(cellvars)); if (!freevars) goto error; - filename = PyString_FromString(c->c_filename); + filename = PyUnicode_DecodeFSDefault(c->c_filename); if (!filename) goto error; Index: Python/importdl.c === --- Python/importdl.c (revision 58422) +++ Python/importdl.c (working copy) @@ -62,7 +62,9 @@ return NULL; } /* Remember the filename as the __file__ attribute */ - if (PyModule_AddStringConstant(m, "__file__", pathname) < 0) + PyObject *path; + path = PyUnicode_DecodeFSDefault(pathname); + if (PyModule_AddObject(m, "__file__", path) < 0) PyErr_Clear(); /
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: > Remove the PyString type check on 'filename' and 'name' in PyCode_New. Oops. I removed one of the ! the checks by mistake. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __Index: Python/ceval.c === --- Python/ceval.c (revision 58454) +++ Python/ceval.c (working copy) @@ -767,7 +767,7 @@ lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL; #endif #if defined(Py_DEBUG) || defined(LLTRACE) - filename = PyString_AsString(co->co_filename); + filename = PyUnicode_AsString(co->co_filename); #endif why = WHY_NOT; Index: Python/traceback.c === --- Python/traceback.c (revision 58454) +++ Python/traceback.c (working copy) @@ -229,10 +229,10 @@ while (tb != NULL && err == 0) { if (depth <= limit) { err = tb_displayline(f, - PyString_AsString( + PyUnicode_AsString( tb->tb_frame->f_code->co_filename), tb->tb_lineno, - PyString_AsString(tb->tb_frame->f_code->co_name)); + PyUnicode_AsString(tb->tb_frame->f_code->co_name)); } depth--; tb = tb->tb_next; Index: Python/pythonrun.c === --- Python/pythonrun.c (revision 58454) +++ Python/pythonrun.c (working copy) @@ -867,7 +867,8 @@ return -1; d = PyModule_GetDict(m); if (PyDict_GetItemString(d, "__file__") == NULL) { - PyObject *f = PyString_FromString(filename); + PyObject *f; + f = PyUnicode_DecodeFSDefault(filename); if (f == NULL) return -1; if (PyDict_SetItemString(d, "__file__", f) < 0) { Index: Python/import.c === --- Python/import.c (revision 58454) +++ Python/import.c (working copy) @@ -74,10 +74,11 @@ 3040 (added signature annotations) 3050 (print becomes a function) 3060 (PEP 3115 metaclass syntax) - 3070 (PEP 3109 raise changes) + 3070 (PEP 3109 raise changes) + 3080 (PEP 3137 make __file__ and __name__ unicode) . */ -#define MAGIC (3070 | ((long)'\r'<<16) | ((long)'\n'<<24)) +#define MAGIC (3080 | ((long)'\r'<<16) | ((long)'\n'<<24)) /* Magic word as global; note that _PyImport_Init() can change the value of this global to accommodate for alterations of how the @@ -652,7 +653,7 @@ /* Remember the filename as the __file__ attribute */ v = NULL; if (pathname != NULL) { - v = PyString_FromString(pathname); + v = PyUnicode_DecodeFSDefault(pathname); if (v == NULL) PyErr_Clear(); } @@ -983,7 +984,7 @@ PySys_WriteStderr("import %s # directory %s\n", name, pathname); d = PyModule_GetDict(m); - file = PyString_FromString(pathname); + file = PyUnicode_DecodeFSDefault(pathname); if (file == NULL) goto error; path = Py_BuildValue("[O]", file); Index: Python/compile.c === --- Python/compile.c (revision 58454) +++ Python/compile.c (working copy) @@ -4001,7 +4001,7 @@ freevars = dict_keys_inorder(c->u->u_freevars, PyTuple_Size(cellvars)); if (!freevars) goto error; - filename = PyString_FromString(c->c_filename); + filename = PyUnicode_DecodeFSDefault(c->c_filename); if (!filename) goto error; Index: Python/importdl.c === --- Python/importdl.c (revision 58454) +++ Python/importdl.c (working copy) @@ -62,7 +62,9 @@ return NULL; } /* Remember the filename as the __file__ attribute */ - if (PyModule_AddStringConstant(m, "__file__", pathname) < 0) + PyObject *path; + path = PyUnicode_DecodeFSDefault(pathname); + if (PyModule_AddObject(m, "__file__", path) < 0) PyErr_Clear(); /* Not important enough to report */ if (_PyImport_FixupExtension(name, pathname) == NULL) Index: Include/unicodeobject.h === --- Include/unicodeobject.h (revision 58454) +++ Include/unicodeobject.h (working copy) @@ -154,6 +154,7 @@ # define PyUnicode_DecodeASCII PyUnicodeUCS2_DecodeASCII # define PyUnicode_DecodeCharmap PyUnicodeUCS2_DecodeCharmap # define PyUnicode_DecodeLatin1 PyUnicodeUCS2_DecodeLatin1 +# define PyUnicode_DecodeFSDefault PyUnicodeUCS2_DecodeFSDefault # define PyUnicode_DecodeRawUnicodeEscape PyUnicodeUCS2_DecodeRawUnicodeEscape # define PyUnicode_DecodeUTF32 PyUnicodeUCS2_DecodeUTF32 # define PyUnicode_DecodeUTF32Stateful PyUnicodeUCS2_DecodeUTF32Stateful @@ -245,6 +246,7 @@ # define PyUnicode_DecodeASCII PyUnicodeUCS4_DecodeASCII # define PyUnicode_Deco
[issue1260] PEP 3137: Remove the buffer API from PyUnicode
Alexandre Vassalotti added the comment: Committed in r58455. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1260> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: Christian wrote: > Alexandre's mangle loop doesn't do the same job as mine. Chars like _ > and - aren't removed from the encoding name and the if clauses don't > catch for example UTF-8 or ISO-8859-1 only UTF8 or ISO8859-1. That isn't true. My mangler does exactly the same thing as your original one. However, I forgot to add Py_CHARMASK to the calls of tolower() and isalnum() which would cause problems on platforms with signed char. > Also he has overseen a PyString_Check in the code repr function. Fixed. > We have to get the codecs up and Py_FileSystemEncoding before we can > decode the filenames. :( I think that the problem needs much more > attention and a proper fix. Maybe adding a global variable, let's say _Py_Codecs_Ready, could be used to notify PyUnicode_DecodeFSDefault that it can use PyUnicode_Decode, instead of relying only on the built-in codecs. That would be much simpler than changing boostrapping process. Here yet another updated patch. The changes are the following: - Add Py_CHARMASK to tolower() and isalnum() calls in PyUnicode_DecodeFSDefault(). - Use PyUnicode_Check(), instead of PyString_Check(), in code_repr(). - Update comments for co_filename and co_name in PyCodeObject struct definition. - Fix a PyString_AS_STRING(co->co_name) instance in compile.c - Replace %S for %U in PyErr_Format() calls for substituting co_name. One more thing, frozen.c needs to be updated. The module data contains a code object with a PyString co_name. However, there is a bug in the imp module (it doesn't detect the encoding from modelines, which cause io.TextIOWrapper to crash) that prevents me from generating the data for frozen.c. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __Index: Python/ceval.c === --- Python/ceval.c (revision 58455) +++ Python/ceval.c (working copy) @@ -767,7 +767,7 @@ lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL; #endif #if defined(Py_DEBUG) || defined(LLTRACE) - filename = PyString_AsString(co->co_filename); + filename = PyUnicode_AsString(co->co_filename); #endif why = WHY_NOT; @@ -2565,7 +2565,7 @@ if (argcount > co->co_argcount) { if (!(co->co_flags & CO_VARARGS)) { PyErr_Format(PyExc_TypeError, -"%S() takes %s %d " +"%U() takes %s %d " "%spositional argument%s (%d given)", co->co_name, defcount ? "at most" : "exactly", @@ -2599,7 +2599,7 @@ int j; if (keyword == NULL || !PyUnicode_Check(keyword)) { PyErr_Format(PyExc_TypeError, -"%S() keywords must be strings", +"%U() keywords must be strings", co->co_name); goto fail; } @@ -2622,7 +2622,7 @@ if (j >= co->co_argcount + co->co_kwonlyargcount) { if (kwdict == NULL) { PyErr_Format(PyExc_TypeError, - "%S() got an unexpected " + "%U() got an unexpected " "keyword argument '%S'", co->co_name, keyword); @@ -2633,7 +2633,7 @@ else { if (GETLOCAL(j) != NULL) { PyErr_Format(PyExc_TypeError, - "%S() got multiple " + "%U() got multiple " "values for keyword " "argument '%S'", co->co_name, @@ -2661,7 +2661,7 @@ continue; } PyErr_Format(PyExc_TypeError, - "%S() needs keyword-only argument %S", + "%U() needs keyword-only argument %S", co->co_name, name); goto fail; } @@ -2671,7 +2671,7 @@ for (i = argcount; i < m; i++) { if (GETLOCAL(i) == NULL) { PyErr_Format(PyExc_TypeError, - "%S() takes %s %d " + "%U() takes %s %d " "%spositional argument%s " "(%d given)", co->co_name, @@ -2699,7 +2699,7 @@ else { if (argcount > 0 || kwcount > 0) { PyErr_Format(PyExc_TypeError, - "%S() takes no arguments (%d given)", + "%U() takes no arguments (%d given)", co->co_name, argcount + kwcount); goto fail; Index: Python/traceback.c === --- Python/traceback.c (revision 58455) +++ Python/traceback.c (working copy) @@ -229,10 +229,10 @@ while (tb != NULL && err == 0) { if (depth <= limit) { err = tb_displayline(f, - PyString_AsString( + PyUnicode_AsString( tb->tb_frame->f_code->co_filename), tb->tb_lineno, - PyString_AsStrin
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: > I have a question for Alexandre related to frozen.c -- why is there a > mode line with an encoding involved in freezing hello.py? For some reason which I don't know about, freeze.py tries to read all the modules accessible from sys.path: # collect all modules of the program dir = os.path.dirname(scriptfile) path[0] = dir mf = modulefinder.ModuleFinder(path, debug, exclude, replace_paths) The problem is the imp module, which modulefinder uses, does not detect the encoding of the files from the mode-line. This causes TextIOWrapper to crash when it tries to read modules using an encoding other than ASCII or UTF-8. Here an example: >>> import imp >>> imp.find_module('heapq')[0].read() Traceback (most recent call last): ... UnicodeDecodeError: 'utf8' codec can't decode bytes in position 1428-1430: invalid data I probably should open a seperate issue in the tracker for this, though. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: I thought of another way to implement PyUnicode_DecodeFSDefault. If Py_FileSystemDefaultEncoding is set, decode with the codecs module, otherwise use UTF-8 + replace. This works because when Py_FileSystemDefaultEncoding is initialized at the end of Py_InitializeEx(), the codecs module is ready to be used. Here's what it looks like: PyObject* PyUnicode_DecodeFSDefault(const char *s) { Py_ssize_t size = (Py_ssize_t)strlen(s); /* During the early bootstrapping process, Py_FileSystemDefaultEncoding can be undefined. If it is case, decode using UTF-8. The following assumes that Py_FileSystemDefaultEncoding is set to a built-in encoding during the bootstrapping process where the codecs aren't ready yet. */ if (Py_FileSystemDefaultEncoding) { return PyUnicode_Decode(s, size, Py_FileSystemDefaultEncoding, "replace"); } else { return PyUnicode_DecodeUTF8(s, size, "replace"); } } It is not perfect, since the extra function calls in the codecs module causes test_profile and test_doctest to fail. However, I think this is much simpler that the previous versions. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __Index: Python/ceval.c === --- Python/ceval.c (revision 58455) +++ Python/ceval.c (working copy) @@ -767,7 +767,7 @@ lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL; #endif #if defined(Py_DEBUG) || defined(LLTRACE) - filename = PyString_AsString(co->co_filename); + filename = PyUnicode_AsString(co->co_filename); #endif why = WHY_NOT; @@ -2565,7 +2565,7 @@ if (argcount > co->co_argcount) { if (!(co->co_flags & CO_VARARGS)) { PyErr_Format(PyExc_TypeError, -"%S() takes %s %d " +"%U() takes %s %d " "%spositional argument%s (%d given)", co->co_name, defcount ? "at most" : "exactly", @@ -2599,7 +2599,7 @@ int j; if (keyword == NULL || !PyUnicode_Check(keyword)) { PyErr_Format(PyExc_TypeError, -"%S() keywords must be strings", +"%U() keywords must be strings", co->co_name); goto fail; } @@ -2622,7 +2622,7 @@ if (j >= co->co_argcount + co->co_kwonlyargcount) { if (kwdict == NULL) { PyErr_Format(PyExc_TypeError, - "%S() got an unexpected " + "%U() got an unexpected " "keyword argument '%S'", co->co_name, keyword); @@ -2633,7 +2633,7 @@ else { if (GETLOCAL(j) != NULL) { PyErr_Format(PyExc_TypeError, - "%S() got multiple " + "%U() got multiple " "values for keyword " "argument '%S'", co->co_name, @@ -2661,7 +2661,7 @@ continue; } PyErr_Format(PyExc_TypeError, - "%S() needs keyword-only argument %S", + "%U() needs keyword-only argument %S", co->co_name, name); goto fail; } @@ -2671,7 +2671,7 @@ for (i = argcount; i < m; i++) { if (GETLOCAL(i) == NULL) { PyErr_Format(PyExc_TypeError, - "%S() takes %s %d " + "%U() takes %s %d " "%spositional argument%s " "(%d given)", co->co_name, @@ -2699,7 +2699,7 @@ else { if (argcount > 0 || kwcount > 0) { PyErr_Format(PyExc_TypeError, - "%S() takes no arguments (%d given)", + "%U() takes no arguments (%d given)", co->co_name, argcount + kwcount); goto fail; Index: Python/traceback.c === --- Python/traceback.c (revision 58455) +++ Python/traceback.c (working copy) @@ -229,10 +229,10 @@ while (tb != NULL && err == 0) { if (depth <= limit) { err = tb_displayline(f, - PyString_AsString( + PyUnicode_AsString( tb->tb_frame->f_code->co_filename), tb->tb_lineno, - PyString_AsString(tb->tb_frame->f_code->co_name)); + PyUnicode_AsString(tb->tb_frame->f_code->co_name)); } depth--; tb = tb->tb_next; Index: Python/pythonrun.c === --- Python/pythonrun.c (revision 58455) +++ Python/pythonrun.c (working copy) @@ -867,7 +867,8 @@ return -1; d = PyModule_GetDict(m); if (PyDict_GetItemString(d, "__file__") == NULL) { - PyObject *f = PyString_FromString(filename); + PyObject *f; + f = PyUnico
[issue1280] PEP 3137: Make PyString's indexing and iteration return integers
New submission from Alexandre Vassalotti: Here a preliminary patch to make PyString return integers on indexing and iteration. There is still quite a few XXX in the patch, that I would like to fix. However, the good thing is all tests passes. -- components: Interpreter Core files: string_iter_ret_ints.patch messages: 56442 nosy: alexandre.vassalotti severity: normal status: open title: PEP 3137: Make PyString's indexing and iteration return integers versions: Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1280> __Index: Objects/stringobject.c === --- Objects/stringobject.c (revision 58458) +++ Objects/stringobject.c (working copy) @@ -1233,23 +1233,13 @@ static PyObject * string_item(PyStringObject *a, register Py_ssize_t i) { - char pchar; - PyObject *v; + if (i < 0) + i += Py_Size(a); if (i < 0 || i >= Py_Size(a)) { PyErr_SetString(PyExc_IndexError, "string index out of range"); return NULL; } - pchar = a->ob_sval[i]; - v = (PyObject *)characters[pchar & UCHAR_MAX]; - if (v == NULL) - v = PyString_FromStringAndSize(&pchar, 1); - else { -#ifdef COUNT_ALLOCS - one_strings++; -#endif - Py_INCREF(v); - } - return v; + return PyInt_FromLong((unsigned char)a->ob_sval[i]); } static PyObject* @@ -5150,8 +5140,8 @@ assert(PyString_Check(seq)); if (it->it_index < PyString_GET_SIZE(seq)) { - item = PyString_FromStringAndSize( - PyString_AS_STRING(seq)+it->it_index, 1); + item = PyInt_FromLong( + (unsigned char)seq->ob_sval[it->it_index]); if (item != NULL) ++it->it_index; return item; Index: Lib/modulefinder.py === --- Lib/modulefinder.py (revision 58458) +++ Lib/modulefinder.py (working copy) @@ -367,7 +367,7 @@ consts = co.co_consts LOAD_LOAD_AND_IMPORT = LOAD_CONST + LOAD_CONST + IMPORT_NAME while code: -c = code[0] +c = chr(code[0]) if c in STORE_OPS: oparg, = unpack('t', c)[0] is not True: +for c in b'\x01\x7f\xff\x0f\xf0': +# XXX str8 constructor uses UTF-8 by default. So, to converting +# XXX to int to a str8 of length-1 require this odd maneuver. +x = str8(bytes(chr(255), 'latin-1')) +if struct.unpack('>t', x)[0] is not True: raise TestFailed('%c did not unpack as True' % c) test_bool() Index: Lib/test/string_tests.py === --- Lib/test/string_tests.py (revision 58458) +++ Lib/test/string_tests.py (working copy) @@ -558,6 +558,10 @@ a = self.type2test('DNSSEC') b = self.type2test('') for c in a: +# Special case for the str8, since indexing returns a integer +# XXX Maybe it would be a good idea to seperate str8's tests... +if self.type2test == str8: +c = chr(c) b += c hash(b) self.assertEqual(hash(a), hash(b)) Index: Lib/test/test_bytes.py === --- Lib/test/test_bytes.py (revision 58458) +++ Lib/test/test_bytes.py (working copy) @@ -347,7 +347,7 @@ sample = str8("Hello world\n\x80\x81\xfe\xff") buf = memoryview(sample) b = bytes(buf) -self.assertEqual(b, bytes(map(ord, sample))) +self.assertEqual(b, bytes(sample)) def test_to_str(self): sample = "Hello world\n\x80\x81\xfe\xff" Index: Lib/dis.py === --- Lib/dis.py (revision 58458) +++ Lib/dis.py (working copy) @@ -117,8 +117,7 @@ extended_arg = 0 free = None while i < n: -c = code[i] -op = ord(c) +op = code[i] if i in linestarts: if i > 0: print() @@ -134,7 +133,7 @@ print(opname[op].ljust(20), end=' ') i = i+1 if op >= HAVE_ARGUMENT: -oparg = ord(code[i]) + ord(code[i+1])*256 + extended_arg +oparg = code[i] + code[i+1]*256 + extended_arg extended_arg = 0 i = i+2 if op == EXTENDED_ARG: @@ -162,8 +161,7 @@ n = len(code) i = 0 while i < n: -c = code[i] -op = ord(c) +op = code[i] if i == lasti: print('-->', end=' ') else: print(' ', end=' ') if i in labels: print('>>', end=' ') @@ -172,7 +170,7 @@ print(opname[op].ljust(15), end=' ') i = i+1 if op >= HAVE_ARGUMEN
[issue1278] imp.find_module() ignores -*- coding: Latin-1 -*-
Changes by Alexandre Vassalotti: -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1278> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: I wrote in msg56419: > It is not perfect, since the extra function calls in the codecs module > causes test_profile and test_doctest to fail. How this was resolved? __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1302] Fixes for profile/cprofile
Alexandre Vassalotti added the comment: I don't think it's possible to add shortcuts in PyUnicode_Decode for UTF-16 and UTF-32 because the byte-order can be different depending of the platform. So, these two need to pass through the codecs module. I am sure if it's better, but I factored out the normalization routine into its own function. Added file: http://bugs.python.org/file8589/py3k_profile_fix-3.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1302> __Index: Objects/unicodeobject.c === --- Objects/unicodeobject.c (revision 58587) +++ Objects/unicodeobject.c (working copy) @@ -1049,29 +1049,55 @@ return NULL; } +static char * +normalize(const char *enc) +{ +register size_t i; +size_t len = strlen(enc); +char *p; + +p = PyMem_Malloc(len + 1); +if (p == NULL) +return NULL; +for (i = 0; i < len; i++) { +register char ch = enc[i]; +if (ch == ' ') +ch = '-'; +else +ch = tolower(Py_CHARMASK(ch)); +} +p[i] = '\0'; +return p; +} + PyObject *PyUnicode_Decode(const char *s, - Py_ssize_t size, - const char *encoding, - const char *errors) + Py_ssize_t size, + const char *encoding, + const char *errors) { PyObject *buffer = NULL, *unicode; Py_buffer info; +char *enc; if (encoding == NULL) - encoding = PyUnicode_GetDefaultEncoding(); +encoding = PyUnicode_GetDefaultEncoding(); +enc = normalize(encoding); + /* Shortcuts for common default encodings */ -if (strcmp(encoding, "utf-8") == 0) +if (strcmp(enc, "utf-8") == 0) return PyUnicode_DecodeUTF8(s, size, errors); -else if (strcmp(encoding, "latin-1") == 0) +else if (strcmp(enc, "latin-1") == 0) return PyUnicode_DecodeLatin1(s, size, errors); #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T) -else if (strcmp(encoding, "mbcs") == 0) +else if (strcmp(enc, "mbcs") == 0) return PyUnicode_DecodeMBCS(s, size, errors); #endif -else if (strcmp(encoding, "ascii") == 0) +else if (strcmp(enc, "ascii") == 0) return PyUnicode_DecodeASCII(s, size, errors); +PyMem_Free(enc); + /* Decode via the codec registry */ buffer = NULL; if (PyBuffer_FillInfo(&info, (void *)s, size, 1, PyBUF_SIMPLE) < 0) Index: Lib/test/regrtest.py === --- Lib/test/regrtest.py (revision 58587) +++ Lib/test/regrtest.py (working copy) @@ -1119,6 +1119,15 @@ if not os.path.supports_unicode_filenames: self.expected.add('test_pep277') +# doctest, profile and cProfile tests fail when the encoding +# of the filesystem is not built-in, because of the extra calls +# to the codecs module. +builtin_enc = ("utf-8", "latin-1", "ascii", "mbcs") +if sys.getfilesystemencoding().lower() not in builtin_enc: +self.expected.add('test_profile') +self.expected.add('test_cProfile') +self.expected.add('test_doctest') + try: from test import test_socket_ssl except ImportError: ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1286] fileinput, StringIO, and cStringIO do not support the with protocol
Alexandre Vassalotti added the comment: Do you have a use-case for this? In Py3k, I don't think adding support for the 'with' statement to StringIO makes any sense, since the close() method does nothing. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1286> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1302] Fixes for profile/cprofile
Alexandre Vassalotti added the comment: > I still don't understand why you are using (sizeof lower) - 2 It is simply to avoid duplicating the constant (a.k.a. the Don't Repeat Yourself (DRY) rule). > and what &lower[(sizeof lower) - 2] returns. Is it the memory address > of lower[17]? It the address of lower[18] to be exact. (l < &lower[(sizeof lower) - 2]) is simply tricky notation to check the bound of the array. Personally, I used like to subtract pointer, ((lower - l + 1) < (sizeof lower)) to get the bound. But now, I find Guido's trick more cute (and less error-prone). :) __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1302> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1302] Fixes for profile/cprofile
Alexandre Vassalotti added the comment: > Personally, I used like to subtract pointer, ((lower - l + 1) < > (sizeof lower)) to get the bound. Doh. I got it backward. It's (l - lower + 1), not (lower - l + 1). > But now, I find Guido's trick more cute (and less error-prone). At least, I got that right. :) __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1302> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1395] py3k: duplicated line endings when using read(1)
Alexandre Vassalotti added the comment: On 11/7/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 11/7/07, Christian Heimes <[EMAIL PROTECTED]> wrote: > > > > Christian Heimes added the comment: > > > > By the way what happened to the SoC project related to Python's new IO > > subsystem? IIRC somebody was working on a C optimization of the io lib. > > > > I think it was Alexandre Vassalotti. Is that right, Alexandre? Or am I > mixing you up? (If you ca, please respond to the bug.) I think so. My GSoC project was to merge the interface of cPickle/pickle and cStringIO/StringIO. I am still working on it, albeit slowly (my school homework is killing all my free time, right now). My work on StringIO and BytesIO is basically done; I just need to add newline translation and encoding support before it can be merged into the trunk. The cPickle/pickle merge is mostly done (i.e., all the current tests passes); I am right now in the bug-hunting phase. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1395> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1395] py3k: duplicated line endings when using read(1)
Alexandre Vassalotti added the comment: On 11/7/07, Guido van Rossum wrote: > Cool. How hard do you think it would be to extend your work on > StringIO into a translation of TextIOWrapper into C? Well, StringIO and TextIOWrapper are quite different. The only part that I could reuse, from StringIO, would be the newline translation facilities. I think that a perfect translation to C of the current Python implementation of TextIOWrapper will be burdensome (due to things like function annotations) but not too hard to do. Nevertheless, that would be neat project for me. I could start to work it by mid-December, along with the other performance enhancements I have in mind. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1395> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1338] pickling bytes?
Alexandre Vassalotti added the comment: Please assign this to me. I am planning to fix this, along a few other bugs, with my new revision of the pickle protocol. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1338> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue892902] problem with pickling newstyle class instances
Alexandre Vassalotti added the comment: Please assign this bug to me. Note that neither cPickle or pickle is able to load the stream generated by cPickle correctly: >>> g = group(None) >>> subitem(g) >>> g[0].parent is g True >>> gp = cPickle.loads(cPickle.dumps(g)) >>> gp[0].parent is gp False I don't think that will be easy to fix, but I will try to see what I can do. -- components: +Library (Lib) -Interpreter Core nosy: +alexandre.vassalotti type: -> behavior Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue892902> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue939395] cPickle.Pickler: in list mode, no way to set protocol
Alexandre Vassalotti added the comment: Please mark this bug as Won't Fix. As Tim said, the list-based interface of cPickle is unsupported and pending removal. Please use the pickle.dumps and pickle.loads functions if you want string interface to pickle. -- nosy: +alexandre.vassalotti Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue939395> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1761028] pickle - cannot unpickle circular deps with custom __hash__
Alexandre Vassalotti added the comment: Please assign this bug to me. This certainly doesn't look easy to fix. I will look into it, but I can promise that I can fix it. -- nosy: +alexandre.vassalotti type: -> behavior versions: +Python 2.6, Python 3.0 _ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1761028> _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1338] pickling bytes?
Alexandre Vassalotti added the comment: Thank you, Georg! __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1338> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue892902] problem with pickling newstyle class instances
Changes by Alexandre Vassalotti: -- assignee: -> alexandre.vassalotti Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue892902> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1761028] pickle - cannot unpickle circular deps with custom __hash__
Changes by Alexandre Vassalotti: -- assignee: -> alexandre.vassalotti _ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1761028> _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1536] pickle's documentation is severely outdated
New submission from Alexandre Vassalotti: The current documentation for the pickle module (and related modules) is outdated and should be updated. For example, the section "Pickling and unpickling extension types" [1] is not only confusing, but outright wrong. For Python 2.6, the documentation of cPickle should be updated to reflect the differences between the cPickle and pickle.py modules. It would be also be appropriate to note that cPickle is pending removal and should not be used anymore. [1]: http://docs.python.org/dev/library/pickle.html#pickling-and-unpickling-extension-types -- assignee: alexandre.vassalotti components: Documentation messages: 58063 nosy: alexandre.vassalotti priority: normal severity: normal status: open title: pickle's documentation is severely outdated versions: Python 2.6, Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1536> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue956303] Update pickle docs to describe format of persistent IDs
Alexandre Vassalotti added the comment: This should be fixed along issue1536. -- assignee: -> alexandre.vassalotti dependencies: +pickle's documentation is severely outdated nosy: +alexandre.vassalotti Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue956303> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue655802] cPickle not always same as pickle
Alexandre Vassalotti added the comment: This should be fixed along with issue1536. I am assigning this to me, instead of Fred Drake. (Is that okay with you, Fred?) -- assignee: fdrake -> alexandre.vassalotti dependencies: +pickle's documentation is severely outdated nosy: +alexandre.vassalotti versions: +Python 2.5, Python 2.6 Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue655802> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1536] pickle's documentation is severely outdated
Alexandre Vassalotti added the comment: [I just realized that dependencies in the tracker should go the other way around.] -- dependencies: +Update pickle docs to describe format of persistent IDs, cPickle not always same as pickle __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1536> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue655802] cPickle not always same as pickle
Changes by Alexandre Vassalotti: -- dependencies: -pickle's documentation is severely outdated Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue655802> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue956303] Update pickle docs to describe format of persistent IDs
Changes by Alexandre Vassalotti: -- dependencies: -pickle's documentation is severely outdated Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue956303> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1283] PyBytes (buffer) .extend method needs to accept any iterable of ints
Alexandre Vassalotti added the comment: Here a patch that adds support for any iterable (or sequence) of integers to bytearray.extend(). -- assignee: -> alexandre.vassalotti keywords: +patch nosy: +alexandre.vassalotti resolution: accepted -> Added file: http://bugs.python.org/file8847/byte_extend.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1283> __Index: Objects/bytesobject.c === --- Objects/bytesobject.c (revision 59259) +++ Objects/bytesobject.c (working copy) @@ -2487,24 +2487,6 @@ return NULL; } -PyDoc_STRVAR(extend__doc__, -"B.extend(iterable int) -> None\n\ -\n\ -Append all the elements from the iterator or sequence to the\n\ -end of B."); -static PyObject * -bytes_extend(PyBytesObject *self, PyObject *arg) -{ -/* XXX(gps): The docstring says any iterable int will do but the - * bytes_setslice code only accepts something supporting PEP 3118. - * A list or tuple of 0 <= int <= 255 is supposed to work. */ -/* bug being tracked on: http://bugs.python.org/issue1283 */ -if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) -return NULL; -Py_RETURN_NONE; -} - - PyDoc_STRVAR(reverse__doc__, "B.reverse() -> None\n\ \n\ @@ -2591,6 +2573,66 @@ Py_RETURN_NONE; } +PyDoc_STRVAR(extend__doc__, +"B.extend(iterable int) -> None\n\ +\n\ +Append all the elements from the iterator or sequence to the\n\ +end of B."); +static PyObject * +bytes_extend(PyBytesObject *self, PyObject *arg) +{ +PyObject *it, *item, *tmp, *res; +Py_ssize_t len_hint, buf_size = 0, len = 0; +int value; +char *buf; + +/* bytes_setslice code only accepts something supporting PEP 3118. */ +if (PyObject_CheckBuffer(arg)) { +if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) +return NULL; + +Py_RETURN_NONE; +} + +it = PyObject_GetIter(arg); +if (it == NULL) +return NULL; + +len_hint = _PyObject_LengthHint(arg); +if (len_hint != -1) { +buf_size = len_hint; +} +else { +PyErr_Clear(); +buf_size = 32; +} + +buf = (char *)PyMem_Malloc(buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); + +while ((item = PyIter_Next(it)) != NULL) { +if (! _getbytevalue(item, &value)) +return NULL; +buf[len++] = value; +Py_DECREF(item); +if (len >= buf_size) { +buf_size = len + (len >> 1); +buf = PyMem_Realloc(buf, buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); +} +} +Py_DECREF(it); + +tmp = PyBytes_FromStringAndSize(buf, len); +res = bytes_extend(self, tmp); +Py_DECREF(tmp); +PyMem_Free(buf); + +return res; +} + PyDoc_STRVAR(pop__doc__, "B.pop([index]) -> int\n\ \n\ Index: Lib/test/test_bytes.py === --- Lib/test/test_bytes.py (revision 59259) +++ Lib/test/test_bytes.py (working copy) @@ -529,6 +529,21 @@ a.extend(a) self.assertEqual(a, orig + orig) self.assertEqual(a[5:], orig) +a = bytearray(b'') +a.extend(map(int, orig * 50)) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(iter(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(list(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +self.assertRaises(ValueError, a.extend, [0, 1, 2, 3, 256]) +self.assertEqual(len(a), 0) def test_remove(self): b = bytearray(b'hello') ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1283] PyBytes (buffer) .extend method needs to accept any iterable of ints
Alexandre Vassalotti added the comment: Made 2 minor enhancements to the patch: + Added the proper type-cast to PyMem_Realloc call. + Changed (len >> 1) to (len >> 1) + 1, just to be sure that the buffer doesn't overflow if __length_hint__ return 0 or 1 erroneously. Added file: http://bugs.python.org/file8848/byte_extend-2.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1283> __Index: Objects/bytesobject.c === --- Objects/bytesobject.c (revision 59259) +++ Objects/bytesobject.c (working copy) @@ -2487,24 +2487,6 @@ return NULL; } -PyDoc_STRVAR(extend__doc__, -"B.extend(iterable int) -> None\n\ -\n\ -Append all the elements from the iterator or sequence to the\n\ -end of B."); -static PyObject * -bytes_extend(PyBytesObject *self, PyObject *arg) -{ -/* XXX(gps): The docstring says any iterable int will do but the - * bytes_setslice code only accepts something supporting PEP 3118. - * A list or tuple of 0 <= int <= 255 is supposed to work. */ -/* bug being tracked on: http://bugs.python.org/issue1283 */ -if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) -return NULL; -Py_RETURN_NONE; -} - - PyDoc_STRVAR(reverse__doc__, "B.reverse() -> None\n\ \n\ @@ -2591,6 +2573,66 @@ Py_RETURN_NONE; } +PyDoc_STRVAR(extend__doc__, +"B.extend(iterable int) -> None\n\ +\n\ +Append all the elements from the iterator or sequence to the\n\ +end of B."); +static PyObject * +bytes_extend(PyBytesObject *self, PyObject *arg) +{ +PyObject *it, *item, *tmp, *res; +Py_ssize_t len_hint, buf_size = 0, len = 0; +int value; +char *buf; + +/* bytes_setslice code only accepts something supporting PEP 3118. */ +if (PyObject_CheckBuffer(arg)) { +if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) +return NULL; + +Py_RETURN_NONE; +} + +it = PyObject_GetIter(arg); +if (it == NULL) +return NULL; + +len_hint = _PyObject_LengthHint(arg); +if (len_hint != -1) { +buf_size = len_hint; +} +else { +PyErr_Clear(); +buf_size = 32; +} + +buf = (char *)PyMem_Malloc(buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); + +while ((item = PyIter_Next(it)) != NULL) { +if (! _getbytevalue(item, &value)) +return NULL; +buf[len++] = value; +Py_DECREF(item); +if (len >= buf_size) { +buf_size = len + (len >> 1) + 1; +buf = (char *)PyMem_Realloc(buf, buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); +} +} +Py_DECREF(it); + +tmp = PyBytes_FromStringAndSize(buf, len); +res = bytes_extend(self, tmp); +Py_DECREF(tmp); +PyMem_Free(buf); + +return res; +} + PyDoc_STRVAR(pop__doc__, "B.pop([index]) -> int\n\ \n\ Index: Lib/test/test_bytes.py === --- Lib/test/test_bytes.py (revision 59259) +++ Lib/test/test_bytes.py (working copy) @@ -529,6 +529,21 @@ a.extend(a) self.assertEqual(a, orig + orig) self.assertEqual(a[5:], orig) +a = bytearray(b'') +a.extend(map(int, orig * 50)) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(iter(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(list(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +self.assertRaises(ValueError, a.extend, [0, 1, 2, 3, 256]) +self.assertEqual(len(a), 0) def test_remove(self): b = bytearray(b'hello') ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1283] PyBytes (buffer) .extend method needs to accept any iterable of ints
Alexandre Vassalotti added the comment: There is a reference leak in my previous patches. So, I updated (again) the patch. There is still another possible leak if the PyMem_Realloc return NULL (i.e., the system is out of memory), but I don't think it worth fixing. Added file: http://bugs.python.org/file8849/byte_extend-3.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1283> __Index: Objects/bytesobject.c === --- Objects/bytesobject.c (revision 59259) +++ Objects/bytesobject.c (working copy) @@ -2487,24 +2487,6 @@ return NULL; } -PyDoc_STRVAR(extend__doc__, -"B.extend(iterable int) -> None\n\ -\n\ -Append all the elements from the iterator or sequence to the\n\ -end of B."); -static PyObject * -bytes_extend(PyBytesObject *self, PyObject *arg) -{ -/* XXX(gps): The docstring says any iterable int will do but the - * bytes_setslice code only accepts something supporting PEP 3118. - * A list or tuple of 0 <= int <= 255 is supposed to work. */ -/* bug being tracked on: http://bugs.python.org/issue1283 */ -if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) -return NULL; -Py_RETURN_NONE; -} - - PyDoc_STRVAR(reverse__doc__, "B.reverse() -> None\n\ \n\ @@ -2591,6 +2573,69 @@ Py_RETURN_NONE; } +PyDoc_STRVAR(extend__doc__, +"B.extend(iterable int) -> None\n\ +\n\ +Append all the elements from the iterator or sequence to the\n\ +end of B."); +static PyObject * +bytes_extend(PyBytesObject *self, PyObject *arg) +{ +PyObject *it, *item, *tmp, *res; +Py_ssize_t len_hint, buf_size = 0, len = 0; +int value; +char *buf; + +/* bytes_setslice code only accepts something supporting PEP 3118. */ +if (PyObject_CheckBuffer(arg)) { +if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) +return NULL; + +Py_RETURN_NONE; +} + +it = PyObject_GetIter(arg); +if (it == NULL) +return NULL; + +len_hint = _PyObject_LengthHint(arg); +if (len_hint != -1) { +buf_size = len_hint; +} +else { +PyErr_Clear(); +buf_size = 32; +} + +buf = (char *)PyMem_Malloc(buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); + +while ((item = PyIter_Next(it)) != NULL) { +if (! _getbytevalue(item, &value)) { +Py_DECREF(item); +Py_DECREF(it); +return NULL; +} +buf[len++] = value; +Py_DECREF(item); +if (len >= buf_size) { +buf_size = len + (len >> 1) + 1; +buf = (char *)PyMem_Realloc(buf, buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); +} +} +Py_DECREF(it); + +tmp = PyBytes_FromStringAndSize(buf, len); +res = bytes_extend(self, tmp); +Py_DECREF(tmp); +PyMem_Free(buf); + +return res; +} + PyDoc_STRVAR(pop__doc__, "B.pop([index]) -> int\n\ \n\ Index: Lib/test/test_bytes.py === --- Lib/test/test_bytes.py (revision 59259) +++ Lib/test/test_bytes.py (working copy) @@ -529,6 +529,21 @@ a.extend(a) self.assertEqual(a, orig + orig) self.assertEqual(a[5:], orig) +a = bytearray(b'') +a.extend(map(int, orig * 50)) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(iter(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(list(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +self.assertRaises(ValueError, a.extend, [0, 1, 2, 3, 256]) +self.assertEqual(len(a), 0) def test_remove(self): b = bytearray(b'hello') ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1283] PyBytes (buffer) .extend method needs to accept any iterable of ints
Alexandre Vassalotti added the comment: Here yet another revision of the patch. This one makes bytearray.extend() try to determine the length of its argument a bit more aggressively -- i.e., also uses PyObject_Length(). Added file: http://bugs.python.org/file8856/byte_extend-4.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1283> __Index: Objects/bytesobject.c === --- Objects/bytesobject.c (revision 59259) +++ Objects/bytesobject.c (working copy) @@ -2487,24 +2487,6 @@ return NULL; } -PyDoc_STRVAR(extend__doc__, -"B.extend(iterable int) -> None\n\ -\n\ -Append all the elements from the iterator or sequence to the\n\ -end of B."); -static PyObject * -bytes_extend(PyBytesObject *self, PyObject *arg) -{ -/* XXX(gps): The docstring says any iterable int will do but the - * bytes_setslice code only accepts something supporting PEP 3118. - * A list or tuple of 0 <= int <= 255 is supposed to work. */ -/* bug being tracked on: http://bugs.python.org/issue1283 */ -if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) -return NULL; -Py_RETURN_NONE; -} - - PyDoc_STRVAR(reverse__doc__, "B.reverse() -> None\n\ \n\ @@ -2591,6 +2573,70 @@ Py_RETURN_NONE; } +PyDoc_STRVAR(extend__doc__, +"B.extend(iterable int) -> None\n\ +\n\ +Append all the elements from the iterator or sequence to the\n\ +end of B."); +static PyObject * +bytes_extend(PyBytesObject *self, PyObject *arg) +{ +PyObject *it, *item, *tmp, *res; +Py_ssize_t buf_size = 0, len = 0; +int value; +char *buf; + +/* bytes_setslice code only accepts something supporting PEP 3118. */ +if (PyObject_CheckBuffer(arg)) { +if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) +return NULL; + +Py_RETURN_NONE; +} + +it = PyObject_GetIter(arg); +if (it == NULL) +return NULL; + +/* Try to determine the length of the argument. */ +buf_size = PyObject_Length(arg); +if (buf_size < 0) +buf_size = _PyObject_LengthHint(arg); +if (buf_size < 0) { +/* The length of the argument is unknown. */ +PyErr_Clear(); +buf_size = 32; +} + +buf = (char *)PyMem_Malloc(buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); + +while ((item = PyIter_Next(it)) != NULL) { +if (! _getbytevalue(item, &value)) { +Py_DECREF(item); +Py_DECREF(it); +return NULL; +} +buf[len++] = value; +Py_DECREF(item); +if (len >= buf_size) { +buf_size = len + (len >> 1) + 1; +buf = (char *)PyMem_Realloc(buf, buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); +} +} +Py_DECREF(it); + +tmp = PyBytes_FromStringAndSize(buf, len); +res = bytes_extend(self, tmp); +Py_DECREF(tmp); +PyMem_Free(buf); + +return res; +} + PyDoc_STRVAR(pop__doc__, "B.pop([index]) -> int\n\ \n\ Index: Lib/test/test_bytes.py === --- Lib/test/test_bytes.py (revision 59259) +++ Lib/test/test_bytes.py (working copy) @@ -529,6 +529,21 @@ a.extend(a) self.assertEqual(a, orig + orig) self.assertEqual(a[5:], orig) +a = bytearray(b'') +a.extend(map(int, orig * 50)) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(iter(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(list(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +self.assertRaises(ValueError, a.extend, [0, 1, 2, 3, 256]) +self.assertEqual(len(a), 0) def test_remove(self): b = bytearray(b'hello') ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1283] PyBytes (buffer) .extend method needs to accept any iterable of ints
Alexandre Vassalotti added the comment: Done. Is there any other issue with the patch? Added file: http://bugs.python.org/file8857/byte_extend-5.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1283> __Index: Objects/bytesobject.c === --- Objects/bytesobject.c (revision 59259) +++ Objects/bytesobject.c (working copy) @@ -2487,24 +2487,6 @@ return NULL; } -PyDoc_STRVAR(extend__doc__, -"B.extend(iterable int) -> None\n\ -\n\ -Append all the elements from the iterator or sequence to the\n\ -end of B."); -static PyObject * -bytes_extend(PyBytesObject *self, PyObject *arg) -{ -/* XXX(gps): The docstring says any iterable int will do but the - * bytes_setslice code only accepts something supporting PEP 3118. - * A list or tuple of 0 <= int <= 255 is supposed to work. */ -/* bug being tracked on: http://bugs.python.org/issue1283 */ -if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) -return NULL; -Py_RETURN_NONE; -} - - PyDoc_STRVAR(reverse__doc__, "B.reverse() -> None\n\ \n\ @@ -2591,6 +2573,72 @@ Py_RETURN_NONE; } +PyDoc_STRVAR(extend__doc__, +"B.extend(iterable int) -> None\n\ +\n\ +Append all the elements from the iterator or sequence to the\n\ +end of B."); +static PyObject * +bytes_extend(PyBytesObject *self, PyObject *arg) +{ +PyObject *it, *item, *tmp, *res; +Py_ssize_t buf_size = 0, len = 0; +int value; +char *buf; + +/* bytes_setslice code only accepts something supporting PEP 3118. */ +if (PyObject_CheckBuffer(arg)) { +if (bytes_setslice(self, Py_Size(self), Py_Size(self), arg) == -1) +return NULL; + +Py_RETURN_NONE; +} + +it = PyObject_GetIter(arg); +if (it == NULL) +return NULL; + +/* Try to determine the length of the argument. */ +buf_size = PyObject_Length(arg); +if (buf_size < 0) +buf_size = _PyObject_LengthHint(arg); +if (buf_size < 0) { +/* The length of the argument is unknown. */ +PyErr_Clear(); +buf_size = 32; +} + +buf = (char *)PyMem_Malloc(buf_size * sizeof(char)); +if (buf == NULL) +return PyErr_NoMemory(); + +while ((item = PyIter_Next(it)) != NULL) { +if (! _getbytevalue(item, &value)) { +Py_DECREF(item); +Py_DECREF(it); +return NULL; +} +buf[len++] = value; +Py_DECREF(item); +if (len >= buf_size) { +buf_size = len + (len >> 1) + 1; +buf = (char *)PyMem_Realloc(buf, buf_size * sizeof(char)); +if (buf == NULL) { +Py_DECREF(it); +return PyErr_NoMemory(); +} +} +} +Py_DECREF(it); + +tmp = PyBytes_FromStringAndSize(buf, len); +res = bytes_extend(self, tmp); +Py_DECREF(tmp); +PyMem_Free(buf); + +return res; +} + PyDoc_STRVAR(pop__doc__, "B.pop([index]) -> int\n\ \n\ Index: Lib/test/test_bytes.py === --- Lib/test/test_bytes.py (revision 59259) +++ Lib/test/test_bytes.py (working copy) @@ -529,6 +529,21 @@ a.extend(a) self.assertEqual(a, orig + orig) self.assertEqual(a[5:], orig) +a = bytearray(b'') +a.extend(map(int, orig * 50)) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(iter(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +a.extend(list(map(int, orig * 50))) +self.assertEqual(a, orig * 50) +self.assertEqual(a[-5:], orig) +a = bytearray(b'') +self.assertRaises(ValueError, a.extend, [0, 1, 2, 3, 256]) +self.assertEqual(len(a), 0) def test_remove(self): b = bytearray(b'hello') ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1294] Management of KeyboardInterrupt in cmd.py
Alexandre Vassalotti added the comment: First, I would like to say thank you both for spending your time trying to do a contribution to Python. However, I believe the current behavior of cmd.py is correct. The module documentation states clearly that "End of file on input is processed as the command 'EOF'." For the KeyboardInterrupt issue, it thinks the exception should be handled by the application with a try-statement. For example, >>> import sys, cmd >>> c = cmd.Cmd() >>> try: ... c.cmdloop() ... except KeyboardInterrupt: ... print "\nGot keyboard interrupt. Exiting..." ... sys.exit(0) ... (Cmd) ^C Got keyboard interrupt. Exiting... I am closing and marking this bug as rejected. If you feel this is inappropriate, please request with an appropriate explanation to reopen it. -- components: +Library (Lib) -None nosy: +alexandre.vassalotti priority: -> low resolution: -> rejected status: open -> closed type: behavior -> rfe __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1294> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1005] Patches to rename Queue module to queue
Alexandre Vassalotti added the comment: Thank you, Paul, for the patches. I reviewed the patches and improved them. Here the list of changes: - Added unit test for the import fixers. - Renamed Lib/Queue.py to Lib/queue.py. - Updated Tools/webchecker/wsgui.py. - Updated the documentation. Hopefully, the patches won't get out of date by the time we do the library reorganization. -- assignee: -> brett.cannon keywords: +py3k nosy: +alexandre.vassalotti, brett.cannon versions: +Python 3.0 -Python 2.6 Added file: http://bugs.python.org/file8865/queue-import-fixer.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1005> __Index: tests/test_fixers.py === --- tests/test_fixers.py (revision 59312) +++ tests/test_fixers.py (working copy) @@ -1273,6 +1273,7 @@ "cStringIO": ("io", ["StringIO"]), "__builtin__" : ("builtins", ["open", "Exception", "__debug__", "str"]), + "Queue": ("queue", ["Empty", "Full", "Queue"]), } def test_import_module(self): Index: fixes/fix_imports.py === --- fixes/fix_imports.py (revision 59312) +++ fixes/fix_imports.py (working copy) @@ -4,6 +4,7 @@ * StringIO -> io * cStringIO -> io * md5 -> hashlib + * Queue -> queue """ # Author: Collin Winter @@ -16,7 +17,8 @@ MAPPING = {"StringIO": ("io", ["StringIO"]), "cStringIO": ("io", ["StringIO"]), - "__builtin__" : ("builtins", builtin_names), + "__builtin__": ("builtins", builtin_names), + "Queue": ("queue", ["Empty", "Full", "Queue"]), } ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1005] Patches to rename Queue module to queue
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file8868/queue-rename-misc.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1005> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1005] Patches to rename Queue module to queue
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file8867/queue-rename-lib-and-tests.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1005> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1005] Patches to rename Queue module to queue
Changes by Alexandre Vassalotti: -- assignee: collinwinter -> brett.cannon nosy: -collinwinter __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1005> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1005] Patches to rename Queue module to queue
Changes by Alexandre Vassalotti: -- assignee: brett.cannon -> collinwinter nosy: +collinwinter Added file: http://bugs.python.org/file8866/queue-rename-documentation.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1005> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1214] Timeout in CGIXMLRPCRequestHandler under IIS
Alexandre Vassalotti added the comment: Looks good to me. I updated the patch to use .get() with a default value instead of a if-statement with .has_key(). -- nosy: +alexandre.vassalotti priority: -> normal Added file: http://bugs.python.org/file8869/check-content-length.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1214> __Index: Lib/SimpleXMLRPCServer.py === --- Lib/SimpleXMLRPCServer.py (revision 59313) +++ Lib/SimpleXMLRPCServer.py (working copy) @@ -599,7 +599,8 @@ else: # POST data is normally available through stdin if request_text is None: -request_text = sys.stdin.read() +content_length = os.environ.get('CONTENT_LENGTH', -1) +request_text = sys.stdin.read(content_length) self.handle_xmlrpc(request_text) ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553] An errornous __length_hint__ can make list() raise a SystemError
New submission from Alexandre Vassalotti: If an iterator with a __length_hint__ method that returns a negative integer is passed to list(), a SystemError is raised. >>> class A: ... def __iter__(self): ... return self ... def __length_hint__(self): ... return -1 ... >>> list(A()) Traceback (most recent call last): File "", line 1, in SystemError: NULL result without error in PyObject_Call -- messages: 58176 nosy: alexandre.vassalotti priority: normal severity: normal status: open title: An errornous __length_hint__ can make list() raise a SystemError type: behavior versions: Python 2.5, Python 2.6, Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1553> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1283] PyBytes (buffer) .extend method needs to accept any iterable of ints
Alexandre Vassalotti added the comment: Thank you Gregory for the review! Committed to r59314. -- resolution: -> accepted status: open -> closed __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1283> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553] An errornous __length_hint__ can make list() raise a SystemError
Alexandre Vassalotti added the comment: Fixed for Py3k in r59316. Should this gets backported? -- status: open -> __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1553> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553] An errornous __length_hint__ can make list() raise a SystemError
Changes by Alexandre Vassalotti: -- status: -> open __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1553> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue614555] Rewrite _reduce and _reconstructor in C
Changes by Alexandre Vassalotti: -- nosy: +alexandre.vassalotti Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue614555> ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1266570] PEP 349: allow str() to return unicode
Alexandre Vassalotti added the comment: The PEP has been deferred and the patch is out of date. So, is this change still wanted? -- nosy: +alexandre.vassalotti resolution: -> out of date _ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1266570> _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1530] doctest should return error if not all tests passed
Alexandre Vassalotti added the comment: Committed in r59411. Thanks! -- resolution: -> accepted status: open -> closed __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1530> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1530] doctest should return error if not all tests passed
Alexandre Vassalotti added the comment: Looks good to me. Here's slightly modified patch ready to be committed. -- keywords: +patch nosy: +alexandre.vassalotti priority: -> low Added file: http://bugs.python.org/file8893/doctest.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1530> __Index: Lib/doctest.py === --- Lib/doctest.py (revision 59409) +++ Lib/doctest.py (working copy) @@ -2657,12 +2657,15 @@ sys.path.insert(0, dirname) m = __import__(filename[:-3]) del sys.path[0] -testmod(m) +failures, _ = testmod(m) else: -testfile(filename, module_relative=False) +failures, _ = testfile(filename, module_relative=False) +if failures: +return 1 else: r = unittest.TextTestRunner() r.run(DocTestSuite()) +return 0 if __name__ == "__main__": -_test() +sys.exit(_test()) ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1573] Improper use of the keyword-only syntax makes the parser crash
Changes by Alexandre Vassalotti: -- versions: +Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1573> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1573] Improper use of the keyword-only syntax makes the parser crash
New submission from Alexandre Vassalotti: I found that the parser fails to handle correctly the (incorrect) case where the single-star (*), used for delimiting keyword-only arguments, is immediately followed by a **keywords parameter: >>> def f(*, **kw): ... pass ... python: Python/ast.c:652: handle_keywordonly_args: Assertion `kwonlyargs != ((void *)0)' failed. [1]7872 abort (core dumped) ./python -- components: Interpreter Core keywords: py3k messages: 58299 nosy: alexandre.vassalotti severity: normal status: open title: Improper use of the keyword-only syntax makes the parser crash type: crash __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1573> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1573] Improper use of the keyword-only syntax makes the parser crash
Alexandre Vassalotti added the comment: Amaury is right. "def f(*, **kw): pass" should raise a SyntaxError. The keyword-only delimiter is useless since the **kw parameter already only accepts keywords. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1573> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1596] Broken pipes should be handled better in 2.x
New submission from Alexandre Vassalotti: I think Python 2.x should mimic, or improve on, the behavior of Py3k for handling broken pipes. That is: 1. Don't print the message "close failed: [Errno 32] Broken pipe", from filemodule.c; since this is impossible to override from Python. 2. Check sys.stdout if it's closed before calling the print routine. Raise an IOError if so. Finally, here's some examples to illustrate what I am saying: alex:~% python -c "print" | head -n0 close failed: [Errno 32] Broken pipe alex:~% python -c "while 1: print" | head -n0 [The loop continues running, even if the output stream is closed] ^C Traceback (most recent call last): File "", line 1, in KeyboardInterrupt close failed: [Errno 32] Broken pipe alex:~% py3k -c "print()" | head -n0 alex:~% py3k -c "while 1: print()" | head -n0 Traceback (most recent call last): ... IOError: [Errno 32] Broken pipe -- components: Interpreter Core messages: 58469 nosy: alexandre.vassalotti severity: normal status: open title: Broken pipes should be handled better in 2.x type: rfe versions: Python 2.5, Python 2.6 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1596> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1313119] urlparse "caches" parses regardless of encoding
Alexandre Vassalotti added the comment: Fixed in r59480. -- nosy: +alexandre.vassalotti resolution: -> fixed status: open -> closed _ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1313119> _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1333] merge urllib and urlparse functionality
Changes by Alexandre Vassalotti: -- resolution: accepted -> __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1333> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1621] Python should compile with -Wstrict-overflow when using gcc
Alexandre Vassalotti added the comment: I compiled Python using gcc 4.3.0 with the -Wstrict-overflow, and that's the only warning I got: Objects/doubledigits.c: In function ‘_PyFloat_Digits’: Objects/doubledigits.c:313: error: assuming signed overflow does not occur when assuming that (X + c) < X is always false I am sure yet how to interpret it, though. It says that the overflow check is in _PyFloat_Digits(), line 313 is in the function add_big(). It probably means that add_big() gets inlined. I tried to set -finline-limit=0, but strangely the overflow warning disappears... I will try to investigate this further, when I will have a bit more time in my hands. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1621> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
New submission from Alexandre Vassalotti: Finally, here is my C implementation of BytesIO. The code is well tested and include the proper unit tests. The only remaining issues are: - The behavior of the close() method. - The failure of test_profile and test_cProfile. Currently, I have no idea how to fix the tests for the profilers. The weird thing is both pass when run with: % ./python Lib/test/regrtest.py -R: test_profile -- components: Interpreter Core, Library (Lib), Tests files: _bytesio.c keywords: patch messages: 59436 nosy: alexandre.vassalotti, gvanrossum priority: normal severity: normal status: open title: Fast BytesIO implementation + misc changes versions: Python 3.0 Added file: http://bugs.python.org/file9084/_bytesio.c __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file9085/add-bytesio-setup.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file9086/swap-initstdio-initsite.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file9087/test_memoryio.py __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file9088/remove-old-stringio-test.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file9089/truncate-semantic-change.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: Added file: http://bugs.python.org/file9090/io-misc-fixes.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: -- nosy: +brett.cannon __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Alexandre Vassalotti added the comment: So, here's one big patch. I have updated the behavior of close(), so that > The profile tests often fail when io.py changes because they happen to > depend on "golden output" which includes line numbers of code in io.py > that happens to be traced during the test. Yes, I knew that. But, how can I fix the test so that it passes even if _bytesio is not available? Oh, one more thing. In the misc fixes for io.py, I added a checkClosed in IOBase.readline(). As a side-effect, this make __next__ raises a ValueError, instead of StopIteration. Is that correct? >>> f = open("load.py") [45681 refs] >>> next(f) 'import sys\n' [45700 refs] >>> f.close() [45703 refs] >>> next(f) Traceback (most recent call last): File "", line 1, in File "/home/alex/src/python.org/py3k/Lib/io.py", line 1440, in __next__ line = self.readline() File "/home/alex/src/python.org/py3k/Lib/io.py", line 1449, in readline raise ValueError("read from closed file") ValueError: read from closed file [45703 refs] Added file: http://bugs.python.org/file9096/bytesio+misc-fixes.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Alexandre Vassalotti added the comment: [grrr, I eat my words] > So, here's one big patch. I have updated the behavior of close(), so that ... it matches the behavior of 2.x. > As a side-effect, this make __next__ raises a ValueError, instead of StopIteration. ... when the file is closed. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1753] TextIOWrapper.write writes utf BOM for every string
Alexandre Vassalotti added the comment: Committed fix in r59832. Thanks! P.S. Guido, what this comment, in write(), is about? # XXX What if we were just reading? __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1753> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1753] TextIOWrapper.write writes utf BOM for every string
Changes by Alexandre Vassalotti: -- resolution: -> fixed __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1753> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1753] TextIOWrapper.write writes utf BOM for every string
Changes by Alexandre Vassalotti: -- status: open -> closed __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1753> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Changes by Alexandre Vassalotti: Removed file: http://bugs.python.org/file9096/bytesio+misc-fixes.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1751] Fast BytesIO implementation + misc changes
Alexandre Vassalotti added the comment: I got a patch that also fixes the profiler tests. That was easy after all. :-) Added file: http://bugs.python.org/file9099/bytesio+misc-fixes-2.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1751> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1621] Do not assume signed integer overflow behavior
Alexandre Vassalotti added the comment: Hm. I don't get any warning, related to the overflow issue, neither with -Wstrict-overflow=3, nor -Wstrict-overflow=5. Are the cPickle warnings already fixed? __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1621> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1286] fileinput, StringIO, and cStringIO do not support the with protocol
Alexandre Vassalotti added the comment: FYI, StringIO and BytesIO, in Python 3K, already support the context management protocol. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1286> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1817] module-cgi: handling GET and POST together
New submission from Alexandre Fiori: It looks like module-cgi cannot handle GET and POST together when using FieldStorage. For instance, a is available through cgi.FormContent but not cgi.FieldStorage when there are other in the html form. Very strange. -- messages: 59848 nosy: alef13 severity: normal status: open title: module-cgi: handling GET and POST together versions: Python 2.5 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1817> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1817] module-cgi: handling GET and POST together
Alexandre Fiori added the comment: Here it is, babe. I've made appropriate changes to make it read QUERY_STRING when it's available within POST method. It's currently being parsed by parse_qsl() in read_urlencoded() as it should be in regular GET or POST. I didn't touch CONTENT_LENGTH since it has the size of the POST data without QUERY_STRING's size. That would affect read() of such POST data. It's working. Thanks for the incentive, it's my first patch for python. Added file: http://bugs.python.org/file9148/cgi.py.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1817> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1817] module-cgi: handling GET and POST together
Alexandre Fiori added the comment: Now I added support for parsing query string within multipart/*. The only issue now is that it mixes FieldStorage with MiniFieldStorage. However, I don't think it's a problem. Added file: http://bugs.python.org/file9180/cgy.py.patch-1 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1817> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6551] test_codecs fails when ran after test_zipimport and test_mailbox.
Alexandre Vassalotti added the comment: The patch looks good to me. -- resolution: -> accepted ___ Python tracker <http://bugs.python.org/issue6551> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1023290] Conversion of longs to bytes and vice-versa.
Alexandre Vassalotti added the comment: Here's an updated patch. - Renamed tobytes() to to_bytes() and frombytes() to from_bytes(). - Moved the changes to pickle to a different patch. - Made the NULL-checks more consistent with the rest of long's code. - Fixed the type check of the `length' parameter of to_bytes() to use PyIndex_Check() instead of PyLong_Check(). -- dependencies: +Move the special-case for integer objects out of PyBytes_FromObject. Added file: http://bugs.python.org/file15333/long_and_bytes_conversion-3.diff ___ Python tracker <http://bugs.python.org/issue1023290> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5683] Speed up cPickle's pickling generally
Alexandre Vassalotti added the comment: Last august, I worked on integrating Collin's optimization work into py3k in a local Mercurial branch. So, I can champion these changes into py3k, if Collin is unavailable. And if Collin allows me, I would like to merge the other pickle optimizations currently in Unladen Swallow. -- ___ Python tracker <http://bugs.python.org/issue5683> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7128] cPickle looking for non-existent package copyreg
Alexandre Vassalotti added the comment: But, that change was undone already, no? In fact, it is you, Georg, that reverted my mess in r63493. ;-) -- nosy: +alexandre.vassalotti ___ Python tracker <http://bugs.python.org/issue7128> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7128] cPickle looking for non-existent package copyreg
Alexandre Vassalotti added the comment: Yes. But, I would remove the ugly copy_reg_str variable from cPickle as the patch. -- Added file: http://bugs.python.org/file15377/remove_copyreg_str.diff ___ Python tracker <http://bugs.python.org/issue7128> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com