[issue28147] Memory leak in new 3.6 dictionary resize

INADA Naoki added the comment: Ah, is the leak happen in 3.6b1? dict.pop() in 3.6b1 doesn't combine split table. It was a bug and fixed in master branch already. Regardless the leak, I agree that grow dict when popping is bad idea. I'll improve your patch. --

[issue28126] Py_MEMCPY: Use memcpy on Windows?

Christian Heimes added the comment: My change doesn't break or change 3rd party application. They might get a tiny bit faster on Windows, if they have used Py_MEMCPY() for up to 16 bytes. That's it. -- ___ Python tracker

[issue28126] Py_MEMCPY: Use memcpy on Windows?

STINNER Victor added the comment: I don't think that it's very useful to discuss the status of the macro. Documenting the change doesn't harm ;-) -- ___ Python tracker

[issue28147] Memory leak in new 3.6 dictionary resize

STINNER Victor added the comment: The CPython test suite uses a counter on memory allocations. Please add an unit test which triggers the memory leak, but you don't need many iterations. One iteartion should be enough to be catched by the unit test. Try: ./python -m test -R 3:3 test_dict.

[issue28126] Py_MEMCPY: Use memcpy on Windows?

Serhiy Storchaka added the comment: The Py_MEMCPY name doesn't start by an underscore, and it is not inside the "#ifndef Py_LIMITED_API" block. But it is not documented either. I don't know what is the status of this macro. If this is a part of public API, this change should be documented in

[issue28127] Add _PyDict_CheckConsistency()

STINNER Victor added the comment: Ok, I pushed my new function. Expensive checks are disabled by default: define DEBUG_PYDICT to enable them (ex: gcc -D DEBUG_PYDICT). Thanks for the review Eric, Naoki & Xiang! -- resolution: -> fixed status: open -> closed

[issue28127] Add _PyDict_CheckConsistency()

Roundup Robot added the comment: New changeset ee44c971b3af by Victor Stinner in branch '3.6': Add _PyDict_CheckConsistency() https://hg.python.org/cpython/rev/ee44c971b3af New changeset 070cc3b9d5cc by Victor Stinner in branch 'default': Merge 3.6 https://hg.python.org/cpython/rev/070cc3b9d5cc

[issue28114] parse_envlist(): os.execve(), os.spawnve(), etc. crash in Python 3.6.0 when env contains byte strings

STINNER Victor added the comment: Berker and/or Eryksun: Please write unit tests for your patch. I just added a new SpawnTests to test_os which tests all os.spawn*() functions. Please add at least one unit test with env contains a bytes key/value entry. Maybe add a use_bytes=False parameter

[issue28114] parse_envlist(): os.execve(), os.spawnve(), etc. crash in Python 3.6.0 when env contains byte strings

Roundup Robot added the comment: New changeset 0ca42273c714 by Victor Stinner in branch '3.6': Issue #28114: Add unit tests on os.spawn*() https://hg.python.org/cpython/rev/0ca42273c714 -- nosy: +python-dev ___ Python tracker

[issue28147] Memory leak in new 3.6 dictionary resize

INADA Naoki added the comment: I don't understand the leak yet. > Each time dict_resize is called, it gets a new, larger size `> minused`. If > this is triggered many times, it will keep growing in size by a factor of two > each time, as the previous size is passed as minused for the next

[issue28145] Fix whitespace in C source code

Martin Panter added the comment: What is the reasoning behind this? It seems like trading one person’s style, fashion, or editor settings for another. I think it is better to just tolerate existing styles, unless they cause significant problems. But maybe see what other people think. The

[issue28126] Py_MEMCPY: Use memcpy on Windows?

Christian Heimes added the comment: Isn't the C API section reserved for C API changes? I haven't changed the C API but rather optimized the core. -- ___ Python tracker

[issue28147] Memory leak in new 3.6 dictionary resize

Min RK added the comment: I can add the cpython_only decorator, but I'm not sure it is the right thing to do. I would expect the code in the test to pass on any Python implementation, which would suggest that it should not be cpython_only, right? If you still think so, I'll add it.

[issue28150] Error CERTIFICATE_VERIFY_FAILED in macOS

Changes by Christian Heimes : -- priority: normal -> critical type: crash -> security ___ Python tracker ___

[issue28150] Error CERTIFICATE_VERIFY_FAILED in macOS

Christian Heimes added the comment: We are not going to use certifi in rc and final releases. It's only in b1 because the infrastructure and cert handling code is not yet in place. Ned has a plan. -- nosy: +christian.heimes, ned.deily ___ Python

[issue28150] Error CERTIFICATE_VERIFY_FAILED in macOS

New submission from Jason R. Coombs: I saw the notice when installing Python 3.6.0b1 on Mac that said that the bundled OpenSSL is no longer used and thus 'certifi' will need to be installed and kept up-to-date. At first, I thought, "no big deal" and pushed forward, but since, I've

[issue28149] Incorrect indentation under “else” in _bsddb.c

New submission from Martin Panter: Compiling Python 2.7 gives: /home/proj/python/cpython/Modules/_bsddb.c: In function ‘newDBObject’: /home/proj/python/cpython/Modules/_bsddb.c:936:5: warning: this ‘else’ clause does not guard... [-Wmisleading-indentation] else ^~~~

[issue28148] Also stop using localtime() in timemodule

New submission from Ed Schouten: In issue 28067, we changed _datetimemodule to stop using localtime() and gmtime(), which is nice. I actually needed such a change for CloudABI (https://mail.python.org/pipermail/python-dev/2016-July/145708.html) which does not provide the thread-unsafe

[issue28147] Memory leak in new 3.6 dictionary resize

Berker Peksag added the comment: Can you wrap the test with @support.cpython_only decorator? The patch fixes the memory leak demonstrated in test-dict-pop.py. -- nosy: +berker.peksag, haypo, methane stage: -> patch review ___ Python tracker

[issue28147] Memory leak in new 3.6 dictionary resize

Changes by Min RK : -- title: Memory leak in dictionary resize -> Memory leak in new 3.6 dictionary resize ___ Python tracker ___

[issue28147] Memory leak in dictionary resize

Min RK added the comment: This patch fixes the memory leak in split-dict resizing. Each time dict_resize is called, it gets a new, larger size `> minused`. If this is triggered many times, it will keep growing in size by a factor of two each time, as the previous size is passed as minused for

[issue28147] Memory leak in dictionary resize

New submission from Min RK: There is a memory leak in the new dictionary resizing in 3.6, which can cause memory exhaustion in just a few iterations. I don't fully understand the details of the bug, but it happens when resizing a dict with a split table several times. The only way that I

[issue28146] Confusing error messages in str.format()

Eric V. Smith added the comment: See also issue 27772. The difference in the error messages is due to the first ones looking for specific invalid combinations (in this case things the string formatter does not understand), and the last one which is "I have no idea what you're asking for".

[issue22543] -W option cannot use non-standard categories

Torsten Landschoff added the comment: Wow, this was news to me and I just ran into it in python 2.7. Checked in Python 3 and it's still there: ``` (py3)->torsten.landschoff@horatio:~$ python3 --version Python 3.6.0a3+ (py3)->torsten.landschoff@horatio:~$ python3 -W

[issue28146] Confusing error examples in str.format()

New submission from Serhiy Storchaka: For example: >>> '{:04}'.format('abc') Traceback (most recent call last): File "", line 1, in ValueError: '=' alignment not allowed in string format specifier There is no any '=' in the format string. >>> '{: }'.format('abc') Traceback (most recent

[issue28146] Confusing error messages in str.format()

Changes by Serhiy Storchaka : -- title: Confusing error examples in str.format() -> Confusing error messages in str.format() ___ Python tracker

[issue28145] Fix whitespace in C source code

New submission from Francisco Couzo: I changed most of the tabs in the source code to spaces, and removed trailing whitespace. I also made some scripts that generate code use spaces and not generate trailing whitespace. (makesetup and makeunicodedata.py) I fixed a typo in Modules/makesetup

[issue28126] Py_MEMCPY: Use memcpy on Windows?

Serhiy Storchaka added the comment: Shouldn't the NEWS entity be in the C API section? -- nosy: +serhiy.storchaka ___ Python tracker ___

[issue28143] ASDL compatibility with Python 3 system interpreter

Changes by Serhiy Storchaka : -- nosy: +benjamin.peterson ___ Python tracker ___

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Serhiy Storchaka added the comment: I support Stefan. Just requiring 8-byte align is the easiest solution, but it doesn't work with memoryview without expensive memory allocation and copying. Look at the FNV code. It supports non-4-byte aligned data, and does it in a safe and efficient way.

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Stefan Krah added the comment: I see. No, most NumPy arrays are C-contiguous. Multi-dimmensional arrays are contiguous, too. Non C-contiguous arrays arise mostly during slicing or if they're Fortran-order to begin with. But NumPy aside, it's weird to have slice of a huge regular bytes view

[issue27441] redundant assignments to ob_size of new ints that _PyLong_New returned

Changes by Mark Dickinson : -- nosy: +mark.dickinson ___ Python tracker ___ ___

[issue28142] windows installer not adding PYTHONHOME

Eryk Sun added the comment: PYTHONHOME should only be set temporarily for special cases. If running python.exe without PYTHONHOME fails to find the standard library, then your system is misconfigured. For additional help with this problem, ask on either python-list or python-win32.

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Christian Heimes added the comment: memory_hash has to convert buffers unless the buffer is a single-dimensional, C-style and contiguous buffer. A NumPy matrix has more than one dimension, so it must be converted. https://hg.python.org/cpython/file/tip/Objects/memoryobject.c#l2854 if

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Stefan Krah added the comment: I don't understand this. Could you explain? -- ___ Python tracker ___ ___

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Christian Heimes added the comment: memoryview() has to create a copy for NumPy memoryviews already. -- ___ Python tracker ___

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Stefan Krah added the comment: s/unaligned/not 8-byte-aligned/ -- ___ Python tracker ___ ___ Python-bugs-list

[issue28144] Decrease empty_keys_struct's dk_refcnt

New submission from Xiang Zhang: There is no dummy_struct any more. So I think we can decrease empty_keys_struct's dk_refcnt. -- files: empty_keys.patch keywords: patch messages: 276405 nosy: haypo, methane, xiang.zhang priority: normal severity: normal status: open title: Decrease

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Stefan Krah added the comment: Numpy itself internally doesn't. Consumers of numpy arrays use memoryviews. Numpy is often used as a library these days, even for simple things like storing a 2-d table, which can easily be several TB. It is also easy to generate unaligned data by just taking a

[issue28123] _PyDict_GetItem_KnownHash ignores DKIX_ERROR return

Xiang Zhang added the comment: Update the patch with unittest. -- Added file: http://bugs.python.org/file44655/issue28123_v2.patch ___ Python tracker ___

[issue27374] Cygwin: Makefile does not install DLL import library

Masayuki Yamamoto added the comment: I'm so sorry that I misspelled your name, Erik. -- ___ Python tracker ___

[issue28143] ASDL compatibility with Python 3 system interpreter

New submission from Malthe Borch: Many systems today use Python 3 as the default interpreter "python". The Python 2.7 build fails because of syntax incompatibility. Attached patch fixes this. -- components: Build files: 0001-Allow-make-to-be-run-under-Python-3.patch keywords: patch

[issue28114] parse_envlist(): os.execve(), os.spawnve(), etc. crash in Python 3.6.0 when env contains byte strings

Eryk Sun added the comment: Berker, this is basically what I had in my initial patch on the Unix side. I also addressed the Windows issues in parse_envlist and fsconvert_strdup. I'm uploading that patch for reference. It needs a test. I also need to verify that there are no additional

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Christian Heimes added the comment: How often does NumPy create a C-style, single dimensional, continuous memoryview? I would assume that it deals with matrices, Fortran data and/or other strides, multi-dimensional data in almost all cases. -- ___

[issue27374] Cygwin: Makefile does not install DLL import library

Masayuki Yamamoto added the comment: Hello, Eric. I tried to build the '_speedups' C extension module of simplejson (third party library) using the python built on cygwin. As you pointed out, the build of '_speedups' C extension module at the stage of the link failed. build log: Python3.5.2+

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Stefan Krah added the comment: Ah, yes. But compilers optimize memcpy and this is a guaranteed slowdown for the unaligned memoryview case. -- ___ Python tracker

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Christian Heimes added the comment: It's totally possible. Benjamin's patch implements it like I have suggested it. -- ___ Python tracker ___

[issue28134] socket.socket(fileno=fd) does not work as documented

Christian Heimes added the comment: Martin, the documentation says "If fileno is specified, the other arguments are ignored, causing the socket with the specified file descriptor to return." It's a direct quote from Python 3's socket library documentation. For a non-native speaker like me,

[issue28055] pyhash's siphash24 assumes alignment of the data pointer

Stefan Krah added the comment: For memoryview this is not possible: It is explicitly unaligned and the feature is used in e.g. NumPy. -- ___ Python tracker

[issue26830] Refactor Tools/scripts/google.py

Berker Peksag added the comment: Thanks! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.7 -Python 3.6 ___ Python tracker

[issue26830] Refactor Tools/scripts/google.py

Roundup Robot added the comment: New changeset f8c11b61cfb7 by Berker Peksag in branch 'default': Issue #26830: Refactor Tools/scripts/google.py https://hg.python.org/cpython/rev/f8c11b61cfb7 -- nosy: +python-dev ___ Python tracker

[issue28114] parse_envlist(): os.execve(), os.spawnve(), etc. crash in Python 3.6.0 when env contains byte strings

STINNER Victor added the comment: The issue is not specific to Windows, the following example also crash on Linux: import os, sys args = [sys.executable, "-c", "pass"] os.execve(args[0], args, os.environb) -- ___ Python tracker

[issue28114] parse_envlist(): os.execve(), os.spawnve(), etc. crash in Python 3.6.0 when env contains byte strings

Changes by STINNER Victor : -- title: Crash in unicodeobject.c find_maxchar_surrogates on python-3.6.0b1 for Windows -> parse_envlist(): os.execve(), os.spawnve(), etc. crash in Python 3.6.0 when env contains byte strings

[issue28119] Explicit null dereferenced in formatter_unicode.c

Christian Heimes added the comment: Thanks! You fixed the problem and Coverity is no longer complaining. The code is more readable, too. Are you talking about these lines? Yes, they were confusing me. if (locale_info->thousands_sep == NULL) { Py_DECREF(locale_info->decimal_point); }

[issue27778] PEP 524: Add os.getrandom()

Christian Heimes added the comment: Oh sorry, I looked in the wrong location and missed it. * if (PyErr_CheckSignals() < 0) {return NULL;} does not free buffer with PyMem_Free(buffer); * The function allocates memory once with PyMem_Malloc() and later a second time with

[issue28142] windows installer not adding PYTHONHOME

Changes by Berker Peksag : -- nosy: +steve.dower, zach.ware ___ Python tracker ___

[issue28114] Crash in unicodeobject.c find_maxchar_surrogates on python-3.6.0b1 for Windows

Berker Peksag added the comment: Here's a patch for Unix. I will add Windows support when I get my Windows VM. > [...] (and add a test - test_crashers, presumably?) [...] Unfortunately, test_crashers doesn't run since 2011 (skipped in 481ad9129a0f.) parse_envlist() is only used by os.execve()

[issue26830] Refactor Tools/scripts/google.py

Changes by Francisco Couzo : Added file: http://bugs.python.org/file44652/scripts_google_v4.patch ___ Python tracker ___

[issue28142] windows installer not adding PYTHONHOME

New submission from Memo: Hi, when i run python from elevated CMD it is failed to start and give error system codecs not found... The problem is python installer for windows "python-3.5.2-amd64.exe" is not adding the PYTHONHOME environment variable. When I manually added it problem solved.

[issue27778] PEP 524: Add os.getrandom()

Nick Coghlan added the comment: Right, the only missing piece now is documentation of the ENOSYS case, which end users may encounter if a Python 3.6 binary that supports os.getrandom() is run against an older kernel. That's pretty easy to trigger via containers, as getrandom() was added in

[issue28141] shutil.copystat utime lookup fails on certain Android file systems

New submission from Jerry A: shutil.copystat fails on certain Android filesystems when trying lookup("utime") (line 194 of https://hg.python.org/cpython/file/3.5/Lib/shutil.py) Using Python 2.7 and Python 3.5 on Android 6.0.1., when I am on the "typical" user file system, and I run a script

<    1   2