[issue29644] Importing webbrowser outputs a message on stderr

[Nick Coghlan]

Nick Coghlan added the comment:

I considered that problem of "subprocess invocation as a side-effect of 
import", but in this case the output is needed by the immediately following 
"register_X_browsers()" call, as it affects how those browsers get inserted 
into webbrowser._tryorder.

That means there isn't any obvious way to delay it without a fairly significant 
refactoring of how the module works :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29644] Importing webbrowser outputs a message on stderr

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I see this because my main desktop is KDE, but optional libkf5config-bin 
package is not installed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29644] Importing webbrowser outputs a message on stderr

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +259

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24241] webbrowser default browser detection and/or public API for _trylist.

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +258

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29644] Importing webbrowser outputs a message on stderr

[Nick Coghlan]

Nick Coghlan added the comment:

Ah, and I think I'm not seeing this locally because I have both Gtk and KDE 
components installed (my main desktop is KDE, but Firefox and various other 
components are still Gtk apps).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9285] Add a profile decorator to profile and cProfile

[Louie Lu]

Changes by Louie Lu :


--
pull_requests: +257

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9285] Add a profile decorator to profile and cProfile

[Louie Lu]

Louie Lu added the comment:

giampaolo: it seems that contextmanager will somehow make a bad return in  
`trace_dispatch_return`:

$ ./python tests.py
# logging.fatal('%r %r' % (frame.f_code.co_name, self.cur[-3]))
CRITICAL:root:'runblock' '' ('profile', 0, '')
CRITICAL:root:'__enter__' '' ('profile', 0, '')
Traceback (most recent call last):
  File "tests.py", line 18, in 
with p.runblock():
  File "/home/grd/Python/cpython/Lib/contextlib.py", line 82, in __enter__
return next(self.gen)
  File "/home/grd/Python/cpython/Lib/profile.py", line 256, in trace_dispatch_i
if self.dispatch[event](self, frame, t):
  File "/home/grd/Python/cpython/Lib/profile.py", line 346, in 
trace_dispatch_return
raise AssertionError("Bad return", self.cur[-3])
AssertionError: ('Bad return', ('profile', 0, ''))

I make a workaround in GitHub PR, that skip the assert when self.cur[-3] is 
('profile', 0, ''), and this work fine with your test cases.

p.s. I'm not very sure about the assertion of bad return, maybe this workaround 
may cause some side affact I didn't notice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Andrei Fokau]

Andrei Fokau added the comment:

Docs promise already support for namespace packages, so just a minor 
clarification was done.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29644] Importing webbrowser outputs a message on stderr

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Just stderr. stdout contains the requested value.

And I think that an external program shouldn't be ran at import time. It would 
be better to defer its run until its output is required.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29644] Importing webbrowser outputs a message on stderr

[Nick Coghlan]

Nick Coghlan added the comment:

I think an appropriate way of handling this would be to route both stdout and 
stderr in the subprocess to subprocess.DEVNULL.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29644] Importing webbrowser outputs a message on stderr

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Importing webbrowser has a side effect. It outputs a message on stderr.

>>> import webbrowser
/usr/bin/xdg-settings: 1: /usr/bin/xdg-settings: kreadconfig5: not found

This is a regression introduced in issue24241.

--
components: Library (Lib)
messages: 288542
nosy: daves, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Importing webbrowser outputs a message on stderr
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24241] webbrowser default browser detection and/or public API for _trylist.

[Nick Coghlan]

Changes by Nick Coghlan :


--
assignee:  -> ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24241] webbrowser default browser detection and/or public API for _trylist.

[Nick Coghlan]

Nick Coghlan added the comment:

After merging this, I belatedly realised that codecov's complaint about the 
lack of code coverage for the diff was actually valid: now that "preferred" is 
a public API, it should have a cross-platform test case.

That should be a fairly straightforward test to write, as it can just use an 
arbitrary string, and then poke around in _tryorder directly to make sure it is 
updated as expected - it doesn't need to use a valid browser reference.

--
stage: resolved -> test needed
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun added the comment:

Richard wrote:
> while a handle is open with share mode X, you can only reopen 
> the file if you also use share mode X

To clarify, the share mode is not a property of a handle. It's a property of a 
File object. A handle is a generic reference to any kind of kernel object, not 
just a File.

The following is a brief discussion about access sharing and the way file 
deletion works in Windows. Both of these tend to frustrate Unix programmers who 
end up supporting Windows. In this discussion, a "File", with an uppercase 'F', 
is the Windows kernel object type that references an open device or file-system 
directory, file, or stream. A "file", with a lowercase 'f', is a data file.

Shared access is implemented for File objects [1] and tracked in a SHARE_ACCESS 
record. When opening a file or directory, its shared access state is updated by 
the kernel function IoCheckShareAccess [2]. Discretionary shared access is 
primarily a concern for file systems, but volume devices and disk devices (e.g. 
\\.\C: and \\.\PhysicalDrive0) also use it. Devices that are flagged for 
mandatory exclusive access [3] (e.g. \\.\COM1) generally ignore the share mode. 
Some non-exclusive devices also ignore the share mode (e.g. \\.NUL and \\.CON). 

If it's not ignored, the share mode affects delete, write, read, and execute 
access. The following File access rights are affected: DELETE, FILE_WRITE_DATA, 
FILE_APPEND_DATA, FILE_READ_DATA, and FILE_EXECUTE. The share mode thus affects 
any combination of generic access -- GENERIC_ALL, GENERIC_WRITE, GENERIC_READ, 
GENERIC_EXECUTE. 

A File object's requested sharing is stored in its SharedDelete, SharedWrite, 
and SharedRead members. The granted access that's relevant to the share mode is 
stored in the DeleteAccess, WriteAccess (write/append), and ReadAccess 
(read/execute) members. Given these values, checking for a sharing violation 
and updating the shared access counts uses the following logic:

RequireSharedDelete = DeleteAccessCount > 0;
RequireSharedWrite = WriteAccessCount > 0;
RequireSharedRead = ReadAccessCount > 0;
DenyDeleteAccess = SharedDeleteCount < OpenCount;
DenyWriteAccess = SharedWriteCount < OpenCount;
DenyReadAccess = SharedReadCount < OpenCount;

if (RequireSharedDelete && !SharedDelete ||
RequireSharedWrite && !SharedWrite ||
RequireSharedRead && !SharedRead ||
DenyDeleteAccess && DeleteAccess ||
DenyWriteAccess && WriteAccess ||
DenyReadAccess && ReadAccess)
{
return STATUS_SHARING_VIOLATION;
}

OpenCount++;
DeleteAccessCount += DeleteAccess;
WriteAccessCount += WriteAccess;
ReadAccessCount += ReadAccess;
SharedDeleteCount += SharedDelete;
SharedWriteCount += SharedWrite;
SharedReadCount += SharedRead;

For example, to be granted delete access, all existing File object references 
must share delete access. However, if a file is opened with delete sharing but 
delete access hasn't been granted, then it can be opened again without delete 
sharing.

The SHARE_ACCESS structure is usually stored in a file (or stream) control 
block (FCB/SCB), which is a structure that coordinates access to a file or 
directory across multiple File objects. The FsContext member of a File object 
points at the FCB. 

A file system stores its private state for a File in a context control block 
(CCB), to which the File's FsContext2 member points. The CCB is where a file 
system tracks, for example, whether the file should be deleted when the object 
is closed. 

Deleting a file sets a delete disposition in the FCB/SCB (or LCB if hard links 
are supported). The file can't be unlinked until all referencing File objects 
have been closed and the underlying FCB/SCB/LCB is closing. Until then a 
'deleted' file is still linked in the parent directory and prevents the 
directory from being deleted.

A deleted but still referenced file is in a semi-zombie state. Windows file 
systems don't allow opening such a file for any access, but it can still be 
accessed via existing objects. If one of these File objects has delete access, 
it can be used to unset the delete disposition (e.g. via 
SetFileInformationByHandle) to make the file accessible again.

[1]: https://msdn.microsoft.com/en-us/library/ff545834
[2]: https://msdn.microsoft.com/en-us/library/ff548341
[3]: https://msdn.microsoft.com/en-us/library/ff563827

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24241] webbrowser default browser detection and/or public API for _trylist.

[Nick Coghlan]

Nick Coghlan added the comment:

To summarise the changes that were made during the PR review, here's the 
eventual commit message for the merged PR:


 bpo-24241: Improve preferred webbrowser handling (#85)

- Add 'preferred' argument to webbrowser.register
- Use xdg-settings to specify preferred X browser

The first change replaces the existing undocumented tri-state
'try_order' parameter with the documented boolean keyword-only
'preferred' parameter. Setting it to True places the browser at the
front of the list, preferring it as the return to a subsequent get() call.

The second change adds a private `_os_preferred_browser` setting
and then uses that to make the default browser reported by
`xdg-settings` first in the try list when running under X (or
another environment that sets the `DISPLAY` variable).
This avoids the problem where the first entry in the tryorder
queue otherwise defaults to xdg-open, which doesn't support
the "new window" option.


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24241] webbrowser default browser detection and/or public API for _trylist.

[Nick Coghlan]

Changes by Nick Coghlan :


--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun added the comment:

Nick wrote:
> 1. Open the file in the current process
> 2. Write content to the file
> 3*. Close the file in the current process

In step 1, do you mean calling NamedTemporaryFile with delete=False? In that 
case there's no immediate problem with opening the file again in Windows. If 
you mean calling NamedTemporaryFile with delete=True, then step 3 deletes the 
file. 

Adding support for Windows share modes would be useful in general and would 
help within the current process. However, users may also need to open the 
temporary file in another process. Most programs don't open their files with 
shared delete access. 

There's a workaround to allow the file to be opened normally, but it involves 
setting the delete disposition and then clearing it in a pointless dance. It 
would be better to implement an option such as delete=AFTER_CM_EXIT, to try to 
remove the file without relying on O_TEMPORARY. The downside is that the file 
won't be deleted if the interpreter crashes, gets terminated or if another 
process has the file open without delete sharing.

--
nosy: +eryksun
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28929] Provide a link from documentation back to its source file

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +256

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28929] Provide a link from documentation back to its source file

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +255

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28929] Provide a link from documentation back to its source file

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +254

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29643] --enable-optimizations compiler flag has no effect

[Alex Wang]

Alex Wang added the comment:

At least when I last built Python, configure always said that optimizations 
were not enabled regardless of whether I passed in the flag.

>From what it looked like to me, it's because configure uses the $enableval 
>variable to store the result of the check for --enable-optimizations, but the 
>script checks $withval

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29643] --enable-optimizations compiler flag has no effect

[STINNER Victor]

STINNER Victor added the comment:

The flag has an effect, I tested it today ;-)

Can you please elaborate why you consider that the flag has no effect?

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29643] --enable-optimizations compiler flag has no effect

[Alex Wang]

Changes by Alex Wang :


--
pull_requests: +253

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29643] --enable-optimizations compiler flag has no effect

[Alex Wang]

New submission from Alex Wang:

PR submitted here: https://github.com/python/cpython/pull/129

--
components: Build
messages: 288535
nosy: awang
priority: normal
severity: normal
status: open
title: --enable-optimizations compiler flag has no effect
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +ezio.melotti, michael.foord, rbcollins
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Andrei Fokau]

Andrei Fokau added the comment:

Alright, I made an initial fix in #282. I believe that I still need 
to update the docs and run it with something big, e.g. Django.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Roundup Robot]

Changes by Roundup Robot :


--
pull_requests: +251

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29596] Unfinished sentense in howto/clinic.rst

[Steven Rumbalski]

Steven Rumbalski added the comment:

Truncated sentence was always truncated.

Introduced on commit bebf73511a1250fc768bcb7192b5b3c3fd04d8f2
from Issue #20287: Argument Clinic's output is now configurable, allowing
delaying its output or even redirecting it to a separate file.
https://github.com/python/cpython/commit/bebf73511a1250fc768bcb7192b5b3c3fd04d8f2

Another issue from the same commit that still exists today:

buffer
Save up all most of the output from Clinic, to be written into your file 
near the end.


Perhaps what was meant was 'almost all'?

--
nosy: +Steven Rumbalski

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd (based on deprecated asyncore/asynchat)

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd (based on deprecated asyncore/asynchat)

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
pull_requests: +250

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd (based on deprecated asyncore/asynchat)

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
pull_requests: +249

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd (based on deprecated asyncore/asynchat)

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
pull_requests: +248

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Andrei Fokau]

Andrei Fokau added the comment:

Ok, it's actually not so hard to work around (for Python 3.6, at least):


import os
from unittest import TestLoader

class CustomTestLoader(TestLoader):
def _find_test_path(self, full_path, pattern, namespace=False):
original_isfile = os.path.isfile

def patched_isfile(path):
return str(path).endswith('__init__.py') or original_isfile(path)

os.path.isfile = patched_isfile
result = super()._find_test_path(full_path=full_path, pattern=pattern,
 namespace=namespace)
os.path.isfile = original_isfile
return result


I'll try to submit a pull request if it can be resolved properly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26184] Add versionchanged note for error when create_module() is not defined by loaders

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +247

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9285] Add a profile decorator to profile and cProfile

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

The original patch is basically blocked by the cryptic assertion error reported 
above. It's not clear what it means or how to work around it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9285] Add a profile decorator to profile and cProfile

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29318] Optimize _PyFunction_FastCallDict() for **kwargs

[STINNER Victor]

Changes by STINNER Victor :


--
resolution: fixed -> rejected

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd (based on deprecated asyncore/asynchat)

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
pull_requests: +246

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Andrei Fokau]

New submission from Andrei Fokau:

Hi, As far as I see, unittest.TestLoader doesn't search in PEP-420 packages, 
i.e. packages without __init__.py files.

Is there some motivation behind this, or the loader was just not yet adapted 
for Implicit Namespace Packages?

--
components: Tests
messages: 288529
nosy: Andrei Fokau
priority: normal
severity: normal
status: open
title: Why does unittest.TestLoader.discover still rely on existence of 
__init__.py files?
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29414] Change 'the for statement is such an iterator' in Tutorial

[Marco Buttu]

Changes by Marco Buttu :


--
pull_requests: +245

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26532] build fails with address sanitizer

[Jonas Jelten]

Changes by Jonas Jelten :


--
nosy: +jj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29640] _PyThreadState_Init and fork race leads to inconsistent key list

[Charalampos Stratakis]

Changes by Charalampos Stratakis :


--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26359] CPython build options for out-of-the box performance

[STINNER Victor]

STINNER Victor added the comment:

It seems like it's not possible to install Python configured with 
--enable-optimizations: see my issue #29641, "make install failure when using 
./configure --enable-optimizations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29641] make install failure when using ./configure --enable-optimizations

[STINNER Victor]

Changes by STINNER Victor :


--
title: Missing _sysconfig_data when building+installating Python using PGO+LTO 
-> make install failure when using ./configure --enable-optimizations

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29641] Missing _sysconfig_data when building+installating Python using PGO+LTO

[STINNER Victor]

STINNER Victor added the comment:

Ok, I confirm that it's possible to compile and install Python using PGO+LTO 
without --enable-optimizatioins:

cd ~/cpython
make distclean ||:
./configure --with-lto --prefix=/home/haypo/installed/3.7
make clean
make profile-opt
make install

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29465] Modify _PyObject_FastCall() to reduce stack consumption

[STINNER Victor]

STINNER Victor added the comment:

Hum, code to call functions changed a lot recently, and other issues already 
enhanced the stack usage.

I should measure the stack usage to check if it's still worth it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29641] Missing _sysconfig_data when building+installating Python using PGO+LTO

[STINNER Victor]

STINNER Victor added the comment:

The --enable-optimizations option of ./configure was added by the issue #26359 
(first named --with-optimizations, it was recalled to --enable-optimizations 
later).

See also:
* issue #28032: "never imply --with-lto as part of --with-optimizations"
* issue #28605: Fix the help and What's New entry for --with-optimizations

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29641] Missing _sysconfig_data when building+installating Python using PGO+LTO

[STINNER Victor]

STINNER Victor added the comment:

PGO compilation uses "make profile-removal", maybe my issue is linked to that?

I'm sure that PGO compilation + "make install" worked two months, but using: 
make profile-opt && make install.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29641] Missing _sysconfig_data when building+installating Python using PGO+LTO

[STINNER Victor]

New submission from STINNER Victor:

Hi,

I compiled Python 3.7 using PGO+LTO with "make && make install", but it seems 
like it lacks the _sysconfigdata module.


Moreover, I don't see any "Makefile" file in the installed directory, so the 
"~/installed/3.7/bin/python3  -S -m sysconfig --generate-posix-vars" command 
fails on trying to import _sysconfigdata_m_linux_x86_64-linux-gnu ...   
   

$ find /home/haypo/installed/3.7 -name "Makefile"


Is it correct to use "make && make install" using PGO?

cd ~/cpython
make distclean ||:
./configure --enable-optimizations --with-lto --prefix=/home/haypo/installed/3.7
make clean
make
make install

haypo@speed-python$ ~/installed/3.7/bin/python3 
Failed to import the site module
Traceback (most recent call last):
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 544, in 
main()
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 530, in main
known_paths = addusersitepackages(known_paths)
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 282, in 
addusersitepackages
user_site = getusersitepackages()
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 258, in 
getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
  File "/home/haypo/installed/3.7/lib/python3.7/site.py", line 248, in 
getuserbase
USER_BASE = get_config_var('userbase')
  File "/home/haypo/installed/3.7/lib/python3.7/sysconfig.py", line 601, in 
get_config_var
return get_config_vars().get(name)
  File "/home/haypo/installed/3.7/lib/python3.7/sysconfig.py", line 550, in 
get_config_vars
_init_posix(_CONFIG_VARS)
  File "/home/haypo/installed/3.7/lib/python3.7/sysconfig.py", line 421, in 
_init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata_m_linux_x86_64-linux-gnu'

--
components: Build
messages: 288523
nosy: haypo
priority: normal
severity: normal
status: open
title: Missing _sysconfig_data when building+installating Python using PGO+LTO
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9285] Add a profile decorator to profile and cProfile

[Louie Lu]

Louie Lu added the comment:

Ping. Is there any reason why this patch doesn't accept?

--
nosy: +louielu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Nick Coghlan]

Nick Coghlan added the comment:

John: I don't think it would be clutter to have an explicit reminder about that 
point in the NamedTemporaryFile documentation, so feel free to file a separate 
enhancement issue for it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[John Florian]

John Florian added the comment:

Okay Nick. Thanks for the detailed info. I suspected buffering was a factor, 
but wasn't certain. Would it be worthwhile pursuing a note in the docs or would 
that constitute clutter over what should be a standard assumption? I was thrown 
off course for all the prior uses without issues, but in hindsight I don't know 
offhand how many involved a subprocess.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24905] Allow incremental I/O to blobs in sqlite3

[Aviv Palivoda]

Aviv Palivoda added the comment:

I am not a DB expert but from a quick search I couldn't find a similar API. I 
did find a few API's in other programming languages to sqlite blob:
1. https://jgallagher.github.io/rusqlite/rusqlite/blob/index.html
2. https://godoc.org/github.com/mxk/go-sqlite/sqlite3
3. http://www.ch-werner.de/javasqlite/SQLite/Blob.html

It seems like most of them give the same API as apsw.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27788] platform module's version number doesn't match its docstring

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Hmm, not sure why the merge is not showing up on the ticket.

Here's the link: 
https://github.com/python/cpython/commit/6059ce45aa96f52fa0150e68ea655fbfdc25609a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24905] Allow incremental I/O to blobs in sqlite3

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Is there similar API in other databases?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27788] platform module's version number doesn't match its docstring

[Marc-Andre Lemburg]

Changes by Marc-Andre Lemburg :


--
assignee:  -> lemburg
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27788] platform module's version number doesn't match its docstring

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

The purpose of __version__ in the platform module is to be able to use it with 
other Python as well (and then detect which version is available in 
applications).

So I think it's good to keep it around.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24905] Allow incremental I/O to blobs in sqlite3

[Aviv Palivoda]

Aviv Palivoda added the comment:

I opened a PR in github. I tagged some other developers if that will not start 
a discussion on the API I will post in the python-ideas.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24905] Allow incremental I/O to blobs in sqlite3

[Aviv Palivoda]

Changes by Aviv Palivoda :


--
pull_requests: +244

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[CyberJacob]

CyberJacob added the comment:

Just confirmed, this does affect multiprocessing. script to reproduce:

import sys, multiprocessing

def log_exception(*args):
print 'got exception %s' % (args,)

sys.excepthook = log_exception

def foo():
a = 1 / 0

multiprocessing.Process(target=foo).start()

again, just a normal traceback instead of "got exception"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28747] Expose SSL_CTX_set_cert_verify_callback

[Chi Hsuan Yen]

Changes by Chi Hsuan Yen :


--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Chi Hsuan Yen]

Changes by Chi Hsuan Yen :


--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Jacob Mansfield]

Jacob Mansfield added the comment:

Does this affect threads started with the multiprocessing library as well?

--
nosy: +Jacob Mansfield

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29414] Change 'the for statement is such an iterator' in Tutorial

[Jim Fasarakis-Hilliard]

Jim Fasarakis-Hilliard added the comment:

Go ahead with making the PR, marco. I'll take a look at it too when you do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29104] Left bracket remains in format string result when '\' preceeds it

[Jim Fasarakis-Hilliard]

Jim Fasarakis-Hilliard added the comment:

Serhiy, since review activity has dropped on b.p.o now that the move to github 
was made, would you like to make this into a pull request to get it reviewed 
and merged faster?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29640] _PyThreadState_Init and fork race leads to inconsistent key list

[Ján Stanček]

New submission from Ján Stanček:

Following crash is sporadically observed in RHEL7 anaconda:

(gdb) f 0
#0  PyThread_ReInitTLS () at /usr/src/debug/Python-2.7.5/Python/thread.c:411
411 if (p->id != id) {
(gdb) list
406 keymutex = PyThread_allocate_lock();
407
408 /* Delete all keys which do not match the current thread id */
409 q = 
410 while ((p = *q) != NULL) {
411 if (p->id != id) {
412 *q = p->next;
413 free((void *)p);
414 /* NB This does *not* free p->value! */
415 }
(gdb) p p
$1 = (struct key *) 0x
(gdb) p keyhead
$2 = (struct key *) 0x

key list is protected by keymutex (except for PyThread_ReInitTLS), but there 
doesn't seem to be any protection against concurrent fork(). What seems to 
happen is fork() at the moment when key list is not consistent.

For example, if I initialize new key to 0xfe:

static struct key *find_key(int key, void *value)
// find_key with extra memset()
...
p = (struct key *)malloc(sizeof(struct key));
memset(p, 0xfe, sizeof(struct key));
if (p != NULL) {
p->id = id;
p->key = key;
p->value = value;
p->next = keyhead;
keyhead = p;
}
...

Looking at disassembly, compiler reordered last 2 writes:

   0x004fcb50 <+272>:   callq  0x413d10 
   0x004fcb55 <+277>:   movabs $0xfefefefefefefefe,%rcx
   0x004fcb5f <+287>:   test   %rax,%rax
   0x004fcb62 <+290>:   mov%rcx,(%rax)
   0x004fcb65 <+293>:   mov%rcx,0x8(%rax)
   0x004fcb69 <+297>:   mov%rcx,0x10(%rax)
   0x004fcb6d <+301>:   mov%rcx,0x18(%rax)
   0x004fcb71 <+305>:   je 0x4fcaff 
   0x004fcb73 <+307>:   mov0x2f1e26(%rip),%rdx# 0x7ee9a0 

   0x004fcb7a <+314>:   mov0x2f1dff(%rip),%rdi# 0x7ee980 

   0x004fcb81 <+321>:   xor%r14d,%r14d
   0x004fcb84 <+324>:   mov%rbp,0x8(%rax)
   0x004fcb88 <+328>:   mov%r12d,0x10(%rax)
   0x004fcb8c <+332>:   mov%r13,0x18(%rax)
   0x004fcb90 <+336>:   mov%rax,0x2f1e09(%rip)# 0x7ee9a0 

   0x004fcb97 <+343>:   mov%rdx,(%rax)

Now consider what happens, when different threads call fork() in between these 
2 writes: we updated keyhead, but keyhead->next has not been updated yet.

Now when anaconda hangs, I get:

(gdb) list
407 keymutex = PyThread_allocate_lock();
408
409 /* Delete all keys which do not match the current thread id */
410 q = 
411 while ((p = *q) != NULL) {
412 if (p->id != id) {
413 *q = p->next;
414 free((void *)p);
415 /* NB This does *not* free p->value! */
416 }
(gdb) p p
$1 = (struct key *) 0xfefefefefefefefe
(gdb) p keyhead
$2 = (struct key *) 0xfefefefefefefefe


Here's how I think we get into this state:

-> thread 1
# has GIL
Thread.start
  _start_new_thread(self.__bootstrap, ())
PyThread_start_new_thread(t_bootstrap)
  # spawns thread 3

-> thread 2
...
  # waiting for GIL

-> thread 3
t_bootstrap
  _PyThreadState_Init # does not have GIL yet at this point
_PyGILState_NoteThreadState
  PyThread_set_key_value(autoTLSkey, (void *)tstate)
find_key
  # key list is temporarily not consistent
  # due to compiler reordering couple writes in find_key

-> thread 1
continuing Thread.start
  self.__started.wait()
Event.wait()
  self.__cond.wait 
Condition.wait()
  waiter = _allocate_lock()
  waiter.acquire()
lock_PyThread_acquire_lock
  Py_BEGIN_ALLOW_THREADS
PyEval_SaveThread
  PyThread_release_lock(interpreter_lock);

-> thread 2
...
  # acquired GIL
  os.fork() # forks inconsistent list

-> child
PyOS_AfterFork()
  PyThread_ReInitTLS()
SIGSEGV

Attached patch for python makes it easier to reproduce, by adding delays to 
couple places to make window key list is not consistent larger.

--
components: Interpreter Core
files: bz1268226_reproducer2.tar.gz
messages: 288510
nosy: Ján Stanček
priority: normal
severity: normal
status: open
title: _PyThreadState_Init and fork race leads to inconsistent key list
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file4/bz1268226_reproducer2.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28556] typing.py upgrades

[Ivan Levkivskyi]

Changes by Ivan Levkivskyi :


--
pull_requests: +243

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27435] ctypes library loading and AIX - also for 2.7.X (and later)

[Michael Haubenwallner]

Michael Haubenwallner added the comment:

Let's switch to github-based patches to discuss about:
https://github.com/python/cpython/compare/2.7...haubi:issue27435/2.7

For the library search path I prefer to use loadquery(L_GETLIBPATH), available 
via new _ctypes_aix module now, but also used with L_GETINFO in 
Python/dynload_aix.c already.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:

I updated the PR to cover both Serhiy's change to make the legacy bytecode safe 
to interpret, and a tweaked version of Petr's change to allow the legacy 
bytecode to be imported.

The main tweaks to the latter were:

- more in depth comments explaining the changes
- switching the C level changes to rely on a couple of extern variables 
exported from import.c rather than scattering the backwards compatibility 
numbers throughout the code

I'm also cc'ing Larry into the discussion as 3.5 release manager. Larry, most 
of the context for this change is actually in http://bugs.python.org/issue29514 
where we reported the problems with the magic number change that prompted Petr 
to create a downstream patch for Fedora to restore compatibility with the 
legacy bytecode magic number.

While the answer for 3.6+ is "Let's try to avoid ever doing this again", it 
turns out this is tricky enough to handle that it would be nice to have a 
shared solution upstream in 3.5.4+ that redistributors can collectively adopt, 
rather than everyone needing to come up with their own workaround for the 
problem.

--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29634] Reduce deque repeat execution when maxlen exist and size is not 1

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There is an opportunity of further optimisation. For example 
deque(range(10), maxlen=11) *= 2 copies 10 elements while it is 
enough to copy just 1 element. But I don't know whether it is worth to optimise 
this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29615] SimpleXMLRPCDispatcher._dispatch mangles tracebacks when invoking RPC calls through _dispatch

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +loewis, serhiy.storchaka
stage:  -> patch review
type:  -> behavior
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com