[issue32865] os.pipe creates inheritable FDs with a bad internal state on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> The first is a bug in os.pipe (creation of an inheritable descriptor
> with non-inheritable underlying handle). This can be fixed by using
> _open_osfhandle() correctly.

This is the only issue to be addressed here, and it's an easy fix. The problem 
also occurs in Modules/_io/winconsoleio.c. 

I digressed on the problems with _Py_get_inheritable and _Py_set_inheritable. 
Such problems need to be addressed in a separate issue.

> Does an FD get flagged as a pipe by calling GetFileType in 
> _open_osfhandle?

Yes, it calls GetFileType to check for a bad handle (FILE_TYPE_UNKNOWN) and 
also to flag whether the FD type is FDEV (FILE_TYPE_CHAR) or FPIPE 
(FILE_TYPE_PIPE). See lowio\osfinfo.cpp in the CRT source code.

> Also, it's totally unclear to me how something like GetFileType 
> can interfere with I/O and cause a deadlock.

You're right to question this. I couldn't reproduce this problem in Windows 10 
using a local, anonymous pipe. GetFileType calls NtQueryVolumeInformationFile 
[1] to get the FileFsDeviceInformation for the referenced File. For local 
devices, the I/O Manager can simply copy the requested information from the 
associated Device object (e.g. \Device\NamedPipe); it doesn't even need to call 
the device driver with an IRP_MJ_QUERY_VOLUME_INFORMATION request. Maybe older 
implementations of the I/O Manager always waited to acquire the File's internal 
lock when opened in synchronous mode.

Whether this is (or ever was) a valid reason, it's the reason given for 
skipping the call in the CRT function initialize_inherited_file_handles_nolock 
(lowio\ioinit.cpp). There's nothing we can do about it. The CRT will not 
validate an inherited pipe FD at startup. Also, even this validation can't help 
if the same handle just happens to be assigned to another File at startup.

[1]: 
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-ntqueryvolumeinformationfile

> Both issues cause out-of-sync which is usually hidden and harmless 
> because people prefer subprocess.Popen() 

Python has to respect file descriptors as a common resource in the process. 
Other libraries or an embedding application may be using the same CRT. It's not 
simply a matter of what's commonly used in pure-Python code. That said, 
os.system() is still commonly used, and the CRT implements _wsystem via 
common_spawnv.

> one nasty thought is to poke msvcrt internal structures like in _PyVerify_fd 

That's an ugly option, but it may be the best (or only possible) approach until 
MSVC adds a public function to modify the heritability of an FD.

On a related note, in looking at places where FDs are made non-inheritable, I 
see it's also used in _Py_fopen_obj. This function could be modified to use 
MSVC's "N" flag to ensure the underlying FD is opened with _O_NOINHERIT. glibc 
has a similar "e" flag for O_CLOEXEC.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32865] os.pipe creates inheritable FDs with a bad internal state on Windows

[Nathaniel Smith]

Change by Nathaniel Smith :


--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32867] argparse assertion failure with multiline metavars

[MaT1g3R]

New submission from MaT1g3R :

If I run this script with -h
-8<--
from argparse import ArgumentParser
mapping = ['123456', '12345', '12345', '123']
p = ArgumentParser('11')
p.add_argument('-v', '--verbose', help='verbose mode', action='store_true')
p.add_argument('targets', help='installation targets',  nargs='+', 
metavar='\n'.join(mapping))
p.parse_args()
-8<
I get an error:
-8<
Traceback (most recent call last):
  File "tmp.py", line 7, in 
p.parse_args()
  File "/usr/lib/python3.6/argparse.py", line 1730, in parse_args
args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.6/argparse.py", line 1762, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.6/argparse.py", line 1968, in _parse_known_args
start_index = consume_optional(start_index)
  File "/usr/lib/python3.6/argparse.py", line 1908, in consume_optional
take_action(action, args, option_string)
  File "/usr/lib/python3.6/argparse.py", line 1836, in take_action
action(self, namespace, argument_values, option_string)
  File "/usr/lib/python3.6/argparse.py", line 1020, in __call__
parser.print_help()
  File "/usr/lib/python3.6/argparse.py", line 2362, in print_help
self._print_message(self.format_help(), file)
  File "/usr/lib/python3.6/argparse.py", line 2346, in format_help
return formatter.format_help()
  File "/usr/lib/python3.6/argparse.py", line 282, in format_help
help = self._root_section.format_help()
  File "/usr/lib/python3.6/argparse.py", line 213, in format_help
item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.6/argparse.py", line 213, in 
item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.6/argparse.py", line 334, in _format_usage
assert ' '.join(pos_parts) == pos_usage
AssertionError
-8<--

--
components: Library (Lib)
messages: 312299
nosy: MaT1g3R
priority: normal
severity: normal
status: open
title: argparse assertion failure with multiline metavars
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32862] os.dup2(fd, fd, inheritable=False) behaves inconsistently

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

@eryksun: Thank you for the note! I've commented on #32865. This adds even more 
inconsistency to this corner case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32865] os.pipe creates inheritable FDs with a bad internal state on Windows

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

>Also, it has to skip this check if the FD is flagged as a pipe, because a pipe 
>is likely opened in synchronous mode and blocked on a read in the parent, in 
>which case calling GetFileType would deadlock.

Does an FD get flagged as a pipe by calling GetFileType in _open_osfhandle?

Also, it's totally unclear to me how something like GetFileType can interfere 
with I/O and cause a deadlock.

My summary of this report. There are two issues.

The first is a bug in os.pipe (creation of an inheritable descriptor with 
non-inheritable underlying handle). This can be fixed by using 
_open_osfhandle() correctly.

The second is that os.set_inheritable() / _Py_set_inheritable() is broken on 
Windows because it may put descriptor and handle heritability out-of-sync (as 
happens e.g. in os.dup(), os.dup2(), _Py_fopen and friends). This is hard to 
fix because msvcrt doesn't have something like fcntl(F_SETFD) to change 
_O_NOINHERIT flag (though one nasty thought is to poke msvcrt internal 
structures like in _PyVerify_fd removed in #23524).

Both issues cause out-of-sync which is usually hidden and harmless because 
people prefer subprocess.Popen() which doesn't have the ability to propagate 
msvcrt-level descriptors at all. The issue surfaces itself if pipes and 
os.spawn* are involved, leading to two types of inconsistency in the child:
1) Bogus "valid" descriptors referring to wrong handles
2) Unexpectedly invalid descriptors and leaked handles

--
nosy: +izbyshev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32866] zipimport loader.get_data() requires absolute zip file path

[Barry A. Warsaw]

New submission from Barry A. Warsaw :

Over in https://gitlab.com/python-devs/importlib_resources/issues/48 we have a 
report of a FileNotFoundError when trying to read a resource from a zip file.  
Upon further debugging, I found that zipimport's loader.get_data() raises an 
unexpected OSError.  Interestingly, if the path to the zip file is absolute, 
everything works as expected, but if the path is relative, then it fails.

There's probably a missing abspath() in there somewhere, but as zipimport is 
written in C, I really didn't spend much time digging around in gdb.

--
messages: 312296
nosy: barry
priority: normal
severity: normal
status: open
title: zipimport loader.get_data() requires absolute zip file path
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32862] os.dup2(fd, fd, inheritable=False) behaves inconsistently

[Eryk Sun]

Eryk Sun  added the comment:

In Windows the CRT file descriptor is actually still inheritable. This only 
makes the underlying OS handle non-inheritable. I don't think there's a way to 
make an existing FD non-inheritable using public CRT functions. See issue 32865.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32865] os.pipe creates inheritable FDs with a bad internal state on Windows

[Eryk Sun]

Eryk Sun  added the comment:

Note that the CRT checks at startup whether an inherited FD is valid by calling 
GetFileType. If the handle is invalid or not a File, then the FD effectively is 
not inherited. This doesn't completely avoid the problem, since there's still a 
chance the handle was already assigned to a File at startup. Also, it has to 
skip this check if the FD is flagged as a pipe, because a pipe is likely opened 
in synchronous mode and blocked on a read in the parent, in which case calling 
GetFileType would deadlock.

For example:

>>> os.pipe()
(3, 4)
>>> os.dup2(4, 5, False) # fd 5 is still inheritable
>>> os.spawnl(os.P_WAIT, sys.executable, 'python')

Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40)
[MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, msvcrt
>>> msvcrt.get_osfhandle(5)
420

Handle 420 was not inherited, so it may be unassigned, or it could be for some 
random kernel object. It's assigned in this case:

>>> os.get_handle_inheritable(420)
False

I have a function that calls NtQueryObject to get the type name for a kernel 
handle:

>>> get_type_name(420)
'Semaphore'

Now, let's model what could be a confusing bug for someone using this semaphore:

>>> os.close(5) # BUGBUG
>>> os.get_handle_inheritable(420)
Traceback (most recent call last):
  File "", line 1, in 
OSError: [WinError 6] The handle is invalid

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30638] Additional dependencies and rule for `make regen-all`

[Zachary Ware]

Zachary Ware  added the comment:


New changeset c1b8aedfbabf6e5460b09f4792d80f18051d43d3 by Zachary Ware in 
branch '3.6':
[3.6] bpo-30638: Add clinic to `make regen-all` (GH-5671)
https://github.com/python/cpython/commit/c1b8aedfbabf6e5460b09f4792d80f18051d43d3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21060] Better error message for setup.py upload command without sdist/bdist

[Éric Araujo]

Éric Araujo  added the comment:

Thanks for the feedback, I went with the cleaner phrasing.

--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21060] Better error message for setup.py upload command without sdist/bdist

[Éric Araujo]

Change by Éric Araujo :


--
pull_requests: +5511

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30638] Additional dependencies and rule for `make regen-all`

[miss-islington]

miss-islington  added the comment:


New changeset d5be8e13b0ccf4473300d3a1672fc113702cf66c by Miss Islington (bot) 
in branch '3.7':
bpo-30638: Add clinic to `make regen-all` (GH-5671)
https://github.com/python/cpython/commit/d5be8e13b0ccf4473300d3a1672fc113702cf66c


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32716] setup.py register --repository is broken

[Éric Araujo]

Éric Araujo  added the comment:

There are many open issues with .pypirc.

I don’t think that -r URL is a supported use case though; the docs should say 
that the usage is to add a section in .pypirc to define a repo, then use -r 
repo-section-name.

--
versions:  -Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30638] Additional dependencies and rule for `make regen-all`

[Zachary Ware]

Change by Zachary Ware :


--
pull_requests: +5510

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30638] Additional dependencies and rule for `make regen-all`

[miss-islington]

Change by miss-islington :


--
pull_requests: +5509

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30638] Additional dependencies and rule for `make regen-all`

[Zachary Ware]

Zachary Ware  added the comment:


New changeset d6ff8a7037903497eff95fa32bdac2b6adf71505 by Zachary Ware in 
branch 'master':
bpo-30638: Add clinic to `make regen-all` (GH-5671)
https://github.com/python/cpython/commit/d6ff8a7037903497eff95fa32bdac2b6adf71505


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Yury Selivanov]

Yury Selivanov  added the comment:

Eric, it looks like your recent commit introduced a refleak.  We need to fix it 
before beta2.

~/d/p/cpython (master $) » ./python.exe -m test -R3:3 test_multiprocessing_fork
Run tests sequentially
0:00:00 load avg: 2.52 [1/1] test_multiprocessing_fork
beginning 6 repetitions
123456
..
test_multiprocessing_fork leaked [21, 2, 1] memory blocks, sum=24
test_multiprocessing_fork leaked [2, 0, 0] file descriptors, sum=2
test_multiprocessing_fork failed in 9 min 48 sec

1 test failed:
test_multiprocessing_fork


And just before it:


~/d/p/cpython ((bd093355…) $) » ./python.exe -m test -R3:3 
test_multiprocessing_fork
Run tests sequentially
0:00:00 load avg: 3.70 [1/1] test_multiprocessing_fork
beginning 6 repetitions
123456
..
test_multiprocessing_fork passed in 9 min 12 sec
1 test OK.

Total duration: 9 min 12 sec
Tests result: SUCCESS

--
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Guido van Rossum]

Guido van Rossum  added the comment:

> Isn't 800 lines of C code too high price for speeding up ABCs creation?

I think it's well worth it. This has long felt as a sore point to me.

On Sat, Feb 17, 2018 at 10:27 AM, Ivan Levkivskyi 
wrote:

>
> Ivan Levkivskyi  added the comment:
>
> > Isn't 800 lines of C code too high price for speeding up ABCs creation?
>
> 800 lines of C code is not something hard to notice, so I suppose the
> answer is obvious for all people involved in the work on PR :-)
>
> > ...this can save just several milliseconds at start-up.
>
> The correct way to measure this is relative, not absolute. There are just
> few ABCs used by modules loaded at Python start-up, and it already allowed
> to save 10% of start-up time. My expectation is that the number will be
> similar for a typical Python app. Moreover, `isinstance` and `issubclass`
> (functions called often with ABCs) will be 1.5x faster.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20928] xml.etree.ElementInclude does not include nested xincludes

[Stefan Behnel]

Change by Stefan Behnel :


--
pull_requests: +5508
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20928] xml.etree.ElementInclude does not include nested xincludes

[Stefan Behnel]

Stefan Behnel  added the comment:

The latest "fixed2" patch looks good to me, but the author didn't sign a 
contributors agreement. However, I did, and I already wrote the same thing for 
lxml, so I put together an initial PR.

--
type: behavior -> enhancement
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

> Isn't 800 lines of C code too high price for speeding up ABCs creation?

800 lines of C code is not something hard to notice, so I suppose the answer is 
obvious for all people involved in the work on PR :-)

> ...this can save just several milliseconds at start-up.

The correct way to measure this is relative, not absolute. There are just few 
ABCs used by modules loaded at Python start-up, and it already allowed to save 
10% of start-up time. My expectation is that the number will be similar for a 
typical Python app. Moreover, `isinstance` and `issubclass` (functions called 
often with ABCs) will be 1.5x faster.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31333] Implement ABCMeta in C

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Isn't 800 lines of C code too high price for speeding up ABCs creation? This 
will save several microseconds per ABC creation. Even if the program creates 
1000 ABCs, this can save just several milliseconds at start-up.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32865] os.pipe creates inheritable FDs with a bad internal state on Windows

[Eryk Sun]

New submission from Eryk Sun :

File descriptors in Windows are implemented by the C runtime library's low I/O 
layer. The CRT maps native File handles to Unix-style file descriptors. 
Additionally, in order to support inheritance for spawn/exec, the CRT passes 
inheritable FDs in a reserved field of the CreateProcess STARTUPINFO record. A 
spawned child process uses this information to initialize its FD table at 
startup.

Python's implementation of os.pipe creates File handles directly via 
CreatePipe. These handles are non-inheritable, which we want. However, it wraps 
them with inheritable file descriptors via _open_osfhandle, which we don't 
want. The solution is to include the flag _O_NOINHERIT in the _open_osfhandle 
call, which works even though this flag isn't explicitly documented as 
supported for this function.

Here's an example of the issue.

>>> fdr, fdw = os.pipe()
>>> fdr, fdw
(3, 4)
>>> msvcrt.get_osfhandle(3), msvcrt.get_osfhandle(4)
(440, 444)
>>> os.get_handle_inheritable(440)
False
>>> os.get_handle_inheritable(444)
False

Note that os.get_inheritable assumes that FD and handle heritability are in 
sync, so it only queries handle information. The following FDs are actually 
inheritable, while the underlying File handles are not:

>>> os.get_inheritable(3)
False
>>> os.get_inheritable(4)
False

This is a flawed assumption baked into _Py_get_inheritable and 
_Py_set_inheritable, especially since the latter has no effect on FD 
heritability. The CRT has no public functions to query and modify the 
heritability of existing FDs. Until it does (and maybe Steve Dower can request 
this from the MSVC devs), I see no point in pretending something works when it 
doesn't. This only creates problems.

Let's spawn a child to show that these FDs are inherited in a bad state, which 
is a potential source of bugs and data corruption:

>>> os.spawnl(os.P_WAIT, sys.executable, 'python')

Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40)
[MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, msvcrt
>>> msvcrt.get_osfhandle(3), msvcrt.get_osfhandle(4)
(440, 444)

As you can see, file descriptors 3 and 4 were inherited with the handle values 
from the parent process, but the handles themselves were not inherited. We'll 
be lucky if the handle values happen to be unassigned and remain so. However, 
they may be assigned or subsequently assigned to random kernel objects (e.g. a 
File, Job, Process, Thread, etc).

On a related note, Python always opens files as non-inheritable, even for 
os.open (which IMO makes no sense; we have O_NOINHERIT or O_CLOEXEC for that). 
It's assumed that the FD can be made inheritable via os.set_inheritable, but 
this does not work on Windows, and as things stand with the public CRT API, it 
cannot work. For example:

>>> os.open('test.txt', os.O_WRONLY)
3
>>> msvcrt.get_osfhandle(3)
460
>>> os.set_inheritable(3, True)
>>> os.get_handle_inheritable(460)
True

>>> os.spawnl(os.P_WAIT, sys.executable, 'python')

Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40)
[MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import msvcrt
>>> msvcrt.get_osfhandle(3)
Traceback (most recent call last):
  File "", line 1, in 
OSError: [Errno 9] Bad file descriptor

The Windows handle was of course inherited, but that's not useful in this 
scenario, since the CRT didn't know to pass the FD in the process STARTUPINFO. 
It's essentially a leaked handle in the child.

--
components: IO, Library (Lib), Windows
messages: 312283
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: os.pipe  creates inheritable FDs with a bad internal state on Windows
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32849] Fatal Python error: Py_Initialize: can't initialize sys standard streams

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

ktrace shows that dup(0) succeeded but fstat(0) failed. The symptom is the same 
as in #30225. Could you check whether any of the following quick tests produces 
the same error?

python3 -c 'import os, subprocess, sys; r, w = os.pipe(); os.close(w); 
subprocess.call([sys.executable, "-c", ""], stdin=r)'

python3 -c 'import os, subprocess, sys; r, w = os.pipe(); os.close(r); 
subprocess.call([sys.executable, "-c", ""], stdin=w)'

--
nosy: +izbyshev, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32864] Visual glitches when animating ScrolledText instances using place geometry manager

[Victor Domingos]

New submission from Victor Domingos :

In the current Python 3.7.0b1, on macOS 10.12.6 Sierra (also on 10.11 El 
Capitan), which seems to include a newer Tcl/tk version, there seems to be some 
visual glitches when using the place() geometry manager to animate a ttk.Frame 
containing ScrolledText widgets. These issues do not happen when running the 
same code on current Python 3.6.

Thanks in advance,

With best regards,
Victor Domingos



My current version:

Python 3.7.0b1 (v3.7.0b1:9561d7f501, Jan 30 2018, 19:10:11) 
[Clang 6.0 (clang-600.0.57)] on darwin



Here is an excerpt of my code (animated placement of the container frame):


def show_entryform(self, *event):
if not self.is_entryform_visible:
self.is_entryform_visible = True
self.btn_add.configure(state="disabled")
# Formulário de entrada de dados (fundo da janela)
self.my_statusbar.lift()

if SLOW_MACHINE:
self.entryframe.place(
in_=self.my_statusbar, relx=1, y=0, anchor="se", 
relwidth=1, bordermode="outside")
else:
for y in range(-30, -12, 6):
self.entryframe.update()
y = y**2
self.entryframe.place(
in_=self.my_statusbar, relx=1, y=y, anchor="se", 
relwidth=1, bordermode="outside")
for y in range(-12, -3, 3):
self.entryframe.update()
y = y**2
self.entryframe.place(
in_=self.my_statusbar, relx=1, y=y, anchor="se", 
relwidth=1, bordermode="outside")
for y in range(-3, 0, 1):
self.entryframe.update()
y = y**2
self.entryframe.place(
in_=self.my_statusbar, relx=1, y=y, anchor="se", 
relwidth=1, bordermode="outside")

self.entryframe.lift()


---

And the base class definition for the problematic widgets:


class LabelText(ttk.Frame):
"""
Generate an empty tkinter.scrolledtext form field with a text label above 
it.
"""

def __init__(self, parent, label, style=None, width=0, height=0):
ttk.Frame.__init__(self, parent)
if style:
self.label = ttk.Label(self, text=label, style=style, anchor="w")
else:
self.label = ttk.Label(self, text=label, anchor="w")

self.scrolledtext = ScrolledText(self, font=("Helvetica-Neue", 12),
 highlightcolor="LightSteelBlue2",
 wrap='word',
 width=width,
 height=height)

self.label.pack(side="top", fill="x", expand=False)
self.scrolledtext.pack(side="top", fill="both", expand=True)

--
components: Tkinter, macOS
files: issues.jpg
messages: 312281
nosy: Victor Domingos, gpolo, ned.deily, ronaldoussoren, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Visual glitches when animating ScrolledText instances using place 
geometry manager
type: behavior
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47450/issues.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32846] Deletion of large sets of strings is extra slow

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Reminds of some experiments someone did a while ago as part of the
GIL removal attempts where the ref count integers are all kept in a
separate array. The intent there was to be able to do locking on
a single array rather than on individual decref cells.

This would solve the issue with having to jump around in memory
to decref all objects, but I'm not sure whether the overall win
would be a lot, since deallocation of the memory blocks typically
requires accessing the block itself as well (to update the block
chain list pointers), unless the memory allocator uses some
smart cache local block management as well (I believe that pymalloc
does, but could be wrong).

In any case, this sounds like a fun experiment for a GSoC student.
Perhaps the PSF could donate an AWS EC2 instance with enough RAM to
do the experiments.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32863] Missing support for Emojis in tkinter

[Victor Domingos]

New submission from Victor Domingos :

In the current Python 3.7.0b1, on macOS 10.12.6 Sierra (also on 10.11 El 
Capitan), which seems to include a newer Tcl/tk version, it still does not 
support a variety of UTF characters, including Emoji characters that nowadays 
are of very common use. A quick search on the web returns some hints that maybe 
Tcl/tk could be compiled with different options in order to unlock those 
characters:

http://wiki.tcl.tk/515
https://core.tcl.tk/tk/tktview/6c0d7aec6713ab6a7c3e12dff7f26bff4679bc9d

I am not sure if it is officially supported by now, but at least for me, as a 
Python and tkinter user, it would be a great improvement.

Thanks in advance,

With best regards,
Victor Domingos


My current version:

Python 3.7.0b1 (v3.7.0b1:9561d7f501, Jan 30 2018, 19:10:11) 
[Clang 6.0 (clang-600.0.57)] on darwin


Sample code that fails:

import tkinter as tk
import tkinter.ttk as ttk
app = tk.Tk()
b = ttk.Button(app, text="")


---

Traceback (most recent call last):
  File "", line 1, in 
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/ttk.py",
 line 614, in __init__
Widget.__init__(self, master, "ttk::button", kw)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/ttk.py",
 line 559, in __init__
tkinter.Widget.__init__(self, master, widgetname, kw=kw)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/__init__.py",
 line 2293, in __init__
(widgetName, self._w) + extra + self._options(cnf))
_tkinter.TclError: character U+1f4e9 is above the range (U+-U+) allowed 
by Tcl

--
components: Tkinter, Unicode, macOS
messages: 312279
nosy: Victor Domingos, ezio.melotti, gpolo, ned.deily, ronaldoussoren, 
serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Missing support for Emojis in tkinter
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32846] Deletion of large sets of strings is extra slow

[Luis Pedro Coelho]

Luis Pedro Coelho  added the comment:

I think some of this conversation is going off-topic, but there is no 
disk-swapping in my case.

I realize ours is not a typical setup, but our normal machines have 256GB of 
RAM and the "big memory" compute nodes are >=1TB. Normally, swap is outright 
disabled.

This really is an impressive case study on how much difference cache-locality 
can make.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32854] Add ** Map Unpacking Support for namedtuple

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I wandering if it is worth to add a general function that takes an object and a 
sequence of keys and return a mapping proxy that maps attribute names to values.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32854] Add ** Map Unpacking Support for namedtuple

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Would it be worth adding an example of unpacking such as,
> t(**a._asdict()), or something similar to the documentation ?

There are a myriad of uses for dictionaries and the namedtuple docs don't seem 
like to right place to cover them (**unpacking, str.format_map, str.__mod__, 
etc).  Even the dict and OrderedDict docs don't cover these.

Also, the use of somefunc(**nt._asdict()) isn't a common pattern.  A 
requirement for **unpacking tends to nudge design choices towards an actual 
mapping rather than a tuple or tuple subclass for the underlying data store.  
That is likely why this hasn't come-up before.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22167] iglob() has misleading documentation (does indeed store names internally)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Unfortunately issue25596 didn't change anything about this issue. iglob() still 
stores names (actually DirEntry objects) of all files in a directory before 
starting yielding the first of them. Otherwise we cold exceed the limit of open 
file descriptors.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32852] trace changes sys.argv from list to tuple

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you for your contribution Kyle.

Usually backports a made after merging the original PR. Thanks to bots this is 
done in half-automatical way.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com