[issue36048] Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__

[Oleg Iarygin]

Oleg Iarygin  added the comment:

The reporter gave more details 
():

> Literally this is ok in C++ with Qt:
>
> float x = 2.3, y = 1.1;
> auto p = QPoint(x, y); // QPoint only takes 2 int params.. this works in 
> C++; floats can be always be implicitly converted to int
>
> Same code, more or less, in Python3.10 is now broken:
>
> x = 2.3
> y = 1.1
> p = QPoint(x, y)  # This fails, where previously it worked on every 
> Python version since the age of the dinosaurs...
>
> Note that most of the API for PyQt5 is auto-generated from the function 
> signatures of the C++. So in this case QPoint takes 2 ints for its c'tor 
> (just like in C++).. and breaks on Python 3.10 if given floats, when 
> previously it worked just fine with either ints or floats. This is just 1 
> example. But many codebases that use PyQt5 are hit by breakages like this one 
> now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46020] Optimize long_pow for the common case

[Raymond Hettinger]

New submission from Raymond Hettinger :

The expression 'x * x' is faster than 'x ** 2'.

In Python3.10, the speed difference was enormous.  Due to ceval optimizations, 
the difference in Python3.11 is tighter; however, there is still room for 
improvement.

The code for long_pow() doesn't currently have a fast path for squaring which 
is by far the most important case.

$ python3.10 -m timeit -r 11 -s 'x = 10' 'x ** 2'
100 loops, best of 11: 201 nsec per loop
$ python3.10 -m timeit -r 11 -s 'x = 10' 'x * x'
1000 loops, best of 11: 21.9 nsec per loop

$ python3.11 -m timeit -r 11 -s 'x = 10' 'x ** 2'
1000 loops, best of 11: 32 nsec per loop
$ python3.11 -m timeit -r 11 -s 'x = 10' 'x * x'
2000 loops, best of 11: 17.6 nsec per loop

--
components: Interpreter Core
messages: 408076
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Optimize long_pow for the common case
type: performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44170] ShareableList cannot safely handle multibyte utf-8 characters

[Andrei Kulakov]

Andrei Kulakov  added the comment:

I've confirmed this issue is still present in 3.11.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44170] ShareableList cannot safely handle multibyte utf-8 characters

[Andrei Kulakov]

Andrei Kulakov  added the comment:

We classify 'crash' type as seg faults etc, so changing this to 'behavior' type.

--
nosy: +andrei.avk
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44540] venv: activate.bat fails for venv with special characters in PATH

[Eryk Sun]

Change by Eryk Sun :


Added file: https://bugs.python.org/file50484/deactivate.bat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44540] venv: activate.bat fails for venv with special characters in PATH

[Eryk Sun]

Eryk Sun  added the comment:

There's no parsing problem with delayed expansion (e.g. "!VAR!" instead of 
%VAR%) and for loop variables (e.g. "%%a"). The solution is thus to execute in 
a local scope that enables delayed expansion and command extensions [1]. 

That's easy enough, but then there's the problem of how to evaluate the `set` 
commands in the global scope, without the help of delayed expansion to prevent 
parsing errors. I did some research, and apparently the common trick is to end 
the local scope inside of a `for /f` loop. Then subsequent `set "%%a"` 
statements in iterations of the loop execute in the global scope. 

The final problem is building the command to execute in the local scope, for 
which the `for /f` loop iterates the output lines. `set var` returns a line 
with "var=value", which can be passed back into `set "%%a"`. So I decided to 
build up a sequence of `&` concatenated `set var` and `echo var=` (clear) 
statements in an EXPORTS variable.

I added default values for PROMPT ($P$G) and PATH 
(%SystemRoot%;%SystemRoot%\System32), for when they're not defined. I made sure 
that deactivate.bat does not leave these defaults in place. The original 
values, whether defined or not, should be restored exactly as they were.

All of this adds to the length of the scripts. They're roughly doubled in size. 
But there should be no more problems with an odd number of quotes and/or 
special characters in any of the variables that get set.

---
[1] The activate.bat and deactivate.bat scripts have been naively relying 
on the default enabled state of command extensions, e.g. `for /f`
loops and `if defined` statements.

--
nosy: +eryksun
Added file: https://bugs.python.org/file50483/activate.bat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14965] super() and property inheritance behavior

[Aaron Gallagher]

Aaron Gallagher <_...@habnab.it> added the comment:

I will note, Raymond, that I’ve wanted this for years before discovering
this bpo issue, and I found it because you linked it on Twitter. ;)

On Wed, Dec 8, 2021 at 19:08 Raymond Hettinger 
wrote:

>
> Raymond Hettinger  added the comment:
>
> Another thought:  Given that this tracker issue has been open for a decade
> without resolution, we have evidence that this isn't an important problem
> in practice.
>
> Arguably, people have been better off being nudged in another direction
> toward better design or having been forced to be explicit about what method
> is called and when.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14965] super() and property inheritance behavior

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Another thought:  Given that this tracker issue has been open for a decade 
without resolution, we have evidence that this isn't an important problem in 
practice.

Arguably, people have been better off being nudged in another direction toward 
better design or having been forced to be explicit about what method is called 
and when.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14965] super() and property inheritance behavior

[Raymond Hettinger]

Raymond Hettinger  added the comment:

-0 from me as well.  I don't think this is common or something that should be 
encouraged.  As Andrew points out, "del super().x" doesn't have an obvious 
meaning and it could be regarded as a code smell.

The OP's first example would be an unpleasant API to debug -- it exhibits tight 
coupling between the parent and child class, it has Liskov issues, and it has 
implicit forwarding and indirection through descriptors.  The tight coupling is 
especially problematic because Python's super() isn't guaranteed to call the 
parent class; rather, it can call a sibling class as determined by the MRO 
which cannot be known at the time the class is written.

Another thought is that super() is intentionally not a completely transparent 
proxy.  While an explicit call super().__getitem__(k) works, we've denied 
support for super()[k].  To me, "super().x = 10" and "del super().x" fall in 
the same category.

Looking at the OP's 

Fortunately, it doesn't seem to be a common need to use super() in a property 
setter or deleter to bypass the current class and call setter or deleter in a 
parent class property.  Arguably, this kind of tight coupling isn't good 
design.  The OP's first example would be an unpleasant API to debug.

FWIW,

--
nosy: +rhettinger
versions:  -Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46006] [subinterpreter] _PyUnicode_EqualToASCIIId() issue with subinterpreters

[Inada Naoki]

Inada Naoki  added the comment:

That's too bad.
We can not compare two Unicode by pointer even if both are interned anymore... 
It was a nice optimization.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Short, perfect program to read sentences of webpage

[Cameron Simpson]

On 08Dec2021 23:17, Stefan Ram  wrote:
>  Regexps might have their disadvantages, but when I use them,
>  it is clearer for me to do all the matching with regexps
>  instead of mixing them with Python calls like str.isupper.
>  Therefore, it is helpful for me to have a regexp to match
>  upper and lower case characters separately. Some regexp
>  dialects support "\p{Lu}" and "\p{Ll}" for this.

Aye. I went looking for that in the Python re module docs and could not 
find them. So the comprimise is match any word, then test the word with 
isupper() (or whatever is appropriate).

>  I have not yet incorporated (all) your advice into my code,
>  but I came to the conclusion myself that the repetition of
>  long sequences like r"A-ZÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝ" and
>  not using f strings to insert other strings was especially
>  ugly.

The tricky bit with f-strings and regexps is that \w{3,5} means from 3 
through 5 "word characters". So if you've got those in an f-string 
you're off to double-the-brackets land, a bit like double backslash land 
and non-raw-strings.

Otherwise, yes f-strings are a nice way to compose things.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Odd locale error that has disappeared on reboot.

[Inada Naoki]

On Wed, Dec 8, 2021 at 2:52 AM Chris Green  wrote:
>
>
> At 03:40 last night it suddenly started throwing the following error every
> time it ran:-
>
> Fatal Python error: initfsencoding: Unable to get the locale encoding
> LookupError: unknown encoding: UTF-8
>
> Current thread 0xb6f8db40 (most recent call first):
> Aborted
>
> Running the program from the command line produced the same error.
> Restarting the Pi system has fixed the problem.
>

This error means Python can not find its standard libraries. There are
some possibilities.

* You set the wrong PYTHONHOME
  PYTHONHOME is very rarely useful. It shouldn't be used if you can
not solve this kind of problem.

* Your Python installation is broken.
  Some files are deleted or overwritten. You need to *clean* install
Python again.

Bets,
-- 
Inada Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue39694] Incorrect dictionary unpacking when calling str.format

[Eric V. Smith]

Eric V. Smith  added the comment:

I concur with Raymond.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39694] Incorrect dictionary unpacking when calling str.format

[Raymond Hettinger]

Raymond Hettinger  added the comment:

IMO, there is no actual problem being solved here.  Instead there is just a 
concern that something doesn't feel right.   Given that there is no problem in 
practice, I recommend closing this rather than cluttering docs, tests, or the C 
code for a non-issue.  

The format() method looksup keywords on demand and it can only lookup strings.  
Anything not looked up is ignored.  We have a long history of that working out 
just fine:

 >>> 'The answer is %(answer)s.' % \
 {'answer': 'correct', 10: 'never used'}
 'The answer is correct.'

 >>> 'The answer is {answer}.'.format(
  **{'answer': 'correct', 10: 'never used'})
 'The answer is correct.'

 >>> 'The answer is {answer}.'.format_map(
 {'answer': 'correct', 10: 'never used'})
 'The answer is correct.

One could argue that making any of the above raise an error for a non-string in 
a dict is backwards incompatible and would only serve to break code that is 
already working fine.

I'm going to close this one.  If another core dev feels strongly that this is a 
problem in practice, go ahead and reopen it.  There was a similar change to 
SimpleNamespace but arguably that shouldn't have been done either, but at least 
it had much less risk of breaking existing code that has worked fine for over a 
decade.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34111] python-config breaks when symlinked to another location

[tobik]

tobik  added the comment:

Yes, the bug still exists and all Python versions are affected.

We patch them all in FreeBSD Ports:
https://cgit.freebsd.org/ports/tree/lang/python36/files/patch-Misc__python-config.sh.in
https://cgit.freebsd.org/ports/tree/lang/python37/files/patch-Misc__python-config.sh.in
https://cgit.freebsd.org/ports/tree/lang/python38/files/patch-Misc__python-config.sh.in
https://cgit.freebsd.org/ports/tree/lang/python39/files/patch-Misc__python-config.sh.in
https://cgit.freebsd.org/ports/tree/lang/python310/files/patch-Misc__python-config.sh.in
https://cgit.freebsd.org/ports/tree/lang/python311/files/patch-Misc__python-config.sh.in

--
status: pending -> open
versions: +Python 3.10, Python 3.11, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46019] collections.Counter - Cast list of keys into set to remove iteration over duplicate elements for __le__, __ge__ and __eq__

[Rahul Gupta]

Rahul Gupta  added the comment:

After looking at this again, I agree with you - the key duplication issue seems 
to have gone. Thank you for providing this feedback, it is very helpful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46019] collections.Counter - Cast list of keys into set to remove iteration over duplicate elements for __le__, __ge__ and __eq__

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46019] collections.Counter - Cast list of keys into set to remove iteration over duplicate elements for __le__, __ge__ and __eq__

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> casting c to a set will remove duplicates and allow faster iteration

Sorry, but this doesn't make any sense.  The *c* is either *self* or *other*, 
both of which are instances of Counter which is itself a subclass of dict.  So, 
the input cannot have duplicates keys.

> some minor benchmarks I ran seem to agree.

I'm dubious about the minor benchmarks.  If in fact the effect is real, it is 
merely exploiting an implementation quirk which is tenuous and subject to 
change (the premise would be that converting to a set and looping over a set is 
faster than the native dict iterator for a dict subclass).  Conceptually, it is 
always worse to spend the time and space for first converting to a set.

Besides a speed consideration, there is also a space consideration.  The 
existing code does not use any auxiliary memory.  The proposed code 
unnecessarily builds two new sets and then throws them away.

Thanks for the suggestion, but I am going to decline.  The timings seem 
dubious.  Conceptually, the PR makes the methods do more work.  To the extent 
some timing difference can be observed, it is likely an implementation quirk. 
The PR does not make the code cleaner or clearer, and it loops over a dict 
subclass in an unconventional way.  Also, the PR would have a negative impact 
on memory usage.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46019] collections.Counter - Cast list of keys into set to remove iteration over duplicate elements for __le__, __ge__ and __eq__

[Rahul Gupta]

Change by Rahul Gupta :


--
keywords: +patch
pull_requests: +28222
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46019] collections.Counter - Cast list of keys into set to remove iteration over duplicate elements for __le__, __ge__ and __eq__

[Rahul Gupta]

New submission from Rahul Gupta :

On lines 725, 737 and 749 there is the following code:

'''for c in (self, other) for e in c''' which generates an iterable with all 
the keys in self and other - casting c to a set will remove duplicates and 
allow faster iteration - some minor benchmarks I ran seem to agree.

--
components: Library (Lib)
messages: 408063
nosy: argoop1728
priority: normal
severity: normal
status: open
title: collections.Counter - Cast list of keys into set to remove iteration 
over duplicate elements for __le__,__ge__ and __eq__
type: performance
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Short, perfect program to read sentences of webpage

[MRAB]

On 2021-12-08 23:17, Stefan Ram wrote:

Cameron Simpson  writes:
Instead, consider the \b (word boundary) and \w (word character) 
markers, which will let you break strings up, and then maybe test the 
results with str.isupper().


   Thanks for your comments, most or all of them are
   valid, and I will try to take them into account!

   Regexps might have their disadvantages, but when I use them,
   it is clearer for me to do all the matching with regexps
   instead of mixing them with Python calls like str.isupper.
   Therefore, it is helpful for me to have a regexp to match
   upper and lower case characters separately. Some regexp
   dialects support "\p{Lu}" and "\p{Ll}" for this.

If you want "\p{Lu}" and "\p{Ll}", have a look at the 'regex' module on 
PyPI:


https://pypi.org/project/regex/

[snip]
--
https://mail.python.org/mailman/listinfo/python-list

[issue44289] tarfile.is_tarfile() and tarfile.open() when used with file object may cause tarfile operations to fail

[Andrei Kulakov]

Change by Andrei Kulakov :


--
nosy: +kj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44289] tarfile.is_tarfile() and tarfile.open() when used with file object may cause tarfile operations to fail

[Andrei Kulakov]

Andrei Kulakov  added the comment:

This affects more use cases than just is_tarfile() and getmembers() results.

is_tarfile() calls open() which is the root cause of the issue. Calling open() 
2+ times will also cause the same issue.

In addition to getmembers(), extracting the tar will also silently fail. (and 
possibly other operations).

I've suggested a different fix in the comment on the PR:
https://github.com/python/cpython/pull/26488#issuecomment-989367707

--
nosy: +andrei.avk
title: tarfile.is_tarfile() modifies file object's current position -> 
tarfile.is_tarfile() and tarfile.open() when used with file object may cause 
tarfile operations to fail

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45510] Specialize BINARY_SUBTRACT

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset 5de39f4b412ed5b0f3ed0140c83b2c1f8c707603 by Brandt Bucher in 
branch 'main':
bpo-45510: Check both types when specializing subtraction (GH-29995)
https://github.com/python/cpython/commit/5de39f4b412ed5b0f3ed0140c83b2c1f8c707603


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46008] Prepare runtime/interp/thread state and init for upcoming changes.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +28221
pull_request: https://github.com/python/cpython/pull/29998

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46018] expm1 may incorrectly raise OverflowError on underflow

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +28220
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29997

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46002] py Launcher for Windows with MSYS2

[Zernoxi]

Zernoxi  added the comment:

True, that would work but it not ideal "Unix". I'll closed this since not 
unintended behavior.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46018] expm1 may incorrectly raise OverflowError on underflow

[Steve Dower]

New submission from Steve Dower :

If a C runtime's math functions set errno to ERANGE, we assume it is a valid 
underflow if fabs(result) < 1.0.

However, because expm1 includes a -1.0, it underflows towards -1.0. This fails 
the above check, and so if a runtime's expm1() sets ERANGE we will raise a 
spurious OverflowError.

--
assignee: steve.dower
components: Library (Lib)
messages: 408059
nosy: steve.dower
priority: normal
severity: normal
status: open
title: expm1 may incorrectly raise OverflowError on underflow
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45635] Tidy up error handling in traceback.c / python run.c

[Irit Katriel]

Change by Irit Katriel :


--
pull_requests: +28219
pull_request: https://github.com/python/cpython/pull/29996

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22282] ipaddress module accepts octal formatted IPv4 addresses in IPv6 addresses

[Irit Katriel]

Irit Katriel  added the comment:

This is what I get now on 3.11:


>>> ipaddress.ip_address("::1.0.0.00")
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Users\User\src\cpython\Lib\ipaddress.py", line 54, in ip_address
raise ValueError('%r does not appear to be an IPv4 or IPv6 address' %
^
ValueError: '::1.0.0.00' does not appear to be an IPv4 or IPv6 address

--
nosy: +iritkatriel
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29899] zlib missing when --enable--optimizations option appended

[Irit Katriel]

Irit Katriel  added the comment:

Version 3.3 is no longer supported, and it doesn't look like there is enough 
information here for core devs to understand what happened.

I'll close this, but please create a new issue if you are still having this 
problem with a current version (>= 3.9).

--
nosy: +iritkatriel
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20899] Nested namespace imports do not work inside zip archives

[Irit Katriel]

Irit Katriel  added the comment:

I got the same results as Jonathan.

--
nosy: +iritkatriel
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45510] Specialize BINARY_SUBTRACT

[Brandt Bucher]

Change by Brandt Bucher :


--
pull_requests: +28218
pull_request: https://github.com/python/cpython/pull/29995

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[Python-announce] [RELEASE] Python 3.11.0a3 is available

[Pablo Galindo Salgado]

You can tell that we are slowly getting closer to the first beta as the
number of release blockers that we need to fix on every release starts to
increase [image: :sweat_smile:] But we did it! Thanks to Steve Dower, Ned
Deily, Christian Heimes, Łukasz Langa and Mark Shannon that helped get
things ready for this release :)

Go get the new version here:

https://www.python.org/downloads/release/python-3110a3/

**This is an early developer preview of Python 3.11**

# Major new features of the 3.11 series, compared to 3.10

Python 3.11 is still in development.  This release, 3.11.0a3 is the third
of seven planned alpha releases.

Alpha releases are intended to make it easier to test the current state of
new features and bug fixes and to test the release process.

During the alpha phase, features may be added up until the start of the
beta phase (2022-05-06) and, if necessary, may be modified or deleted up
until the release candidate phase (2022-08-01).  Please keep in mind that
this is a preview release and its use is **not** recommended for production
environments.

Many new features for Python 3.11 are still being planned and written.
Among the new major new features and changes so far:

* [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include
Fine-Grained Error Locations in Tracebacks
* [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups
and except*
* The [Faster Cpython Project](https://github.com/faster-cpython) is
already yielding some exciting results: this version of CPython 3.11 is
~12% faster on the geometric mean of the [PyPerformance benchmarks](
speed.python.org), compared to 3.10.0.
 * Hey, **fellow core developer,** if a feature you find important is
missing from this list, let me know.

The next pre-release of Python 3.11 will be 3.11.0a4, currently scheduled
for Monday, 2022-01-03.

# More resources

* [Online Documentation](https://docs.python.org/3.11/)
* [PEP 664](https://www.python.org/dev/peps/pep-0664/), 3.11 Release
Schedule
* Report bugs at [https://bugs.python.org](https://bugs.python.org).
* [Help fund Python and its community](/psf/donations/).

# And now for something completely different

Rayleigh scattering, named after the nineteenth-century British physicist
Lord Rayleigh is the predominantly elastic scattering of light or other
electromagnetic radiation by particles much smaller than the wavelength of
the radiation. For light frequencies well below the resonance frequency of
the scattering particle, the amount of scattering is inversely proportional
to the fourth power of the wavelength. Rayleigh scattering results from the
electric polarizability of the particles. The oscillating electric field of
a light wave acts on the charges within a particle, causing them to move at
the same frequency. The particle, therefore, becomes a small radiating
dipole whose radiation we see as scattered light. The particles may be
individual atoms or molecules; it can occur when light travels through
transparent solids and liquids but is most prominently seen in gases.

The strong wavelength dependence of the scattering means that shorter
(blue) wavelengths are scattered more strongly than longer (red)
wavelengths. This results in the indirect blue light coming from all
regions of the sky.

# We hope you enjoy those new releases!

Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python
Software Foundation.

Your friendly release team,
Pablo Galindo @pablogsal
Ned Deily @nad
Steve Dower @steve.dower
___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

[RELEASE] Python 3.11.0a3 is available

[Pablo Galindo Salgado]

You can tell that we are slowly getting closer to the first beta as the
number of release blockers that we need to fix on every release starts to
increase [image: :sweat_smile:] But we did it! Thanks to Steve Dower, Ned
Deily, Christian Heimes, Łukasz Langa and Mark Shannon that helped get
things ready for this release :)

Go get the new version here:

https://www.python.org/downloads/release/python-3110a3/

**This is an early developer preview of Python 3.11**

# Major new features of the 3.11 series, compared to 3.10

Python 3.11 is still in development.  This release, 3.11.0a3 is the third
of seven planned alpha releases.

Alpha releases are intended to make it easier to test the current state of
new features and bug fixes and to test the release process.

During the alpha phase, features may be added up until the start of the
beta phase (2022-05-06) and, if necessary, may be modified or deleted up
until the release candidate phase (2022-08-01).  Please keep in mind that
this is a preview release and its use is **not** recommended for production
environments.

Many new features for Python 3.11 are still being planned and written.
Among the new major new features and changes so far:

* [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include
Fine-Grained Error Locations in Tracebacks
* [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups
and except*
* The [Faster Cpython Project](https://github.com/faster-cpython) is
already yielding some exciting results: this version of CPython 3.11 is
~12% faster on the geometric mean of the [PyPerformance benchmarks](
speed.python.org), compared to 3.10.0.
 * Hey, **fellow core developer,** if a feature you find important is
missing from this list, let me know.

The next pre-release of Python 3.11 will be 3.11.0a4, currently scheduled
for Monday, 2022-01-03.

# More resources

* [Online Documentation](https://docs.python.org/3.11/)
* [PEP 664](https://www.python.org/dev/peps/pep-0664/), 3.11 Release
Schedule
* Report bugs at [https://bugs.python.org](https://bugs.python.org).
* [Help fund Python and its community](/psf/donations/).

# And now for something completely different

Rayleigh scattering, named after the nineteenth-century British physicist
Lord Rayleigh is the predominantly elastic scattering of light or other
electromagnetic radiation by particles much smaller than the wavelength of
the radiation. For light frequencies well below the resonance frequency of
the scattering particle, the amount of scattering is inversely proportional
to the fourth power of the wavelength. Rayleigh scattering results from the
electric polarizability of the particles. The oscillating electric field of
a light wave acts on the charges within a particle, causing them to move at
the same frequency. The particle, therefore, becomes a small radiating
dipole whose radiation we see as scattered light. The particles may be
individual atoms or molecules; it can occur when light travels through
transparent solids and liquids but is most prominently seen in gases.

The strong wavelength dependence of the scattering means that shorter
(blue) wavelengths are scattered more strongly than longer (red)
wavelengths. This results in the indirect blue light coming from all
regions of the sky.

# We hope you enjoy those new releases!

Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python
Software Foundation.

Your friendly release team,
Pablo Galindo @pablogsal
Ned Deily @nad
Steve Dower @steve.dower
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue34111] python-config breaks when symlinked to another location

[Irit Katriel]

Irit Katriel  added the comment:

tobik, 3.4/3.5 are no longer maintained. Do you know whether this is impacting 
current versions (>= 3.9)?

--
nosy: +iritkatriel
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36048] Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__

[Oleg Iarygin]

Oleg Iarygin  added the comment:

Here is a report that this change breaks PyQt5 on Fedora:



> [...]
>
> Why do I care? This breaks tons of existing PyQt5 code out there, for 
> example. I wasn't aware of this change to the language until Fedora shipped 
> Python 3.10 and everything broke. So much stuff that uses PyQt5 is broken 
> now. Good job guys!!
>
> [...]

Even though the rest of comment is emotional, we need to check if the problem 
really has place and the PR needs to be retracted until PyQt5 is ported to 
newer Python C API.

--
nosy: +arhadthedev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Short, perfect program to read sentences of webpage

[Peter J. Holzer]

On 2021-12-09 09:42:07 +1100, Cameron Simpson wrote:
> On 08Dec2021 21:41, Stefan Ram  wrote:
> >Julius Hamilton  writes:
> >>This is a really simple program which extracts the text from webpages and
> >>displays them one sentence at a time.
> >
> >  Our teacher said NLTK will not come up until next year, so
> >  I tried to do with regexps. It still has bugs, for example
> >  it can not tell the dot at the end of an abbreviation from
> >  the dot at the end of a sentence!
> 
> This is almost a classic demo of why regexps are a poor tool as a first 
> choice. You can do much with them, but they are cryptic and bug prone.

I don't think that's problem here. The problem is that natural languages
just aren't regular languages. In fact I'm not sure that they fit
anywhere within the Chomsky hierarchy (but if they aren't type-0, that
would be a strong argument against the possibility of human-level AI).

In English, if a sentence ends with an abbreviation you write only a
single dot. So if you look at these two fragments:

For matching strings, numbers, etc. Python provides regular
expressions.

Let's say you want to match strings, numbers, etc. Python provides
regular expresssions for these tasks.

In second case the dot ends a sentence in the first it doesn't. But to
distinguish those cases you need to at least parse the sentences at the
syntax level (which regular expressions can't do), maybe even understand
them semantically.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46017] Tutorial incorrectly refers to skits rather than sketches.

[Steven D'Aprano]

Steven D'Aprano  added the comment:

What difference do you believe there is between skits and sketches?

Definition 1 of skit: "A short comic performance."

https://en.wiktionary.org/wiki/skit#English

Definition 4 of sketch: "A brief, light, or unfinished dramatic, musical, or 
literary work or idea; especially a short, often humorous or satirical scene or 
play, frequently as part of a revue or variety show. Synonym: skit"

https://en.wiktionary.org/wiki/sketch#English


The Pythons themselves sometimes describe their work as "skits" and sometimes 
as "sketches". E.g. quote: "Included are transcripts of Hollywood Bowl Skits..."

http://www.montypython.com/book_Monty%20Python%20Live!/24

while of course it is always "the Dead Parrot sketch".

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

Given that the behavior changes between Python 3.2 (no deadlock) and Python 3.3 
(deadlock), this should be easily bisectable if someone has the right setup to 
build and run Python versions from sources in that range.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Short, perfect program to read sentences of webpage

[Cameron Simpson]

On 08Dec2021 21:41, Stefan Ram  wrote:
>Julius Hamilton  writes:
>>This is a really simple program which extracts the text from webpages and
>>displays them one sentence at a time.
>
>  Our teacher said NLTK will not come up until next year, so
>  I tried to do with regexps. It still has bugs, for example
>  it can not tell the dot at the end of an abbreviation from
>  the dot at the end of a sentence!

This is almost a classic demo of why regexps are a poor tool as a first 
choice. You can do much with them, but they are cryptic and bug prone.

I am not seeking to mock you, but trying to make apparent why regexps 
are to be avoided a lot of the time. They have their place.

You've read the whole re module docs I hope:

https://docs.python.org/3/library/re.html#module-re

>import re
>import urllib.request
>uri = r'''http://example.com/article''' # replace this with your URI!
>request = urllib.request.Request( uri )
>resource = urllib.request.urlopen( request )
>cs = resource.headers.get_content_charset()
>content = resource.read().decode( cs, errors="ignore" )
>content = re.sub( r'''[\r\n\t\s]+''', r''' ''', content )

You're not multiline, so I would recommend a plain raw string:

content = re.sub( r'[\r\n\t\s]+', r' ', content )

No need for \r in the class, \s covers that. From the docs:

  \s
For Unicode (str) patterns:

  Matches Unicode whitespace characters (which includes [ 
  \t\n\r\f\v], and also many other characters, for example the 
  non-breaking spaces mandated by typography rules in many 
  languages). If the ASCII flag is used, only [ \t\n\r\f\v] is 
  matched.

>upper = r"[A-ZÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝ]" # "[\\p{Lu}]"
>lower = r"[a-zµàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ]" # "[\\p{Ll}]"

This is very fragile - you have an arbitrary set of additional uppercase 
characters, almost certainly incomplete, and visually hard to inspect 
for completeness.

Instead, consider the \b (word boundary) and \w (word character) 
markers, which will let you break strings up, and then maybe test the 
results with str.isupper().

>digit = r"[0-9]" #"[\\p{Nd}]"

There's a \d character class for this, covers nondecimal digits too.

>firstwordstart = upper;
>firstwordnext = "(?:[a-zµàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ-])";

Again, an inline arbitrary list of characters. This is fragile.

>wordcharacter = "[A-ZÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝa-zµàáâãäåæçèéêëìíîïð\
>ñòóôõöøùúûüýþÿ0-9-]"

Again inline. Why not construct it?

wordcharacter = upper + lower + digit

but I recommend \w instead, or for this: [\w\d]

>addition = "(?:(?:[']" + wordcharacter + "+)*[']?)?"

As a matter of good practice with regexp strings, use raw quotes:

addition = r"(?:(?:[']" + wordcharacter + r"+)*[']?)?"

even when there are no backslahes.

Seriously, doing this with regexps is difficult. A useful exercise for 
learning regexps, but in the general case not the first tool to reach 
for.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue39694] Incorrect dictionary unpacking when calling str.format

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

In most cases the test is cheap even for large dict (it has constant complexity 
if the dict never contained non-string keys).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Python child process in while True loop blocks parent

[Peter J. Holzer]

On 2021-12-08 18:11:48 +0100, Jen Kris via Python-list wrote:
> To recap, I'm using a pair of named pipes for IPC between C and
> Python.  Python runs as a child process after fork-execv.  The Python
> program continues to run concurrently in a while True loop, and
> responds to requests from C at intervals, and continues to run until
> it receives a signal from C to exit.  C sends signals to Python, then
> waits to receive data back from Python.  My problem was that C was
> blocked when Python started. 
> 
> The solution was twofold:  (1) for Python to run concurrently it must
> be a multiprocessing loop (from the multiprocessing module),

I don't see how this could achieve anything. It starts another (third)
process, but then it just does all the work in that process and just
waits for it. Doing the same work in the original Python process should
have exactly the same effect.

> and (2) Python must terminate its write strings with \n, or read will
> block in C waiting for something that never comes.

That's also strange. You are using os.write in Python and read in C,
both of which shoudn't care about newlines.

> The multiprocessing module sidesteps the GIL; without multiprocessing
> the GIL will block all other threads once Python starts. 

Your Python interpreter runs in a different process than your C code.
There is absolutely no way the GIL could block threads in your C
program. And your Python code doesn't need to use more than one thread
or process.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Short, perfect program to read sentences of webpage

[Jon Ribbens via Python-list]

On 2021-12-08, Julius Hamilton  wrote:
> 1. The HTML extraction is not perfect. It doesn’t produce as clean text as
> I would like. Sometimes random links or tags get left in there. And the
> sentences are sometimes randomly broken by newlines.

Oh. Leaving tags in suggests you are doing this very wrongly. Python
has plenty of open source libraries you can use that will parse the
HTML reliably into tags and text for you.

> 2. Neither is the segmentation perfect. I am currently researching
> developing an optimal segmenter with tools from Spacy.
>
> Brevity is greatly valued. I mean, anyone who can make the program more
> perfect, that’s hugely appreciated. But if someone can do it in very few
> lines of code, that’s also appreciated.

It isn't something that can be done in a few lines of code. There's the
spaces issue you mention for example. Nor is it something that can
necessarily be done just by inspecting the HTML alone. To take a trivial
example:

  powergenitalia  = powergen  italia

but:

  powergenitalia= powergenitalia

but the second with the addition of:

  span { dispaly: block }

is back to "powergen  italia". So you need to parse and apply styles
(including external stylesheets) as well. Potentially you may also need
to execute JavaScript on the page, which means you also need a JavaScript
interpreter and a DOM implementation. Basically you need a complete
browser to do it on general web pages.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue38884] __import__ is not thread-safe on Python 3

[Gregory P. Smith]

Change by Gregory P. Smith :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45711] Simplify the interpreter's (type, val, tb) exception representation

[Irit Katriel]

Change by Irit Katriel :


--
pull_requests: +28217
pull_request: https://github.com/python/cpython/pull/29994

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46016] fcntl module add F_DUP2FD_CLOEXEC

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46016] fcntl module add F_DUP2FD_CLOEXEC

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 267539bff700c2493778c07eeb1642b9584c4826 by David CARLIER in 
branch 'main':
bpo-46016: fcntl module add FreeBSD's F_DUP2FD_CLOEXEC flag support (GH-29993)
https://github.com/python/cpython/commit/267539bff700c2493778c07eeb1642b9584c4826


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46017] Tutorial incorrectly refers to skits rather than sketches.

[Ben Ricketts]

New submission from Ben Ricketts :

https://docs.python.org/3/tutorial/appetite.html

4th paragraph from bottom refers to "skits" where as Monty Python in fact 
perform sketch comedy.It is a minor but important differentiation.

--
assignee: docs@python
components: Documentation
messages: 408049
nosy: benricketts77, docs@python
priority: normal
severity: normal
status: open
title: Tutorial incorrectly refers to skits rather than sketches.
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Python child process in while True loop blocks parent

[Cameron Simpson]

On 08Dec2021 18:11, Jen Kris  wrote:
>Python must terminate its write strings with \n, or read will block in 
>C waiting for something that never comes.

There are two aspects to this:
- if upstream is rding "lines of text" then you need a newline to 
  terminate the lines
- you (probably) should flush the output pipe (Python to C) after the 
  newline

I see you're using file descriptors and os.write() to sent data. This is 
unbuffered, so there is nothing to flush, so you have not encountered 
the second point above.

But if you shift to using a Python file object for your output (eg 
f=os.fdopen(pw)), which would let you use print() or any number of other 
things which do things with Python files) your file object would have a 
buffer and normally that would not be sent to the pipe unless it was 
full.

So your deadlock issue has 2 components:
- you need to terminate your records for upstream (C) to see complete 
  records. Your records are lines, so you need a newline character.
- you need to ensure the whole record has been sent upstream (been 
  written to the pipe). If you use a buffered Python file object for 
  your output, you need to flush it at your synchronisation points or 
  upstream will not receive the buffer contents. That synchronisation 
  point for you is the end of the record.

Hopefully this makes the flow considerations more clear.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46011] Python 3.10 email returns invalid Date: header unchanged.

[Mark Sapiro]

Mark Sapiro  added the comment:

Upon further research I realized this is related to 
https://bugs.python.org/issue30681 and that while there are no message.defects 
the Date: header does have the InvalidDateDefect and its datetime attribute is 
None so I consider this resolved.

--
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39694] Incorrect dictionary unpacking when calling str.format

[Irit Katriel]

Irit Katriel  added the comment:

Reproduced on 3.11.

>>> x = {None: ''}
>>> ''.format(**x)
''
>>> '{x}'.format(**x)
Traceback (most recent call last):
  File "", line 1, in 
KeyError: 'x'

--
components: +Library (Lib)
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40512] [subinterpreters] Meta issue: per-interpreter GIL

[Nathan Jensen]

Change by Nathan Jensen :


--
nosy: +ndjensen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22213] Make pyvenv style virtual environments easier to configure when embedding Python

[Nathan Jensen]

Change by Nathan Jensen :


--
nosy: +ndjensen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46006] [subinterpreter] _PyUnicode_EqualToASCIIId() issue with subinterpreters

[Nathan Jensen]

Change by Nathan Jensen :


--
nosy: +ndjensen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Short, perfect program to read sentences of webpage

[Cameron Simpson]

Assorted remarks inline below:

On 08Dec2021 20:39, Julius Hamilton  wrote:
>deepreader.py:
>
>import sys
>import requests
>import html2text
>import nltk
>
>url = sys.argv[1]

I might spell this:

cmd, url = sys.argv

which enforces exactly one argument. And since you don't care about the 
command name, maybe:

_, url = sys.argv

because "_" is a conventional name for "a value we do not care about".

>sentences = nltk.sent_tokenize(html2text.html2text(requests.get(url).text))

Neat!

># Activate an elementary reader interface for the text
>for index, sentence in enumerate(sentences):

I would be inclined to count from 1, so "enumerate(sentences, 1)".

>  # Print the sentence
>  print(“\n” + str(index) + “/“ + str(len(sentences)) + “: “ + sentence +
>“\n”)

Personally, since print() adds a trailing newline, I would drop the 
final +"\n". If you want an additional blank line, I would put it in the 
input() call below:

>  # Wait for user key-press
>  x = input(“\n> “)

You're not using "x". Just discard input()'s return value:

input("\n> ")

>A lot of refining is possible, and I’d really like to see how some more
>experienced people might handle it.
>
>1. The HTML extraction is not perfect. It doesn’t produce as clean text as
>I would like. Sometimes random links or tags get left in there.

Maybe try beautifulsoup instead of html2text? The module name is "bs4".

>And the
>sentences are sometimes randomly broken by newlines.

I would flatten the newlines. Either the simple:

sentence = sentence.strip().replace("\n", " ")

or maybe better:

sentence = " ".join(sentence.split()

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46016] fcntl module add F_DUP2FD_CLOEXEC

[David CARLIER]

New submission from David CARLIER :

Exposing these specific freebsd constants to the module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44377] Truncated error message of original function while multiprocessing or multithreading

[Andrei Kulakov]

Change by Andrei Kulakov :


--
type:  -> behavior
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10608] [doc] Add a section to Windows FAQ explaining os.symlink

[Eryk Sun]

Eryk Sun  added the comment:

Ingrid's patch misses the requirement to log off after adding a privilege to 
the current account. A new access token is required, from a fresh logon. 

Privileges can also be added to groups, not just to individual user accounts. 
Thus the symlink privilege can be granted to a standard group or well-known 
group, such as "Users" or "Authenticated Users".

Python 3.8+ supports the system's developer mode, which allows unprivileged 
symlink creation. This is just for developer tools, such as Git. Developer mode 
should not be relied on for regular scripts and applications. 

In POSIX, symlinks are widely used to reduce the burden of software 
maintenance. In Windows, however, symlinks are not reliably available, due to 
limited file system support (primarily NTFS, ReFs) and the requirement for a 
privilege that standard users lack by default. Another significant problem in 
Windows is the behavior of the shell API (shell32.dll), which eagerly resolves 
a symlink to its target and thus breaks many use cases for symlinks (e.g. 
virtual environments). Given these problems, using symlinks may actually 
increase the maintenance burden in Windows. Applications have to support a 
fallback path for Windows -- such as hard linking, copying, or launchers -- 
when symlinks either aren't available or for use cases that are unreliable 
(e.g. due to the way symlinks are accessed in the GUI shell).

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Short, perfect program to read sentences of webpage

[MRAB]

On 2021-12-08 19:39, Julius Hamilton wrote:

Hey,

This is something I have been working on for a very long time. It’s one of
the reasons I got into programming at all. I’d really appreciate if people
could input some advice on this.

This is a really simple program which extracts the text from webpages and
displays them one sentence at a time. It’s meant to help you study dense
material, especially documentation, with much more focus and comprehension.
I actually hope it can be of help to people who have difficulty reading. I
know it’s been of use to me at least.

This is a minimally acceptable way to pull it off currently:

deepreader.py:

import sys
import requests
import html2text
import nltk

url = sys.argv[1]

# Get the html, pull out the text, and sentence-segment it in one line of
code

sentences = nltk.sent_tokenize(html2text.html2text(requests.get(url).text))

# Activate an elementary reader interface for the text

for index, sentence in enumerate(sentences):

   # Print the sentence
   print(“\n” + str(index) + “/“ + str(len(sentences)) + “: “ + sentence +
“\n”)


You can shorten that with format strings:

print("\n{}/{}: {}\n".format(index, len(sentences), sentence))

or even:

print(f"\n{index}/{len(sentences)}: {sentence}\n")


   # Wait for user key-press
   x = input(“\n> “)


EOF



That’s it.

A lot of refining is possible, and I’d really like to see how some more
experienced people might handle it.

1. The HTML extraction is not perfect. It doesn’t produce as clean text as
I would like. Sometimes random links or tags get left in there. And the
sentences are sometimes randomly broken by newlines.

2. Neither is the segmentation perfect. I am currently researching
developing an optimal segmenter with tools from Spacy.

Brevity is greatly valued. I mean, anyone who can make the program more
perfect, that’s hugely appreciated. But if someone can do it in very few
lines of code, that’s also appreciated.

Thanks very much,
Julius



--
https://mail.python.org/mailman/listinfo/python-list

[issue44377] Truncated error message of original function while multiprocessing or multithreading

[Andrei Kulakov]

Andrei Kulakov  added the comment:

Prasanth: can you provide a bit more details, are you saying that this line:
https://github.com/python/cpython/blob/2109f7880b65755329a877da3a7f8a362de07350/Lib/multiprocessing/pool.py#L86

.. truncates the exception msg? What is the exact exception type that got 
truncated?

Can you post the exact error output you got here?

--
assignee:  -> andrei.avk
nosy: +andrei.avk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Short, perfect program to read sentences of webpage

[Julius Hamilton]

Hey,

This is something I have been working on for a very long time. It’s one of
the reasons I got into programming at all. I’d really appreciate if people
could input some advice on this.

This is a really simple program which extracts the text from webpages and
displays them one sentence at a time. It’s meant to help you study dense
material, especially documentation, with much more focus and comprehension.
I actually hope it can be of help to people who have difficulty reading. I
know it’s been of use to me at least.

This is a minimally acceptable way to pull it off currently:

deepreader.py:

import sys
import requests
import html2text
import nltk

url = sys.argv[1]

# Get the html, pull out the text, and sentence-segment it in one line of
code

sentences = nltk.sent_tokenize(html2text.html2text(requests.get(url).text))

# Activate an elementary reader interface for the text

for index, sentence in enumerate(sentences):

  # Print the sentence
  print(“\n” + str(index) + “/“ + str(len(sentences)) + “: “ + sentence +
“\n”)

  # Wait for user key-press
  x = input(“\n> “)


EOF



That’s it.

A lot of refining is possible, and I’d really like to see how some more
experienced people might handle it.

1. The HTML extraction is not perfect. It doesn’t produce as clean text as
I would like. Sometimes random links or tags get left in there. And the
sentences are sometimes randomly broken by newlines.

2. Neither is the segmentation perfect. I am currently researching
developing an optimal segmenter with tools from Spacy.

Brevity is greatly valued. I mean, anyone who can make the program more
perfect, that’s hugely appreciated. But if someone can do it in very few
lines of code, that’s also appreciated.

Thanks very much,
Julius
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45711] Simplify the interpreter's (type, val, tb) exception representation

[Irit Katriel]

Irit Katriel  added the comment:


New changeset 2109f7880b65755329a877da3a7f8a362de07350 by Irit Katriel in 
branch 'main':
bpo-45711: Remove unnecessary normalization of exc_info (GH-29922)
https://github.com/python/cpython/commit/2109f7880b65755329a877da3a7f8a362de07350


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46015] Windows venvs do not include DLLs directory

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46015] Windows venvs do not include DLLs directory

[Steve Dower]

Steve Dower  added the comment:


New changeset 7778116c2f573edf320bd55301137a968e4339d8 by Steve Dower in branch 
'main':
bpo-46015: Fixes calculation of sys.path in a venv on Windows (GH-29992)
https://github.com/python/cpython/commit/7778116c2f573edf320bd55301137a968e4339d8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46016] fcntl module add F_DUP2FD_CLOEXEC

[David CARLIER]

Change by David CARLIER :


--
components: FreeBSD
nosy: devnexen, koobs
priority: normal
pull_requests: 28216
severity: normal
status: open
title: fcntl module add F_DUP2FD_CLOEXEC
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45359] TopologicalSorter is not Generic at runtime (but is in typeshed)

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 3cb9731b7e59b0df9a7a9ab6b7746a669958b693 by Jacob Hayes in branch 
'main':
bpo-45359: Support TopologicalSorter type subscript (GH-28714)
https://github.com/python/cpython/commit/3cb9731b7e59b0df9a7a9ab6b7746a669958b693


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45635] Tidy up error handling in traceback.c / python run.c

[Irit Katriel]

Irit Katriel  added the comment:


New changeset f893bb2e01307c92ca19597f44874ecaffe7f06f by Irit Katriel in 
branch 'main':
bpo-45635: refactor print_exception() into smaller functions (GH-29981)
https://github.com/python/cpython/commit/f893bb2e01307c92ca19597f44874ecaffe7f06f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: python problem

[Mats Wichmann]

On 12/8/21 11:18, Larry Warner wrote:

I am new at Python. I have installed Python 3.10.1 and the latest Pycharm.
When I attempt to execute anything via Pycharm or the command line, I
receive a message it can not find Python.

I do not know where Python was loaded or where to find and to update PATH
to the program.


Since you don't mention which operating system you are using or where 
you got your Python from, it's hard for anyone to help.


Do the general notes here help?

https://docs.python.org/3/using/

This problem usually strikes on Windows, where all the commands don't go 
into a small number of directories which are searched by default. 
Following the Windows link in the page above should describe some 
approaches to dealing with that.


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to package a Python command line app?

[Manfred Lotz]

Hi Loris,

On Wed, 08 Dec 2021 15:38:48 +0100
"Loris Bennett"  wrote:

> Hi Manfred,
> 
> Manfred Lotz  writes:
> 
> > The are many possibilities to package a Python app, and I have to
> > admit I am pretty confused.
> >
> > Here is what I have:
> >
> > A Python command line app which requires some packages which are
> > not in the standard library.
> >
> > I am on Linux and like to have an executable (perhaps a zip file
> > with a shebang; whatever) which runs on different Linux systems.
> >
> > Different mean
> > - perhaps different glibc versions
> > - perhaps different Python versions
> >
> > In my specific case this is: 
> > - RedHat 8.4 with Python 3.6.8
> > - Ubuntu 20.04 LTS with Python 3.8.10 
> > - and finally Fedora 33 with Python 3.9.9
> >
> >
> > Is this possible to do? If yes which tool would I use for this?  
> 
> I use poetry[1] on CentOS 7 to handle all the dependencies and create
> a wheel which I then install to a custom directory with pip3.
> 
> You would checkout the repository with your code on the target system,
> start a poetry shell using the Python version required, and then build
> the wheel.  From outside the poetry shell you can set PYTHONUSERBASE
> and then install with pip3.  You then just need to set PYTHONPATH
> appropriately where ever you want to use your software.
> 

In my case it could happen that I do not have access to the target
system but wants to handover the Python app to somebody else. This
person wants just to run it.


> Different Python versions shouldn't be a problem.  If some module
> depends on a specific glibc version, then you might end up in standard
> dependency-hell territory, but you can pin module versions of
> dependencies in poetry, and you could also possibly use different
> branches within your repository to handle that.
> 

I try to avoid using modules which depeng on specific glibc. 

Although, it seems that it doesn't really help for my use case I will
play with poetry to get a better understanding of its capabilities.

-- 
Thanks a lot,
Manfred



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to package a Python command line app?

[Loris Bennett]

Hi Manfred,

Manfred Lotz  writes:

> The are many possibilities to package a Python app, and I have to admit
> I am pretty confused.
>
> Here is what I have:
>
> A Python command line app which requires some packages which are not in
> the standard library.
>
> I am on Linux and like to have an executable (perhaps a zip file with a
> shebang; whatever) which runs on different Linux systems.
>
> Different mean
> - perhaps different glibc versions
> - perhaps different Python versions
>
> In my specific case this is: 
> - RedHat 8.4 with Python 3.6.8
> - Ubuntu 20.04 LTS with Python 3.8.10 
> - and finally Fedora 33 with Python 3.9.9
>
>
> Is this possible to do? If yes which tool would I use for this?

I use poetry[1] on CentOS 7 to handle all the dependencies and create a
wheel which I then install to a custom directory with pip3.

You would checkout the repository with your code on the target system,
start a poetry shell using the Python version required, and then build
the wheel.  From outside the poetry shell you can set PYTHONUSERBASE and
then install with pip3.  You then just need to set PYTHONPATH
appropriately where ever you want to use your software.

Different Python versions shouldn't be a problem.  If some module
depends on a specific glibc version, then you might end up in standard
dependency-hell territory, but you can pin module versions of
dependencies in poetry, and you could also possibly use different
branches within your repository to handle that.

HTH

Loris   

Footnotes: 
[1]  https://python-poetry.org

-- 
This signature is currently under construction.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Odd locale error that has disappeared on reboot.

[Chris Green]

Julio Di Egidio  wrote:
> On 07/12/2021 16:28, Chris Green wrote:
> > I have a very short Python program that runs on one of my Raspberry
> > Pis to collect temperatures from a 1-wire sensor and write them to a
> > database:-
> > 
> >  #!/usr/bin/python3
> >  #
> >  #
> >  # read temperature from 1-wire sensor and store in database with date 
> > and time
> >  #
> >  import sqlite3
> >  import time
> > 
> >  ftxt = 
> > str(open("/sys/bus/w1/devices/28-01204e1e64c3/w1_slave").read(100))
> >  temp = (float(ftxt[ftxt.find("t=") +2:]))/1000
> >  #
> >  #
> >  # insert date, time and temperature into the database
> >  #
> >  tdb = 
> > sqlite3.connect("/home/chris/.cfg/share/temperature/temperature.db")
> >  cr = tdb.cursor()
> >  dt = time.strftime("%Y-%m-%d %H:%M")
> > cr.execute("Insert INTO temperatures (DateTime, Temperature) VALUES(?, 
> round(?, 2))", (dt, temp) 
> >  )
> >  tdb.commit()
> >  tdb.close()
> > 
> > It's run by cron every 10 minutes.
> > 
> > 
> > At 03:40 last night it suddenly started throwing the following error every
> > time it ran:-
> > 
> >  Fatal Python error: initfsencoding: Unable to get the locale encoding
> >  LookupError: unknown encoding: UTF-8
> > 
> >  Current thread 0xb6f8db40 (most recent call first):
> >  Aborted
> > 
> > Running the program from the command line produced the same error.
> > Restarting the Pi system has fixed the problem.
> > 
> > 
> > What could have caused this?  I certainly wasn't around at 03:40! :-)
> > There aren't any automatic updates enabled on the system, the only
> > thing that might have been going on was a backup as that Pi is also
> > my 'NAS' with a big USB drive connected to it.  The backups have been
> > running without problems for more than a year.  Looking at the system
> > logs shows that a backup was started at 03:35 so I suppose that *could*
> > have provoked something but I fail to understand how.
> 
> Since it's a one-off, doesn't sound like a system problem.  The easiest 
> might be that you try-catch that call and retry when needed, and I'd 
> also check that 'ftxt' is what it should be: "external devices" may 
> fail, including when they do produce output...
> 
Well it repeated every ten minutes through the night until I rebooted
the system, so it wasn't really a "one off".  I hasn't repeated since
though.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

How to package a Python command line app?

[Manfred Lotz]

The are many possibilities to package a Python app, and I have to admit
I am pretty confused.

Here is what I have:

A Python command line app which requires some packages which are not in
the standard library.

I am on Linux and like to have an executable (perhaps a zip file with a
shebang; whatever) which runs on different Linux systems.

Different mean
- perhaps different glibc versions
- perhaps different Python versions

In my specific case this is: 
- RedHat 8.4 with Python 3.6.8
- Ubuntu 20.04 LTS with Python 3.8.10 
- and finally Fedora 33 with Python 3.9.9


Is this possible to do? If yes which tool would I use for this?


-- 
Manfred

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: For a hierarchical project, the EXE file generated by "pyinstaller" does not start.

[Mohsen Owzar]

Chris Angelico schrieb am Dienstag, 7. Dezember 2021 um 19:16:54 UTC+1:
> On Wed, Dec 8, 2021 at 4:49 AM Mohsen Owzar  wrote: 
> > *** 
> > GPIOContrl.py 
> > *** 
> > class GPIOControl: 
> > def my_print(self, args): 
> > if print_allowed == 1: 
> > print(args) 
> > 
> > def __init__(self):
> Can't much help with your main question as I don't do Windows, but one 
> small side point: Instead of having a my_print that checks if printing 
> is allowed, you can conditionally replace the print function itself. 
> 
> if not print_allowed: 
> def print(*args, **kwargs): pass 
> 
> ChrisA

Thanks Chris
Your answer didn't help me to solve my problem, but gave me another idea to 
write a conditional print statement.

Regards
Mohsen
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: HTML extraction

[Pieter van Oostrum]

Roland Mueller  writes:

> But isn't bs4 only for SOAP content?
> Can bs4 or lxml cope with HTML code that does not comply with XML as the
> following fragment?
>
> A
> B
> 
>

bs4 can do it, but lxml wants correct XML.

Jupyter console 6.4.0

Python 3.9.9 (main, Nov 16 2021, 07:21:43) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.29.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from bs4 import BeautifulSoup as bs

In [2]: soup = bs('AB')

In [3]: soup.p
Out[3]: A

In [4]: soup.find_all('p')
Out[4]: [A, B]

In [5]: from lxml import etree

In [6]: root = etree.fromstring('AB')
Traceback (most recent call last):

  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py",
 line 3444, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)

  File 
"/var/folders/2l/pdng2d2x18d00m41l6r2ccjrgn/T/ipykernel_96220/3376613260.py",
 line 1, in 
root = etree.fromstring('AB')

  File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring

  File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument

  File "src/lxml/parser.pxi", line 1777, in lxml.etree._parseDoc

  File "src/lxml/parser.pxi", line 1082, in 
lxml.etree._BaseParser._parseUnicodeDoc

  File "src/lxml/parser.pxi", line 615, in 
lxml.etree._ParserContext._handleParseResultDoc

  File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult

  File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError

  File "", line 1
XMLSyntaxError: Premature end of data in tag hr line 1, line 1, column 13
-- 
Pieter van Oostrum 
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

python problem

[Larry Warner]

I am new at Python. I have installed Python 3.10.1 and the latest Pycharm.
When I attempt to execute anything via Pycharm or the command line, I
receive a message it can not find Python.

I do not know where Python was loaded or where to find and to update PATH
to the program.

Larry
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Odd locale error that has disappeared on reboot.

[Dieter Maurer]

Chris Green wrote at 2021-12-7 15:28 +:
>I have a very short Python program that runs on one of my Raspberry
>Pis to collect temperatures from a 1-wire sensor and write them to a
>database:-
> ...
>At 03:40 last night it suddenly started throwing the following error every
>time it ran:-
>
>Fatal Python error: initfsencoding: Unable to get the locale encoding
>LookupError: unknown encoding: UTF-8
>
>Current thread 0xb6f8db40 (most recent call first):
>Aborted
>
>Running the program from the command line produced the same error.
>Restarting the Pi system has fixed the problem.

Python has not its own locale database but uses that of the operating
system. From my point of view, the operating system state seems
to have got corrupted. A restart apparently has ensured a clean state again.
-- 
https://mail.python.org/mailman/listinfo/python-list

[Python-announce] ANN: distlib 0.3.4 released on PyPI

[Vinay Sajip via Python-announce-list]

I've recently released version 0.3.4 of distlib on PyPI [1]. For newcomers,
distlib is a library of packaging functionality which is intended to be
usable as the basis for third-party packaging tools.

The main changes in this release are as follows:

* Fixed #153: Raise warnings in get_distributions() if bad metadata seen, but 
keep
  going.

* Fixed #154: Determine Python versions correctly for Python >= 3.10.

* Updated launcher executables with changes to handle duplication logic.

Code relating to support for Python 2.6 was also removed (support for Python 
2.6 was
dropped in an earlier release, but supporting code wasn't removed until now).

A more detailed change log is available at [2].

Please try it out, and if you find any problems or have any suggestions for 
improvements,
please give some feedback using the issue tracker! [3]

Regards,

Vinay Sajip

[1] https://pypi.org/project/distlib/0.3.4/
[2] https://distlib.readthedocs.io/en/0.3.4/
[3] https://bitbucket.org/pypa/distlib/issues/new

___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

Re: HTML extraction

[Dieter Maurer]

Roland Mueller wrote at 2021-12-7 22:55 +0200:
> ...
>Can bs4 or lxml cope with HTML code that does not comply with XML as the
>following fragment?

`lxml` comes with an HTML parser; that can be configured to check loosely.
-- 
https://mail.python.org/mailman/listinfo/python-list

ANN: distlib 0.3.4 released on PyPI

[Vinay Sajip via Python-list]

I've recently released version 0.3.4 of distlib on PyPI [1]. For newcomers,
distlib is a library of packaging functionality which is intended to be
usable as the basis for third-party packaging tools.

The main changes in this release are as follows:

* Fixed #153: Raise warnings in get_distributions() if bad metadata seen, but 
keep
  going.

* Fixed #154: Determine Python versions correctly for Python >= 3.10.

* Updated launcher executables with changes to handle duplication logic.

Code relating to support for Python 2.6 was also removed (support for Python 
2.6 was
dropped in an earlier release, but supporting code wasn't removed until now).

A more detailed change log is available at [2].

Please try it out, and if you find any problems or have any suggestions for 
improvements,
please give some feedback using the issue tracker! [3]

Regards,

Vinay Sajip

[1] https://pypi.org/project/distlib/0.3.4/
[2] https://distlib.readthedocs.io/en/0.3.4/
[3] https://bitbucket.org/pypa/distlib/issues/new

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python child process in while True loop blocks parent

[Jen Kris via Python-list]

I started this post on November 29, and there have been helpful comments since 
then from Barry Scott, Cameron Simpson, Peter Holzer and Chris Angelico.  
Thanks to all of you.  

I've found a solution that works for my purpose, and I said earlier that I 
would post the solution I found. If anyone has a better solution I would 
appreciate any feedback. 

To recap, I'm using a pair of named pipes for IPC between C and Python.  Python 
runs as a child process after fork-execv.  The Python program continues to run 
concurrently in a while True loop, and responds to requests from C at 
intervals, and continues to run until it receives a signal from C to exit.  C 
sends signals to Python, then waits to receive data back from Python.  My 
problem was that C was blocked when Python started. 

The solution was twofold:  (1) for Python to run concurrently it must be a 
multiprocessing loop (from the multiprocessing module), and (2) Python must 
terminate its write strings with \n, or read will block in C waiting for 
something that never comes.  The multiprocessing module sidesteps the GIL; 
without multiprocessing the GIL will block all other threads once Python 
starts. 

Originally I used epoll() on the pipes.  Cameron Smith and Barry Scott advised 
against epoll, and for this case they are right.  Blocking pipes work here, and 
epoll is too much overhead for watching on a single file descriptor. 

This is the Python code now:

#!/usr/bin/python3
from multiprocessing import Process
import os

print("Python is running")

child_pid = os.getpid()
print('child process id:', child_pid)

def f(a, b):

    print("Python now in function f")

    pr = os.open('/tmp/Pipe_01', os.O_RDONLY)
    print("File Descriptor1 Opened " + str(pr))
    pw = os.open('/tmp/Pipe_02', os.O_WRONLY)
    print("File Descriptor2 Opened " + str(pw))

    while True:

    v = os.read(pr,64)
    print("Python read from pipe pr")
    print(v)

    if v == b'99':
    os.close(pr)
    os.close(pw)
    print("Python is terminating")
    os._exit(os.EX_OK)

    if v != "Send child PID":
    os.write(pw, b"OK message received\n")
    print("Python wrote back")

if __name__ == '__main__':
    a = 0
    b = 0
    p = Process(target=f, args=(a, b,))
    p.start()
    p.join()

The variables a and b are not currently used in the body, but they will be 
later. 

This is the part of the C code that communicates with Python:

    fifo_fd1 = open(fifo_path1, O_WRONLY);
    fifo_fd2 = open(fifo_path2, O_RDONLY);

    status_write = write(fifo_fd1, py_msg_01, sizeof(py_msg_01));
    if (status_write < 0) perror("write");

    status_read = read(fifo_fd2, fifo_readbuf, sizeof(py_msg_01));
    if (status_read < 0) perror("read");
    printf("C received message 1 from Python\n");
    printf("%.*s",(int)buf_len, fifo_readbuf);

    status_write = write(fifo_fd1, py_msg_02, sizeof(py_msg_02));
    if (status_write < 0) perror("write");

    status_read = read(fifo_fd2, fifo_readbuf, sizeof(py_msg_02));
    if (status_read < 0) perror("read");
    printf("C received message 2 from Python\n");
    printf("%.*s",(int)buf_len, fifo_readbuf);

    // Terminate Python multiprocessing
    printf("C is sending exit message to Python\n");
    status_write = write(fifo_fd1, py_msg_03, 2);

    printf("C is closing\n");
    close(fifo_fd1);
    close(fifo_fd2);

Screen output:

Python is running
child process id: 5353
Python now in function f
File Descriptor1 Opened 6
Thread created 0
File Descriptor2 Opened 7
Process ID: 5351
Parent Process ID: 5351
I am the parent
Core joined 0
I am the child
Python read from pipe pr
b'Hello to Python from C\x00\x00'
Python wrote back
C received message 1 from Python
OK message received
Python read from pipe pr
b'Message to Python 2\x00\x00'
Python wrote back
C received message 2 from Python
OK message received
C is sending exit message to Python
C is closing
Python read from pipe pr
b'99'
Python is terminating

Python runs on a separate thread (created with pthreads) because I want the 
flexibility of using this same basic code as a stand-alone .exe, or for a C 
extension from Python called with ctypes.  If I use it as a C extension then I 
want the Python code on a separate thread because I can't have two instances of 
the Python interpreter running on one thread, and one instance will already be 
running on the main thread, albeit "suspended" by the call from ctypes. 

So that's my solution:  (1) Python multiprocessing module; (2) Python strings 
written to the pipe must be terminated with \n. 

Thanks again to all who commented. 



Dec 6, 2021, 13:33 by ba...@barrys-emacs.org:

>
>
>
>> On 6 Dec 2021, at 21:05, Jen Kris <>> jenk...@tutanota.com>> > wrote:
>>
>> Here is what I don't understand from what you said.  "The child process is 
>> created with a single thread—the one that called fork()."  To me that 
>> implies that the thread that called fork() is the same 

[issue45459] Limited API support for Py_buffer

[STINNER Victor]

STINNER Victor  added the comment:

> Py_buffer data = {NULL, NULL};
> The code initializes Py_buffer.buf and Py_buffer.obj as NULL. The remaining 
> fields are whatever random values happens to be on the C stack.

The C language sets other members to 0/NULL with this syntax, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16223] [doc] untokenize returns a string if no encoding token is recognized

[Irit Katriel]

Irit Katriel  added the comment:

The doc has been updated by now:

"It returns bytes, encoded using the ENCODING token, which is the first token 
sequence output by tokenize(). If there is no encoding token in the input, it 
returns a str instead."

https://docs.python.org/3/library/tokenize.html#tokenize.untokenize


The docstring doesn't say this though.

--
components: +Documentation
nosy: +iritkatriel
title: untokenize returns a string if no encoding token is recognized -> [doc] 
untokenize returns a string if no encoding token is recognized
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.3, Python 
3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46015] Windows venvs do not include DLLs directory

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +28215
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29992

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[Petr Viktorin]

Petr Viktorin  added the comment:

The current struct is also likely to continue covering most future uses.
If we decide to add PyBufferEx functions but continue providing the current 
ones (with the current struct), most users won't be affected. (But it'll be a 
bit more work for us than throwing the old API out entirely.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46013] Confusing period in object.__hash__ doc

[Raymond Hettinger]

Raymond Hettinger  added the comment:

A line break would look weird.  Let's add plain English instead.

-   including set, frozenset, and dict. __hash__() should return an integer.
+   including set, frozenset, and dict. The __hash__() method should return an 
integer.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[Christian Heimes]

Christian Heimes  added the comment:

The Py_buffer struct has stayed the same for over a decade and since Python 
2.6.0 and 3.0.0. It is unlikely that it has to be changed in the near future.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29590] Incorrect stack traces when re-entering a generator/coroutine stack via .throw()

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46015] Windows venvs do not include DLLs directory

[Steve Dower]

Steve Dower  added the comment:

+RM

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46015] Windows venvs do not include DLLs directory

[Steve Dower]

New submission from Steve Dower :

Before, a venv would include the DLLs directory in sys.path. Since issue45582, 
it does not, and so anything that relies on a standard library native module 
cannot run.

--
assignee: steve.dower
components: Windows
messages: 408033
nosy: paul.moore, steve.dower, tim.golden, vinay.sajip, zach.ware
priority: release blocker
severity: normal
stage: needs patch
status: open
title: Windows venvs do not include DLLs directory
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45813] Importing asyncio after deleting a coroutine object and before cleaning it up leads to crashing on Python3.11

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45813] Importing asyncio after deleting a coroutine object and before cleaning it up leads to crashing on Python3.11

[Mark Shannon]

Mark Shannon  added the comment:


New changeset d4363d214097b3fca8b7910c2e0e91c8b0873fb2 by Andrew Svetlov in 
branch 'main':
bpo-45813: Drop redundant assertion from frame.clear() (GH-29990)
https://github.com/python/cpython/commit/d4363d214097b3fca8b7910c2e0e91c8b0873fb2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[STINNER Victor]

STINNER Victor  added the comment:

In Python 3.5, I decided to rename the public "PyMemAllocator" structure to 
PyMemAllocatorEx when I added a new "calloc" member. C extensions using 
"PyMemAllocator" fail to build to force developers to set the calloc member.

IMO it's unfortunate to have to rename a structure to force developers to 
update their C code :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[Christian Heimes]

Christian Heimes  added the comment:

I thought of a version field, too. In the end it is going to cause more work 
and trouble than it would benefit us.

Stack-allocated Py_buffer's are typically initialized with

   Py_buffer data = {NULL, NULL};

. The code initializes Py_buffer.buf and Py_buffer.obj as NULL. The remaining 
fields are whatever random values happens to be on the C stack. If we would 
append a version field to the struct, than every project would have to 
initialize the field properly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19888] Three argument type() super call sets __name__ but not __qualname__

[Irit Katriel]

Irit Katriel  added the comment:

This is about documenting a difference between 3.2 and 3.3, so I'm assuming 
it's no longer relevant.

--
nosy: +iritkatriel
resolution:  -> out of date
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15708] PEP 3121, 384 Refactoring applied to socket module

[Irit Katriel]

Irit Katriel  added the comment:

The patch needs to be converted to a github PR.

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15673] PEP 3121, 384 Refactoring applied to testcapi module

[Irit Katriel]

Irit Katriel  added the comment:

The patch needs to be converted to a github PR.

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15389] PEP 3121, 384 refactoring applied to curses module

[Irit Katriel]

Irit Katriel  added the comment:

The patch needs to be converted to a github PR.

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15390] PEP 3121, 384 refactoring applied to _datetime module

[Irit Katriel]

Irit Katriel  added the comment:

The patch needs to be converted to a github PR.

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15651] PEP 3121, 384 refactoring applied to elementtree module

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> duplicate
status: open -> closed
superseder:  -> PEP 3121, 384 refactoring applied to elementtree module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com