[issue31661] Issues with request rate in robotparser

[Nikolay Bogoychev]

Nikolay Bogoychev  added the comment:

Hey Serhiy,

The use of namedtuple was requested specifically at a review, I didn't 
implement it like this initially: https://bugs.python.org/review/16099/#ps6205

I wasn't aware of the performance implications. Could you please explain to me 
the type vs instance in terms of performance (or point me to a resource, a 
quick googling didn't yield anything? How was I supposed to have coded it 
properly?

Cheers,

Nick

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31661] Issues with request rate in robotparser

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

There are issues in implementing support of request rate in robotparser.

req_rate = collections.namedtuple('req_rate',
  'requests seconds')
entry.req_rate = req_rate
entry.req_rate.requests = int(numbers[0])
entry.req_rate.seconds = int(numbers[1])

First, a new namedtuple type is created for every entry. This is slow even with 
recent namedtuple optimizations, and is much slower in 3.6. This wastes a 
memory, since new type is created for every entry. This is definitely wrong, 
since req_rate is set to a namedtuple type instead of namedtuple instance. And 
there is a question why a namedtuple is used here at all. Other classes in this 
module are not namedtuples.

--
components: Library (Lib)
messages: 303508
nosy: XapaJIaMnu, berker.peksag, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Issues with request rate in robotparser
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31660] sys.executable different in os.execv'd python3.6 virtualenv session in python2 vs python3

[Stephen Moore]

New submission from Stephen Moore :

Hi,

I've come across a problem whereby if you do an os.execv to a python3.6 
virtualenv python inside python2.7 vs python3.6 then the resulting python 
session has a different sys.executable.

Where if you os.execv from python2.7 the sys.executable is equal to the 
virtualenv python

Whereas from python3.6 the resulting python session has a sys.executable of the 
system python.

An example of this in play can be found at 
https://gist.github.com/delfick/d750dc83e3b28e90cef8e2bfbd5b175a

Note that I don't see the problem if the virtualenv is based on python2.7 
(https://gist.github.com/delfick/f1ad6872e2614189a7d98f2583ffc564)

--
components: macOS
messages: 303507
nosy: Stephen Moore, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: sys.executable different in os.execv'd python3.6 virtualenv session in 
python2 vs python3
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31659] ssl module should not use textwrap for wrapping PEM format.

[INADA Naoki]

INADA Naoki  added the comment:

For the record, this is part of `import requests`.
`import ssl` took 11 ms and `textwrap` weights 2.4 ms of it.

- ipaddress 2048 us (self 2048 us)
- textwrap 2402 us (self 2402 us)
- _ssl 2744 us (self 2744 us)
- _struct 232 us (self 232 us)
  - struct 411 us (self 180 us)
  - binascii 272 us (self 272 us)
- base64 997 us (self 315 us)
  - ssl 11158 us (self 2969 us)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31659] ssl module should not use textwrap for wrapping PEM format.

[INADA Naoki]

Change by INADA Naoki :


--
keywords: +patch
pull_requests: +3829
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31659] ssl module should not use textwrap for wrapping PEM format.

[INADA Naoki]

Change by INADA Naoki :


--
components: +Library (Lib)
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31659] ssl module should not use textwrap for wrapping PEM format.

[INADA Naoki]

New submission from INADA Naoki :

Current DER_cert_to_PEM_cert() uses textwrap like this:

def DER_cert_to_PEM_cert(der_cert_bytes):
"""Takes a certificate in binary DER format and returns the
PEM version of it as a string."""

f = str(base64.standard_b64encode(der_cert_bytes), 'ASCII', 'strict')
return (PEM_HEADER + '\n' +
textwrap.fill(f, 64) + '\n' +
PEM_FOOTER + '\n')

But textwrap is designed to break lines at word boundary.
So textwrap.fill for base64 text may be slow.

And `import textwrap` is little slow too.

We can use simple slicing instead of textwrap.

--
messages: 303505
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: ssl module should not use textwrap for wrapping PEM format.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27484] Some Examples in Format String Syntax are incorrect or poorly worded

[Rufus V. Smith]

Rufus V. Smith  added the comment:

I happened to run across this in my old email inbox.

It's embarrassing to realize how wrong I was in submitting the report.

Reading it with fresh eyes, your original makes sense.

I don't know what I was thinking.

> On Jul 11, 2016, at 2:10 PM, Raymond Hettinger  wrote:
> 
> 
> Raymond Hettinger added the comment:
> 
> I think all of these read fine as-is though I would change "argument first" 
> to  "first argument" or just "argument".   Adding the word "positional" is a 
> distractor from what the comments are trying to communicate.
> 
> The version numbering is normally done with a directive and outside of the 
> examples.
> 
> --
> nosy: +rhettinger
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21983] segfault in ctypes.cast

[Mark Lawrence]

Change by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation pdf is broken

[Julien Palard]

Julien Palard  added the comment:

One difference I see in the logs of sucessfully building the faq.tex and 
failing to build it is:

   Package hyperref Warning: Token not allowed in a PDF string (Unicode):

Looks like if I remove all non-ascii characters from titles, it builds again.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation pdf is broken

[Julien Palard]

Julien Palard  added the comment:

After an upgrade of my venv, the error is now:

Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
  'faq.aux'
  'faq.out'
  'faq.toc'
Latexmk: Maximum runs of pdflatex reached without getting stable files
Latexmk: All targets (faq.pdf) are up-to-date
Latexmk: Did not finish processing file 'faq.tex':
   'pdflatex' needed too many passes

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation pdf is broken

[Ezio Melotti]

Ezio Melotti  added the comment:

FWIW most of the errors I met while trying to build the pdfs of the main docs 
were caused by the presence of non-latin1 characters.  French should be limited 
to the latin1 range and the error you pasted doesn't seem to be related, 
however that might explain while the Japanese docs are also missing (unless 
this issue got fixed in the meanwhile -- I haven't built the pdfs in a while).

--
nosy: +ezio.melotti
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27494] 2to3 parser failure caused by a comma after a generator expression

[Jakub Stasiak]

Jakub Stasiak  added the comment:

By "forbid" do you mean "forbid in Python" (as in change Python syntax)? I like 
the idea but that seems like a more serious change and 2to3 arguably needs to 
handle code targeting older Python versions anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation pdf is broken

[Julien Palard]

Change by Julien Palard :


--
nosy: +linkid

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 01/10/2017 à 23:33, Daniel Colascione a écrit :
> 
> Huh? Both are very generic.

"Specialized" as in "I didn't expect anyone would want to do such a
thing in pure Python".

> SCM_RIGHTS is "specialized"
> and not supported on all systems, yet it's in stdlib.

Because passing fds between processes was considered useful enough (it's
actually used by multiprocessing itself, for example to implement the
forkserver model).

And regardless, trying to point to other (more or less exotic) features
of the stdlib is not a convincing argument to add a new feature.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation pdf is broken

[Julien Palard]

Julien Palard  added the comment:

Problem happen during pdflatex, I tried a local build and got: 

! Package hyperref Error: Wrong DVI mode driver option `dvipdfmx',
(hyperref)because pdfTeX or LuaTeX is running in PDF mode.

See the hyperref package documentation for explanation.
Type  H   for immediate help.
 ...  
  
l.4362 \ProcessKeyvalOptions{Hyp}
 
? 
! Emergency stop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31630] math.tan has poor accuracy near pi/2 on OpenBSD

[Tim Peters]

Tim Peters  added the comment:

Ah!  So it looks like OpenBSD took its math functions from Sun's fdlibm.  I 
believe wxMaxima does too.  That would certainly explain why they give the same 
answer ;-)

So who's right?  Using "bigfloats" in wxMaxima and feeding its bigfloat tan() 
the exact decimal value corresponding to the binary double denoted by 
1.5707963267948961, it reproduces the result the test is expecting.  Ditto by 
writing my own tan function, using `decimal` with 50 digits, using the 
straightforward series expansions for sin and cos, and then dividing to get tan 
(slow, but without any attempt at "cleverness" it's easy to trust it).

So if fdlibm's author were still active, I'm sure he'd consider this to be a 
bug - it _intended_ to have < 1 ulp error in almost all cases.

It's not argument reduction that's hurting it.  fdlibm stores pi/2 to wy 
more bits than are needed to get the right answer via -1/tan(x - pi/2).  

Regardless, there's nothing Python can do about it short of changing the test.  
As the Python results I pasted before show, the OpenBSD answer _is_ correct for 
some (non-representable in double precision) input value within 1 ulp of the 
actual input value.  So it's a very demanding test to get right.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Daniel Colascione]

Daniel Colascione  added the comment:

On Sun, Oct 1, 2017 at 2:01 PM, Antoine Pitrou 
wrote:

> Perhaps other core developers will disagree with me and agree to include
> (i.e. review, maintain) this functionality.  I simply am not convinced
> it deserves being included, but that's not a veto.

ctypes is a library for operating on native memory and working with native
functions. Performing atomic operations on memory is definitely within its
scope. Why does ctypes include memmove? Why memmove and not
compare-and-exchange? What evidence, if any, would convince you?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Daniel Colascione]

Daniel Colascione  added the comment:

On Sun, Oct 1, 2017 at 2:01 PM, Antoine Pitrou 
wrote:

>
> Antoine Pitrou  added the comment:
>
> Le 01/10/2017 à 21:51, Daniel Colascione a écrit :
> >
> >> That does sound specialized to me :-) Can you give an example of such a
> >> C API?
> >
> > The Linux futex protocol, as described in man futex(7), comes to mind.
> > Maybe you want to manipulate C++ shared_ptr objects --- these objects
> also
> > rely on atomic operations.
>
> That's even more specialized than I expected...
>

Huh? Both are very generic.

>
> > It's still faster than waiting several milliseconds for the GIL.
>
> Are you talking about https://bugs.python.org/issue31653? If so, it's
> just waiting for an appropriate PR to be filed.
>

This is a separate issue. That's about thrashing around less when we take a
lock. This issue is about process A not having to wait on process B to
schedule a thread in order to perform a simple operation on memory that
both processes own.

>
> > I don't understand the opposition to this feature request. It's a trivial
> > amount of code (invoke a compiler intrinsic), makes the API more
> complete,
> > and addresses a real, specific use case as well as some other
> hypothetical
> > use cases.
>
> That's a compiler-dependent compiler intrinsic (or perhaps a whole range
> of them, given there are different widths to cater for), an API wrapping
> it, plus some documentation and tests, that we have to maintain until
> the end of time (at least nominally).
>

It's trivial and easy to support conditionally. SCM_RIGHTS is "specialized"
and not supported on all systems, yet it's in stdlib.

>
> > The standard library already includes a whole web server and HTTP
> > client, a diff engine, various database engines, a facility for parsing
> > email, an NNTP client, a GUI system, and a facility for "determin[ing]
> the
> > type of sound [a] file".
>
> It was determined at the time that the use cases for these justified the
> effort of maintaining them in the stdlib. For a couple of these (such as
> "determining the type of a sound file" or even an NNTP client), I expect
> the decision would be different nowadays :-)
>
> Perhaps other core developers will disagree with me and agree to include
> (i.e. review, maintain) this functionality.  I simply am not convinced
> it deserves being included, but that's not a veto.
>

>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It is not clear to me what API is needed, but I agree with Antoine that ctypes 
doesn't look the appropriate place for it. Maybe in multiprocessing or 
subprocess, or in low-level module providing primitives for multiprocessing or 
subprocess?

--
nosy: +serhiy.storchaka
versions: +Python 3.7 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> Currently, Python exposes "thread identifiers" as unsigned long values across 
> multiple modules:

It happened not so long ago. In 3.6 and earlier "thread identifiers" are signed 
integers. See issue6532, it was massive change. This issue was deferred for 
long time due to the afraid of breaking the word. And it still was not tested 
with third-party code.

>> What happens for code that uses e.g. "%x" to format thread ids?
>It's okay for the opaque object to have a int() conversion.

What happens for C code that uses "%lu" to format thread ids?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31656] Bitwise operations for bytes-type

[Martin Panter]

Martin Panter  added the comment:

There’s already a bug open for this: Issue 19251. Only equal-length strings 
should be supported.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 01/10/2017 à 21:51, Daniel Colascione a écrit :
> 
>> That does sound specialized to me :-) Can you give an example of such a
>> C API?
> 
> The Linux futex protocol, as described in man futex(7), comes to mind.
> Maybe you want to manipulate C++ shared_ptr objects --- these objects also
> rely on atomic operations.

That's even more specialized than I expected...

> It's still faster than waiting several milliseconds for the GIL.

Are you talking about https://bugs.python.org/issue31653? If so, it's
just waiting for an appropriate PR to be filed.

> I don't understand the opposition to this feature request. It's a trivial
> amount of code (invoke a compiler intrinsic), makes the API more complete,
> and addresses a real, specific use case as well as some other hypothetical
> use cases.

That's a compiler-dependent compiler intrinsic (or perhaps a whole range
of them, given there are different widths to cater for), an API wrapping
it, plus some documentation and tests, that we have to maintain until
the end of time (at least nominally).

> The standard library already includes a whole web server and HTTP
> client, a diff engine, various database engines, a facility for parsing
> email, an NNTP client, a GUI system, and a facility for "determin[ing] the
> type of sound [a] file".

It was determined at the time that the use cases for these justified the
effort of maintaining them in the stdlib. For a couple of these (such as
"determining the type of a sound file" or even an NNTP client), I expect
the decision would be different nowadays :-)

Perhaps other core developers will disagree with me and agree to include
(i.e. review, maintain) this functionality.  I simply am not convinced
it deserves being included, but that's not a veto.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31658] xml.sax.parse won't accept path objects

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31658] xml.sax.parse won't accept path objects

[Craig Holmquist]

New submission from Craig Holmquist :

>>> import xml.sax
>>> import pathlib
[...]
>>> xml.sax.parse(pathlib.Path('path/to/file'), handler)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.6/xml/sax/__init__.py", line 33, in parse
parser.parse(source)
  File "/usr/lib/python3.6/xml/sax/expatreader.py", line 105, in parse
source = saxutils.prepare_input_source(source)
  File "/usr/lib/python3.6/xml/sax/saxutils.py", line 355, in 
prepare_input_source
if source.getCharacterStream() is None and source.getByteStream() is None:
AttributeError: 'PosixPath' object has no attribute 'getCharacterStream'

--
components: Library (Lib), XML
messages: 303490
nosy: craigh
priority: normal
severity: normal
status: open
title: xml.sax.parse won't accept path objects
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Daniel Colascione]

Daniel Colascione  added the comment:

On Sun, Oct 1, 2017 at 11:12 AM, Antoine Pitrou 
wrote:

>
> Antoine Pitrou  added the comment:
>
> Le 01/10/2017 à 20:04, Daniel Colascione a écrit :
> >
> > It's not that specialized. You might want atomic updates for coordinating
> > with C APIs that expect callers to have this capability.
>
> That does sound specialized to me :-) Can you give an example of such a
> C API?
>

The Linux futex protocol, as described in man futex(7), comes to mind.
Maybe you want to manipulate C++ shared_ptr objects --- these objects also
rely on atomic operations. For these facilities, you need atomic operations
for *correctness*. Taking a mutex as an alternative is not an option
because there is no C-side mutex to take.

> > You don't need to provide all of those builtins. Users can build them in
> > Python out of atomic-compare-and-exchange. Only compare and exchange
> needs
> > C support. It's not very much code.
>
> I'm assuming you're suggesting to write a loop with an
> atomic-compare-and-exchange.  Bytecode execution in CPython being slow,
> it means you risk a lot more contention (and busy looping) than if the
> primitive was written in C.  Perhaps even a semaphore would be faster :-)
>

It's still faster than waiting several milliseconds for the GIL. Bytecode
isn't *that* slow --- according to ipython, this operation should take a
few hundred nanoseconds. Besides, in a JIT implementation, it'll be as fast
as native code.

>
> > I have little interest in a separate PyPI module. I don't want to have to
> > distribute custom-compiled extension modules.
>
> Understood, but that's not enough of an argument to put something in the
> standard library...
>
> You might want to float your idea on python-ideas to see if you get
> support from people who have a similar need:
> https://mail.python.org/mailman/listinfo/python-ideas
>
>
I don't understand the opposition to this feature request. It's a trivial
amount of code (invoke a compiler intrinsic), makes the API more complete,
and addresses a real, specific use case as well as some other hypothetical
use cases. It costs nothing to add this functionality to the standard
library. The standard library already includes a whole web server and HTTP
client, a diff engine, various database engines, a facility for parsing
email, an NNTP client, a GUI system, and a facility for "determin[ing] the
type of sound [a] file". Why can the standard library include all of these
facilities and not a simple facility for performing a very common kind of
memory operation? Standard library support for this functionality is
essential, as it's not possible to implement in pure Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Daniel Colascione]

Daniel Colascione  added the comment:

On Sun, Oct 1, 2017 at 11:14 AM, Antoine Pitrou 
wrote:

>
> Antoine Pitrou  added the comment:
>
> Note that if there is already a C API to perform atomic ops, you can
> simply use ctypes to invoke that API.  Unfortunately, the aforementioned
> GCC builtins seem to be only available as intrinsics (at least I couldn't
> find a shared library that exposes the __atomic_* functions on my system).
>

Right. I performed the same search. On Windows, at least
InterlockedCompareExchange is exported from kernel32.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Benjamin Peterson]

Benjamin Peterson  added the comment:

On Sun, Oct 1, 2017, at 12:25, Antoine Pitrou wrote:
> Another possibility would be to have a new separate threading API
> returning a "low-level opaque thread handle" that you could pass to
> pthread_kill() and pthread_getcpuclockid().  threading.get_ident() could
> still be used for logging purposes and others.
> 
> A related issue: what if you want to call pthread_kill() on a non-Python
> thread?

C code that provides pthread_t to Python would have to wrap it.

> 
> > I suppose, though you can't usefully use it as an integer. I imagine a
> > comparable and hashable opaque object would be compatible with most
> > code.
> 
> What happens for code that uses e.g. "%x" to format thread ids?

It's okay for the opaque object to have a int() conversion. The
important part is that threading apis don't accept ints.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31634] Consider installing wheel in ensurepip by default

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 01/10/2017 à 21:20, Benjamin Peterson a écrit :
> 
>>> signal.pthread_kill and pdox's proposed time.pthread_getcpuclockid on
>>> https://github.com/python/cpython/pull/3756
>>
>> Given how specialized (and of little actual use) those functions are, I
>> don't think it's much of a problem if they misbehave if you deliberately
>> pass an invalid thread id.
> 
> Okay, so you think this PR is reasonable?

On the principle yes.  I am not interested enough in this feature to
look at the implementation.

Another possibility would be to have a new separate threading API
returning a "low-level opaque thread handle" that you could pass to
pthread_kill() and pthread_getcpuclockid().  threading.get_ident() could
still be used for logging purposes and others.

A related issue: what if you want to call pthread_kill() on a non-Python
thread?

> I suppose, though you can't usefully use it as an integer. I imagine a
> comparable and hashable opaque object would be compatible with most
> code.

What happens for code that uses e.g. "%x" to format thread ids?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Benjamin Peterson]

Benjamin Peterson  added the comment:

On Sun, Oct 1, 2017, at 12:15, Antoine Pitrou wrote:
> 
> Antoine Pitrou  added the comment:
> 
> Le 01/10/2017 à 21:07, Benjamin Peterson a écrit :
> > 
> > signal.pthread_kill and pdox's proposed time.pthread_getcpuclockid on
> > https://github.com/python/cpython/pull/3756
> 
> Given how specialized (and of little actual use) those functions are, I
> don't think it's much of a problem if they misbehave if you deliberately
> pass an invalid thread id.

Okay, so you think this PR is reasonable?

> 
> At least it sounds less of a problem than breaking all code that expects
> a thread id to be an integer.

I suppose, though you can't usefully use it as an integer. I imagine a
comparable and hashable opaque object would be compatible with most
code.

> 
> (btw, pthread_getcpuclockid returns ESRCH on Linux if called with an
> invalid thread id)

More precisely, glibc has a heuristic check for thread id validity,
which dereferences whatever you pass in. It won't save you from
crashing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31630] math.tan has poor accuracy near pi/2 on OpenBSD

[Stefan Krah]

Stefan Krah  added the comment:

OpenBSD reduces to the range [-pi/4,pi/4]:

   https://github.com/openbsd/src/blob/master/lib/libm/src/s_tan.c


According to the man page on i386:

   http://man.openbsd.org/FreeBSD-11.0/math.3

"Argument reduction is not performed accurately for huge arguments, resulting 
in large errors for such arguments to cos(), sin(), and tan()."


pi/2 is not exactly "huge", but it would be interesting if this
occurred on i386.

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 01/10/2017 à 21:07, Benjamin Peterson a écrit :
> 
> signal.pthread_kill and pdox's proposed time.pthread_getcpuclockid on
> https://github.com/python/cpython/pull/3756

Given how specialized (and of little actual use) those functions are, I
don't think it's much of a problem if they misbehave if you deliberately
pass an invalid thread id.

At least it sounds less of a problem than breaking all code that expects
a thread id to be an integer.

(btw, pthread_getcpuclockid returns ESRCH on Linux if called with an
invalid thread id)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Benjamin Peterson]

Benjamin Peterson  added the comment:

On Sun, Oct 1, 2017, at 12:05, Antoine Pitrou wrote:
> 
> Antoine Pitrou  added the comment:
> 
> > You don't think having a function that takes an integer and dereferences it 
> > as memory is a bad idea?
> 
> I'm not sure what you're talking about.  What function is that?

signal.pthread_kill and pdox's proposed time.pthread_getcpuclockid on
https://github.com/python/cpython/pull/3756.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> You don't think having a function that takes an integer and dereferences it 
> as memory is a bad idea?

I'm not sure what you're talking about.  What function is that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Benjamin Peterson]

Benjamin Peterson  added the comment:

You don't think having a function that takes an integer and dereferences it as 
memory is a bad idea?

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31630] math.tan has poor accuracy near pi/2 on OpenBSD

[Tim Peters]

Tim Peters  added the comment:

Of course the relationship is extremely delicate near pi/2.  On my Windows 
Python 3:

>>> import math
>>> (1.5707963267948961).hex()
'0x1.921fb54442d16p+0'
>>> math.tan(float.fromhex('0x1.921fb54442d16p+0')) # what the test expects
1978937966095219.0
>>> math.tan(float.fromhex('0x1.921fb54442d15p+0')) # input 1 ulp less
1374823386397210.2
>>> math.tan(float.fromhex('0x1.921fb54442d17p+0')) # input 1 ulp more
3530114321217157.5

Interestingly, wxMaxima on the same box reproduces the OpenBSD result:

(%i1) tan(1.5707963267948961);
(%o1) 1.978945885716843*10^15

But I don't know how Maxima (or OpenBSD) implement tan().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31630] math.tan has poor accuracy near pi/2 on OpenBSD

[Mark Dickinson]

Mark Dickinson  added the comment:

So actually, the cause of the inaccuracy here could be that OpenBSD _is_ using 
the hardware instructions ...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31630] math.tan has poor accuracy near pi/2 on OpenBSD

[Mark Dickinson]

Mark Dickinson  added the comment:

I'm fairly sure most modern OSs on x86-64 use software implementations of sin, 
cos and tan. At least, I hope so: the x87 hardware versions are notoriously 
inaccurate:

https://randomascii.wordpress.com/2014/10/09/intel-underestimates-error-bounds-by-1-3-quintillion/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Note that if there is already a C API to perform atomic ops, you can simply use 
ctypes to invoke that API.  Unfortunately, the aforementioned GCC builtins seem 
to be only available as intrinsics (at least I couldn't find a shared library 
that exposes the __atomic_* functions on my system).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 01/10/2017 à 20:04, Daniel Colascione a écrit :
> 
> It's not that specialized. You might want atomic updates for coordinating
> with C APIs that expect callers to have this capability.

That does sound specialized to me :-) Can you give an example of such a
C API?

> You don't need to provide all of those builtins. Users can build them in
> Python out of atomic-compare-and-exchange. Only compare and exchange needs
> C support. It's not very much code.

I'm assuming you're suggesting to write a loop with an
atomic-compare-and-exchange.  Bytecode execution in CPython being slow,
it means you risk a lot more contention (and busy looping) than if the
primitive was written in C.  Perhaps even a semaphore would be faster :-)

> I have little interest in a separate PyPI module. I don't want to have to
> distribute custom-compiled extension modules.

Understood, but that's not enough of an argument to put something in the
standard library...

You might want to float your idea on python-ideas to see if you get
support from people who have a similar need:
https://mail.python.org/mailman/listinfo/python-ideas

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Daniel Colascione]

Daniel Colascione  added the comment:

On Oct 1, 2017 10:46 AM, "Antoine Pitrou"  wrote:

Antoine Pitrou  added the comment:

While the use case is reasonable (if specialized),

It's not that specialized. You might want atomic updates for coordinating
with C APIs that expect callers to have this capability.

not sure ctypes is the place to expose such functionality, which can be
quite extensive (see https://gcc.gnu.org/onlinedocs/gcc/_005f_
005fatomic-Builtins.html).

You don't need to provide all of those builtins. Users can build them in
Python out of atomic-compare-and-exchange. Only compare and exchange needs
C support. It's not very much code.

Perhaps

as a separate package on PyPI?

I have little interest in a separate PyPI module. I don't want to have to
distribute custom-compiled extension modules.

--

___
Python tracker 

___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Daniel Colascione]

Daniel Colascione  added the comment:

On Oct 1, 2017 10:19 AM, "Raymond Hettinger"  wrote:

Raymond Hettinger  added the comment:

> Compare-and-exchange is sufficient for avoiding the GIL contention
> I describe above.

If Python objects are involved, it is more complicated than you suggest.

Python objects are not involved. We're talking about memory manipulation on
the same level as ctypes.memmove.

Possibly, multiprocessing can offer a shared counter that creates integer
objects on demand and that offers guaranteed atomic increments and
decrements (as semaphores) do.

Why would it, when ctypes can provide generic functionality?

> one of the nice things about multiprocessing is avoiding
> GIL-introduced latency!

The primary way it achieves this benefit is by avoiding shared state
altogether.

Well, yes, but sometimes shared state is unavoidable, and it's best to
manipulate it as efficiently as possible.

--
nosy: +davin, pitrou, rhettinger

___
Python tracker 

___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31630] math.tan has poor accuracy near pi/2 on OpenBSD

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I would have thought this would be a straight pass-through to the underlying 
hardware FPU tan() instruction so that it would give the same answer for the 
same hardware regardless of O/S.

Perhaps we're seeing a software-only implementation (using CORDIC or somesuch). 
 If so, inputs close to a singularity are the place where the implementation is 
likely to fall apart (the relative error in this example is 1.04001955473).

* 
https://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html
* https://stackoverflow.com/questions/16880376/sin-cos-tan-not-accurate
* 
http://code.activestate.com/recipes/576792-polar-to-rectangular-conversions-using-cordic/

--
nosy: +rhettinger, tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Antoine Pitrou]

Antoine Pitrou  added the comment:

While the use case is reasonable (if specialized), I'm not sure ctypes is the 
place to expose such functionality, which can be quite extensive (see 
https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html). Perhaps as 
a separate package on PyPI?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> On a system with pthreads, the thread_id that Python provides is merely 
> pthread_t casted to unsigned long. This works today, but is in violation of 
> the standard, and could break on systems with exotic pthread_t.

I don't think that follows.  Even if pthread_t is not an integer, it does have 
a binary representation that can be trivially converted to a arbitrary-sized 
Python int in Python-facing APIs such as threading.get_ident().

(and similarly, of course, for converting in the other direction e.g. for 
pthread_kill())

> There is also the ability to introduce undefined behavior, such as sending a 
> signal to an invalid thread id:

I wouldn't worry much about pthread_kill(), a little-used feature.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Compare-and-exchange is sufficient for avoiding the GIL contention
> I describe above.

If Python objects are involved, it is more complicated than you suggest.  
Possibly, multiprocessing can offer a shared counter that creates integer 
objects on demand and that offers guaranteed atomic increments and decrements 
(as semaphores) do.

> one of the nice things about multiprocessing is avoiding 
> GIL-introduced latency!

The primary way it achieves this benefit is by avoiding shared state altogether.

--
nosy: +davin, pitrou, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21983] segfault in ctypes.cast

[Oren Milman]

Oren Milman  added the comment:

IMHO, Lib/ctypes/test/test_cast.py is the relevant test.

Mark, do you still wish to provide a fix for that?
(Otherwise, i would be happy to open a PR.)

--
nosy: +Oren Milman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31622] Make threading.get_ident() return an opaque type

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +ncoghlan, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31655] SimpleNamespace accepts non-string keyword names

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Wouldn't be better to add also the check that keyword names
> for SimpleNamespace constructor are strings?

Yes.

> I don't know how classify this issue, as a bug or an enhancement.

It is arguably a bug fix.  +0 for putting this in Python 3.6.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31553] Extend json.tool to handle jsonlines (with a flag)

[Jeffrey Rackauckas]

Change by Jeffrey Rackauckas :


--
keywords: +patch
pull_requests: +3828
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31656] Bitwise operations for bytes-type

[R. David Murray]

R. David Murray  added the comment:

What happens when you apply a bitwise operation to two bytes objects of unequal 
length?

Since the answer to that question is not obvious, we simply don't support the 
operation.

If you want to make a proposal for this the python-ideas mailing list would be 
the appropriate forum.

--
nosy: +r.david.murray
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31657] unit test for optimization levels does not cover __debug__ case

[diana]

Change by diana :


--
keywords: +patch
pull_requests: +3827
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31657] unit test for optimization levels does not cover __debug__ case

[diana]

New submission from diana :

There are currently three supported optimization levels (0, 1, and 2). Briefly 
summarized, they do the following.

0: no optimizations
1: remove assert statements and __debug__ blocks
2: remove docstrings, assert statements, and __debug__ blocks

The current compile() tests for optimization levels in Lib/test/test_builtin.py 
covers the assert and docstring cases, but it doesn't test that __debug__ code 
blocks are included or excluded based on the optimization level.

For example, if you change Python/compile.c to always include __debug__ blocks 
regardless of the optimization level, the existing compile() tests will 
continue to pass.

$ git diff Python/compile.c
diff --git a/Python/compile.c b/Python/compile.c
index 280ddc39e3..d65df098bb 100644
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -4143,7 +4143,7 @@ expr_constant(struct compiler *c, expr_ty e)
 /* optimize away names that can't be reassigned */
 id = PyUnicode_AsUTF8(e->v.Name.id);
 if (id && strcmp(id, "__debug__") == 0)
-return !c->c_optimize;
+return 1;
 return -1;
 case NameConstant_kind: {
 PyObject *o = e->v.NameConstant.value;

--
messages: 303465
nosy: diana
priority: normal
severity: normal
status: open
title: unit test for optimization levels does not cover __debug__ case
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31656] Bitwise operations for bytes-type

[Matthias Gilch]

New submission from Matthias Gilch :

I've seen that the bytes type does not support bitwise operations, but I think 
users would expect that those operations will work.

--
components: Interpreter Core
messages: 303464
nosy: MGilch
priority: normal
severity: normal
status: open
title: Bitwise operations for bytes-type
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31655] SimpleNamespace accepts non-string keyword names

[R. David Murray]

R. David Murray  added the comment:

Sorry, not broken, it just raises a type error, which is appropriate :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31654] ctypes should support atomic operations

[SilentGhost]

Change by SilentGhost :


--
nosy: +amaury.forgeotdarc, belopolsky, meador.inge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Steve Dower]

Steve Dower  added the comment:

I don't really care, since you're comparing multiple unsupported technologies, 
but my guess is that the egg install injects the library much earlier on 
sys.path, which avoids most of the relatively expensive file system scan.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31655] SimpleNamespace accepts non-string keyword names

[R. David Murray]

Change by R. David Murray :


--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31655] SimpleNamespace accepts non-string keyword names

[R. David Murray]

R. David Murray  added the comment:

Well, it would be an "unenhancement" (reducing functionality :).  

What I think it would be classed as is an API bug fix, and those generally can 
be made only in feature releases.  I can imagine code depending on this 
ability, since we have had code in the wild that depends on the ability to put 
non-string keys in an object's __dict__ even before SimpleNamespace existed.

(By the way, dir is broken if an object's __dict__ contains non-string keys.)

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31478] assertion failure in random.seed() in case the seed argument has a bad __abs__() method

[Oren Milman]

Change by Oren Milman :


--
pull_requests: +3826

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Using nested _PyMem_DebugRawRealloc() looks suspicions to me. This may be a bug.

But even without nested _PyMem_DebugRawRealloc() writing to the extra memory 
after using realloc() looks wrong to me. This can break other non-trivial 
system allocators which write an information past the allocated block. This can 
cause a segfault if unused memory pages are returned to OS.

After creating the PR I have found that it literally restores the code of 2.7 
and 3.3. 3.4 and later contain this bug. The bug looks enough serious to me for 
fixing it in 3.4 and 3.5.

--
nosy: +larry
title: Crash in _PyUnicode_DecodeUnicodeEscape on OpenBSD -> Writing in freed 
memory in _PyMem_DebugRawRealloc() after shrinking a memory block
versions: +Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Crash in _PyUnicode_DecodeUnicodeEscape on OpenBSD

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +3825
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31555] Windows pyd slower when not loaded via load_dynamic

[Safihre]

Safihre  added the comment:

If you know the problem, would you also know the solution?
What is the difference in how they generate sys.path entries and how does this 
affect performance during execution of the module function?
Just want to understand what is going on :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31308] forkserver process isn't re-launched if it died

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Looks like this will not make into 3.6.3.

I'd like to sum up the discussion a bit:
- I think at the very least least we must protect the forkserver against 
SIGINT, like the semaphore tracker already is
- I think it's beneficial to also restart it on demand if necessary
- I can't think of a use case where the user asks "no, please don't restart the 
forkserver on demand" as there is no way to start it manually anyway, and not 
restarting it breaks functionality

Also, bpo-31310 is a similar issue for the semaphore tracker.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Crash in _PyUnicode_DecodeUnicodeEscape on OpenBSD

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
priority: normal -> critical

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Crash in _PyUnicode_DecodeUnicodeEscape on OpenBSD

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

There several bugs in the memory allocator.

Incorrectly detected the case when realloc() resizes a memory block in-place. 
Wrong address is used for filling the extra memory with DEADBYTE.

-if (q == oldq && nbytes < original_nbytes) {
+if (q == oldq - 2*SST && nbytes < original_nbytes) {
 /* shrinking:  mark old extra memory dead */
-memset(q + nbytes, DEADBYTE, original_nbytes - nbytes);
+memset(q + 2*SST + nbytes, DEADBYTE, original_nbytes - nbytes);
 }

But fixing this exposes other problem. _PyMem_DebugRawRealloc() is called 
recursively. _PyMem_DebugRawRealloc calls api->alloc.realloc which is 
_PyMem_DebugRawRealloc. There are two nested debug allocators. The block is 
nested in other block, both have their own header and footer.

|header1|header2|--|footer2|footer1|

_PyMem_DebugRawRealloc fills the extra memory with DEADBYTE.

|header|---..unused..|footer|
|header|---|footer|X|

But in case of nested _PyMem_DebugRawRealloc's, the outer one (which 
reallocates the inner block), overwrites the footer of the outer block.

|header1|header2|..unused..|footer2|footer1|
|header1|header2|..unused|footer1|X| after inner 
realloc
|header1|header2||footer2|Y|XXX| after outher 
realloc

XXX are DEADBYTEs written by the inner allocator, YYY are DEADBYTEs written by 
the outer allocator.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[Antoine Pitrou]

Change by Antoine Pitrou :


--
type: enhancement -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31465] Allow _PyType_Lookup() to raise exceptions

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests:  -3632

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31465] Allow _PyType_Lookup() to raise exceptions

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I'm positive to this change in general, but this is too complex and delicate 
code. In needs careful reviewing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31336] Speed up _PyType_Lookup() for class creation

[Stefan Behnel]

Stefan Behnel  added the comment:

Thanks for the reviews, and thank you for merging it, Serhiy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31336] Speed up _PyType_Lookup() for class creation

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31336] Speed up _PyType_Lookup() for class creation

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you for your contribution Stefan! I had doubts about your initial (much 
simpler) changes, but changed my mind.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31336] Speed up _PyType_Lookup() for class creation

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 2102c789035ccacbac4362589402ac68baa2cd29 by Serhiy Storchaka 
(scoder) in branch 'master':
bpo-31336: Speed up type creation. (#3279)
https://github.com/python/cpython/commit/2102c789035ccacbac4362589402ac68baa2cd29


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31651] io.FileIO cannot write more than 2GB (-4096) bytes??? must be documented (if not fixed)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> On Linux, write() (and similar system calls) will transfer at most 0x7000 
> (2,147,479,552) bytes, returning the number of bytes 
> actually transferred.  (This is true on both 32-bit and 64-bit 
> systems.)

This is platform-depending limitation. It can be be changed in future. In 
addition, there are other causes of writing not all data (see `man 2 write`).

> Also, it might be nice to add a note on top, that this module is for 'low 
> level' IO interface, and that it is recommended to use regular file type for 
> typical file operations (not io.FileIO) to avoid necessity of dealing 
> limitations such as the one mentioned.

This is not true for the module overall. And this is already documented for 
io.RawIOBase:

"""
Raw binary I/O typically provides low-level access to an underlying OS device 
or API, and does not try to encapsulate it in high-level primitives (this is 
left to Buffered I/O and Text I/O, described later in this page).
"""

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com