Re: [Python-Dev] RFC: PEP 587 "Python Initialization Configuration": 3rd version

[Gregory Szorc]

On 5/20/2019 4:09 AM, Victor Stinner wrote:
> Hi Gregory,
> 
> IMHO your remarks are not directly related to the PEP 587 and can be
> addressed in parallel.
> 
>> It sounds like PyOxidizer and Hermetic Python are on the same page and
>> we're working towards a more official solution. But I want to make sure
>> by explicitly stating what PyOxidizer is doing.
>>
>> Essentially, to facilitate in-memory import, we need to register a
>> custom sys.meta_path importer *before* any file-based imports are
>> attempted. (...)
>> I /think/ providing a 2-phase
>> initialization that stops between _Py_InitializeCore() and
>> _Py_InitializeMainInterpreter() would get the job done for PyOxidizer
>> today. (...)
> 
> Extract of PEP 587: "This extracts a subset of the API design from the
> PEP 432 development and refactoring work that is now considered
> sufficiently stable to make public (allowing 3rd party embedding
> applications access to the same configuration APIs that the native
> CPython CLI is now using)."
> 
> We know that my PEP 587 is incomplete, but the work will continue in
> Python 3.9 to support your use case.
> 
> The PEP 587 introduces an experimental separation between "core" and
> "main" initialization phases. PyConfig._init_main=0 stops at the
> "core" phase, then you are free to run C and Python,
> _Py_InitializeMain() finishes the Python initialization ("main"
> phase).
> 
> 
>>> In the "Isolate Python" section, I suggest to set the "isolated"
>>> parameter to 1 which imply setting user_site_directory to 0. So
>>> sys.path isn't modified afterwards. What you pass to PyConfig is what
>>> you get in sys.path in this case.
>>
>> Regarding site.py, I agree it is problematic for embedding scenarios.
>> Some features of site.py can be useful. Others aren't. It would be
>> useful to have more granular control over which bits of site.run are
>> run. My naive suggestion would be to add individual flags to control
>> which functions site.py:main() runs. That way embedders can cherry-pick
>> site.py features without having to manually import the module and call
>> functions within. That feels much more robust for long-term maintainability.
> 
> I agree that more work can be done on the site module. IMHO core
> features which are needed by everybody should be done before calling
> site. Maybe using a frozen "presite" module or whatever. I would be
> interested to make possible to use Python for most cases without the
> site module.
> 
> 
>> Regarding Python calling exit(), this is problematic for embedding
>> scenarios.
> 
> I am working on that. I fixed dozens of functions. For example,
> Py_RunMain() should not longer exit if there is an uncaught SystemExit
> when calling PyErr_Print(). SystemExit is now handled separately
> before calling PyErr_Print(). The work is not done, but it should be
> way better than Python 3.6 and 3.7 state.
> 
> 
>> This thread called attention to exit() during interpreter
>> initialization. But it is also a problem elsewhere. For example,
>> PyErr_PrintEx() will call Py_Exit() if the exception is a SystemExit.
>> There's definitely room to improve the exception handling mechanism to
>> give embedders better control when SystemExit is raised. As it stands,
>> we need to check for SystemExit manually and reimplement
>> _Py_HandleSystemExit() to emulate its behavior for e.g. exception value
>> handling (fun fact: you can pass non-None, non-integer values to
>> sys.exit/SystemExit).
> 
> I don't know well these functions, maybe new functions are needed. It
> can be done without/outside the PEP 587.
> 
> 
>> Having a more efficient member lookup for BuiltinImporter and
>> FrozenImporter might shave off a millisecond or two from startup. This
>> would require some kind of derived data structure. (...)
> 
> I don't think that the structures/API to define frozen/builtin modules
> has to change. We can convert these lists into an hash table during
> the initialization of the importlib module.
> 
> I'm not saying that the current API is perfect, just that IMHO it can
> be solved without the API.
> 
> 
>> Unfortunately, as long as there is a global data structure that can be 
>> mutated any time
>> (the API contract doesn't prohibit modifying these global arrays after
>> initialization), you would need to check for "cache invalidation" on
>> every lookup, undermining performance benefits.
> 
> Do you really expect an application modifying these lists dynamically?

At this time, not really. But the mark of good API design IMO is that
its flexibility empowers new and novel ideas and ways of doing things.
Bad APIs (including the use of global variables) inhibit flexibility and
constrain creativity and advanced usage.

For this particular item, I could see some potential uses in processes
hosting multiple, independent interpreters. Maybe you want to give each
interpreter its own set of modules. That's not something you see today
because of the GIL and all the other global state in CPython. But w

Re: [Python-Dev] Expected stability of PyCode_New() and types.CodeType() signatures

[Simon Cross]

On Sat, Jun 1, 2019 at 12:29 AM Terry Reedy  wrote:

> We try not to.  There is already too much work.
> ...
> That was removed from the PR before it was merged.
> ...
> That was merged.
> ...
> Another chunk was added to What's New.
>

Woot to all of these! :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Expected stability of PyCode_New() and types.CodeType() signatures

[Terry Reedy]

On 5/31/2019 5:28 PM, Simon Cross wrote:
As the maintainer of Genshi, one the libraries affected by the CodeType 
and similar changes, I thought I could add a users perspective to the 
discussion:


Genshi is a templating engine that parses Python code from inside its 
templates. It supports Python 2.6+, 3.2+, pypy2 and pypy3. It parses 
Python using compile(...) and then walks the AST to, for example, 
translate variable lookups into template context lookups.


Pretty much every major release of Python has broken Genshi's Python 
parsing in some way. While on the one hand this is a bit annoying, I 
also don't want Python to stop evolving where it makes sense.


My requests to core developers are largely pragmatic:

* Try not to change things unless there's a good reason to (i.e. it 
makes Python better).


We try not to.  There is already too much work.

* Don't try declare that these things shouldn't be used (there is not 
much I can do about that now).


That was removed from the PR before it was merged.


* Do warn people that these things evolve with the language.


That was merged.

* If changes do happen, try make them visible and give a clear 
description of what has changed.


Another chunk was added to What's New.

Also many thanks to the core developers who've submitted patches to 
update Genshi in the past -- that was awesome of you.


The new CodeType.replace will remove some potential sources of breakages 
in the future, so thank you very much for adding that.



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 590: vectorcall without tp_call

[Steve Dower]

On 29May2019 1311, Petr Viktorin wrote:

On 5/29/19 3:36 PM, Jeroen Demeyer wrote:

On 2019-05-29 15:29, Petr Viktorin wrote:

That sounds like a good idea for PyType_FromSpec.


I don't think we're planning to support vectorcall in PyType_FromSpec 
for now. That's maybe for 3.9 when vectorcall is no longer provisional.



For static types I either wouldn't bother at all, or only check in debug
builds and fail with Py_FatalError.


So basically an assert(...)?


Yes. That's the usual way to let C extension authors know they did 
something wrong.


It's the usual way to let core developers know they did something wrong :)

I don't want to reignite the whole debug/release extension build debate 
again, but I'll point out that it's very common to develop extension 
modules against the release binaries on Windows, which means you won't 
see asserts coming from core (because they've been optimised out). So if 
this is something we're going to detect at runtime regardless of build, 
using Py_FatalError/abort or raising a SystemError is preferable. 
Otherwise we'll be forcing users to debug a segfault.


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Expected stability of PyCode_New() and types.CodeType() signatures

[Simon Cross]

As the maintainer of Genshi, one the libraries affected by the CodeType and
similar changes, I thought I could add a users perspective to the
discussion:

Genshi is a templating engine that parses Python code from inside its
templates. It supports Python 2.6+, 3.2+, pypy2 and pypy3. It parses Python
using compile(...) and then walks the AST to, for example, translate
variable lookups into template context lookups.

Pretty much every major release of Python has broken Genshi's Python
parsing in some way. While on the one hand this is a bit annoying, I also
don't want Python to stop evolving where it makes sense.

My requests to core developers are largely pragmatic:

* Try not to change things unless there's a good reason to (i.e. it makes
Python better).
* Don't try declare that these things shouldn't be used (there is not much
I can do about that now).
* Do warn people that these things evolve with the language.
* If changes do happen, try make them visible and give a clear description
of what has changed.

Also many thanks to the core developers who've submitted patches to update
Genshi in the past -- that was awesome of you.

The new CodeType.replace will remove some potential sources of breakages in
the future, so thank you very much for adding that.

Schiavo
Simon
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

[Brett Cannon]

On Fri, May 31, 2019 at 11:12 AM Tim Peters  wrote:

> [Tim]
> >> I'm keen to get feedback on this before merging the PR, because this
> >> case is so very much larger than anything I've ever tried that I'm
> >> wary that there may be more than one "surprise" lurking here. ...
>
> [Inada Naoki ]
> > I started r5a.4xlarge EC2 instance and started arena.py.
> > I will post the result in next 12 hours.
>
> Thank you!  Wrapping this up, Inada attached the results to the bug
> report.  For the OP's original big case, the time to reclaim the tree
> storage dropped from nearly 5 hours to about 70 seconds.  The time to
> build the tree didn't materially change (it was a bit faster after the
> patch, but offhand didn't look significantly faster to me).
>
> Since I called this a "design flaw" rather than "a bug", I'm not
> inclined to backport it.  If someone else thinks it should be, that's
> up to them.  I'm _assuming_ Github won't decide to backport it on its
> own - but maybe I'm wrong (Github workflow is still pretty magical to
> me).
>

Backports only happen if you add the appropriate labels to the PR itself
(and if that wasn't clear from the devguide then please open an issue so we
can fix that oversight).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 595: Improving bugs.python.org

[Carol Willing]

Nathaniel Smith wrote:


On Fri, May 31, 2019 at 11:39 AM Barry Warsaw  wrote:



On May 31, 2019, at 01:22, Antoine Pitrou  wrote:



I second this.

There are currently ~7000 bugs open on bugs.python.org.  The Web UI
makes a good job of actually being able to navigate through these bugs,
search through them, etc.

Did the Steering Council conduct a usability study of Github Issues
with those ~7000 bugs open?  If not, then I think the acceptance of
migrating to Github is a rushed job.  Please reconsider.












Thanks for your feedback Antoine.

This is a tricky issue, with many factors and tradeoffs to consider.  
I really appreciate Ezio and Berker working on PEP 595, so we can put 
all these issues on the table.


I think one of the most important tradeoffs is balancing the needs of 
existing developers (those who actively triage bugs today), and 
future contributors.  But this and other UX issues are difficult to 
compare on our actual data right now.  I fully expect that just as 
with the switch to git, we’ll do lots of sample imports and 
prototyping to ensure that GitHub issues will actually work for us 
(given our unique requirements), and to help achieve the proper 
balance.  It does us no good to switch if we just anger all the 
existing devs.


IMHO, if the switch to GH doesn’t improve our workflow, then it 
definitely warrants a reevaluation.  I think things will be better, 
but let’s prove it.



Perhaps we should put an explicit step on the transition plan, after
the prototyping, that's "gather feedback from prototypes, re-evaluate,
make final go/no-go decision"? I assume we'll want to do that anyway,
and having it formally written down might reassure people. It might
also encourage more people to actually try out the prototypes if we
make it very clear that they're going to be asked for feedback.

-n




As Barry mentioned, there are many considerations.

- Will GitHub provide a similar experience as bpo? Can the features 
valued by existing developers be served in a suitable way by GitHub issues?
- What opportunity costs exist for remaining on bpo? Planning, 
reporting, integration with services, and accessibility are some.
- Chicken and egg question: Are there 7000 open issues because the 
existing workflow with bpo inhibits acting on open issues with patches?


Antoine, as for large projects on GitHub with many issues, TypeScript is 
a reasonable proxy with close to 4000 open issues yet only 126 open PRs. 
There are some nice planning things that have been done in the repo 
https://github.com/microsoft/TypeScript/issues as well as good 
documentation around their process and workflow.


Thanks to Ezio and Berker for raising points in PEP 595.

- Carol
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 595: Improving bugs.python.org

[Nathaniel Smith]

On Fri, May 31, 2019 at 11:39 AM Barry Warsaw  wrote:
>
> On May 31, 2019, at 01:22, Antoine Pitrou  wrote:
>
> > I second this.
> >
> > There are currently ~7000 bugs open on bugs.python.org.  The Web UI
> > makes a good job of actually being able to navigate through these bugs,
> > search through them, etc.
> >
> > Did the Steering Council conduct a usability study of Github Issues
> > with those ~7000 bugs open?  If not, then I think the acceptance of
> > migrating to Github is a rushed job.  Please reconsider.
>
> Thanks for your feedback Antoine.
>
> This is a tricky issue, with many factors and tradeoffs to consider.  I 
> really appreciate Ezio and Berker working on PEP 595, so we can put all these 
> issues on the table.
>
> I think one of the most important tradeoffs is balancing the needs of 
> existing developers (those who actively triage bugs today), and future 
> contributors.  But this and other UX issues are difficult to compare on our 
> actual data right now.  I fully expect that just as with the switch to git, 
> we’ll do lots of sample imports and prototyping to ensure that GitHub issues 
> will actually work for us (given our unique requirements), and to help 
> achieve the proper balance.  It does us no good to switch if we just anger 
> all the existing devs.
>
> IMHO, if the switch to GH doesn’t improve our workflow, then it definitely 
> warrants a reevaluation.  I think things will be better, but let’s prove it.

Perhaps we should put an explicit step on the transition plan, after
the prototyping, that's "gather feedback from prototypes, re-evaluate,
make final go/no-go decision"? I assume we'll want to do that anyway,
and having it formally written down might reassure people. It might
also encourage more people to actually try out the prototypes if we
make it very clear that they're going to be asked for feedback.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 595: Improving bugs.python.org

[Barry Warsaw]

On May 31, 2019, at 01:22, Antoine Pitrou  wrote:

> I second this.
> 
> There are currently ~7000 bugs open on bugs.python.org.  The Web UI
> makes a good job of actually being able to navigate through these bugs,
> search through them, etc.
> 
> Did the Steering Council conduct a usability study of Github Issues
> with those ~7000 bugs open?  If not, then I think the acceptance of
> migrating to Github is a rushed job.  Please reconsider.

Thanks for your feedback Antoine.

This is a tricky issue, with many factors and tradeoffs to consider.  I really 
appreciate Ezio and Berker working on PEP 595, so we can put all these issues 
on the table.

I think one of the most important tradeoffs is balancing the needs of existing 
developers (those who actively triage bugs today), and future contributors.  
But this and other UX issues are difficult to compare on our actual data right 
now.  I fully expect that just as with the switch to git, we’ll do lots of 
sample imports and prototyping to ensure that GitHub issues will actually work 
for us (given our unique requirements), and to help achieve the proper balance. 
 It does us no good to switch if we just anger all the existing devs.

IMHO, if the switch to GH doesn’t improve our workflow, then it definitely 
warrants a reevaluation.  I think things will be better, but let’s prove it.

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 595: Improving bugs.python.org

[Brett Cannon]

On Fri, May 31, 2019 at 1:23 AM Antoine Pitrou  wrote:

>
> I second this.
>
> There are currently ~7000 bugs open on bugs.python.org.  The Web UI
> makes a good job of actually being able to navigate through these bugs,
> search through them, etc.
>
> Did the Steering Council conduct a usability study of Github Issues
> with those ~7000 bugs open?


What I can personally speak to is I work on
https://github.com/microsoft/vscode-python for a living and we're at 928
issues. I also interact with and know the team behind
https://github.com/microsoft/vscode at 5,439 issues.
https://github.com/rust-lang/rust/ is as 4,806. https://github.com/golang/go
is at 4,467.

Whether it's 928 or 5,439, to me both numbers are well beyond being small
enough to remember everything in the repo and having to rely on search,
labels, etc. to find anything and it's worked out for me and my team at
work. The VS Code team has also never told me that they were unhappy with
GitHub for their issue tracker at their scale either (I've chatted to them
about their automatic classification bot and they just gave an internal
talk on how they manage the project).

-Brett


>   If not, then I think the acceptance of
> migrating to Github is a rushed job.  Please reconsider.
>
> Regards
>
> Antoine.
>
>
>
> On Thu, 23 May 2019 22:17:24 +0200
> Ezio Melotti  wrote:
> > Hello,
> > Berker and I have been working on a PEP that suggests we keep using
> > and improving bugs.python.org and Roundup instead of switching to
> > GitHub Issues as proposed by PEP 581.
> >
> > The PEP covers:
> > * What are the advantages of Roundup over GitHub issues;
> > * What features are missing in Roundup and how can we add them;
> > * Issues with PEP 581;
> > * Issues with the migration plan proposed by PEP 588;
> >
> > The rendered version of PEP 595 is available at
> > https://www.python.org/dev/peps/pep-0595/
> >
> > For reference, you can consult PEP 581 and 588 at
> > https://www.python.org/dev/peps/pep-0581/ and
> > https://www.python.org/dev/peps/pep-0588/
> >
> > The full text of the PEP is include below.  We are planning to update
> > the PEP to include the feedback we receive and to update the status of
> > features as we implement them (we also have a Google Summer of Code
> > students working on it).
> >
> > Best Regards,
> > Ezio Melotti
> >
> >
> > 
> > PEP: 595
> > Title: Improving bugs.python.org
> > Author: Ezio Melotti , Berker Peksag
> > 
> > Status: Draft
> > Type: Process
> > Content-Type: text/x-rst
> > Created: 12-May-2019
> >
> >
> > Abstract
> > 
> >
> > This PEP proposes a list of improvements to make bugs.python.org
> > more usable for contributors and core developers.  This PEP also
> > discusses why remaining on Roundup should be preferred over
> > switching to GitHub Issues, as proposed by :pep:`581`.
> >
> >
> > Motivation
> > ==
> >
> > On May 14th, 2019 :pep:`581` has been accepted [#]_ without much
> > public discussion and without a clear consensus [#]_.  The PEP
> > contains factual errors and doesn't address some of the
> > issues that the migration to GitHub Issues might present.
> >
> > Given the scope of the migration, the amount of work required,
> > and how it will negatively affect the workflow during the
> > transition phase, this decision should be re-evaluated.
> >
> >
> > Roundup advantages over GitHub Issues
> > =
> >
> > This section discusses reasons why Roundup should be preferred
> > over GitHub Issues and Roundup features that are not available
> > on GitHub Issues.
> >
> > * **Roundup is the status quo.**  Roundup has been an integral
> >   part of the CPython workflow for years.  It is a stable product
> >   that has been tested and customized to adapt to our needs as the
> >   workflow evolved.
> >
> >   It is possible to gradually improve it and avoid the disruption
> >   that a switch to a different system would inevitabily bring to
> >   the workflow.
> >
> > * **Open-source and Python powered.**  Roundup is an open-source
> >   project and is written in Python.  By using it and supporting
> >   it, we also support the Python ecosystem.  Several features
> >   developed for bpo have also been ported to upstream Roundup
> >   over the years.
> >
> > * **Fully customizable.**  Roundup can be (and has been) fully
> >   customized to fit our needs.
> >
> > * **Finer-grained access control.**  Roundup allows the creation
> >   of different roles with different permissions (e.g. create,
> >   view, edit, etc.) for each individual property, and users can
> >   have multiple roles.
> >
> > * **Flexible UI.**  While Roundup UI might look dated, it is
> >   convenient and flexible.
> >
> >   For example, on the issue page, each field (e.g. title, type,
> >   versions, status, linked files and PRs, etc.) have appropriate
> >   UI elements (input boxes, dropdowns, tables, etc.) that are
> >   easy to set and also provide a convenient way to get info about

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

[Tim Peters]

[Tim]
>> I'm keen to get feedback on this before merging the PR, because this
>> case is so very much larger than anything I've ever tried that I'm
>> wary that there may be more than one "surprise" lurking here. ...

[Inada Naoki ]
> I started r5a.4xlarge EC2 instance and started arena.py.
> I will post the result in next 12 hours.

Thank you!  Wrapping this up, Inada attached the results to the bug
report.  For the OP's original big case, the time to reclaim the tree
storage dropped from nearly 5 hours to about 70 seconds.  The time to
build the tree didn't materially change (it was a bit faster after the
patch, but offhand didn't look significantly faster to me).

Since I called this a "design flaw" rather than "a bug", I'm not
inclined to backport it.  If someone else thinks it should be, that's
up to them.  I'm _assuming_ Github won't decide to backport it on its
own - but maybe I'm wrong (Github workflow is still pretty magical to
me).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2019-05-24 - 2019-05-31)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7028 (-39)
  closed 41839 (+118)
  total  48867 (+79)

Open issues with patches: 2820 


Issues opened (50)
==

#26423: Integer overflow in wrap_lenfunc() on 64-bit build of Windows 
https://bugs.python.org/issue26423  reopened by vstinner

#34725: Py_GetProgramFullPath() odd behaviour in Windows
https://bugs.python.org/issue34725  reopened by vstinner

#35753: Importing call from unittest.mock directly causes ValueError
https://bugs.python.org/issue35753  reopened by pablogsal

#36889: Merge StreamWriter and StreamReader into just asyncio.Stream
https://bugs.python.org/issue36889  reopened by koobs

#36991: zipfile: AttributeError on extract
https://bugs.python.org/issue36991  reopened by berker.peksag

#37038: Make idlelib/run.py runnable.
https://bugs.python.org/issue37038  opened by terry.reedy

#37039: IDLE: Improve zoomheight doc and behavior.
https://bugs.python.org/issue37039  opened by terry.reedy

#37040: checking for membership in itertools.count enters infinite loo
https://bugs.python.org/issue37040  opened by Dan Rose

#37041: IDLE: path browser unusable on some displays
https://bugs.python.org/issue37041  opened by aroberge

#37042: wait_for(coro, timeout=0) memleak
https://bugs.python.org/issue37042  opened by ixje

#37043: Buildbots fail when new files are added
https://bugs.python.org/issue37043  opened by jaraco

#37044: Build/test artifacts not ignored for framework build
https://bugs.python.org/issue37044  opened by jaraco

#37048: ssl module: QUIC support for HTTP/3
https://bugs.python.org/issue37048  opened by christian.heimes

#37052: Add examples for mocking async for and async context manager i
https://bugs.python.org/issue37052  opened by xtreak

#37055: Numerous warnings with blake2 module
https://bugs.python.org/issue37055  opened by remi.lapeyre

#37060: import ctypes fails with a statically linked interpreter due t
https://bugs.python.org/issue37060  opened by indygreg

#37062: `AutoNumber` class in enum documentation: support *args in con
https://bugs.python.org/issue37062  opened by rrt

#37063: Incorrect application of func.__defaults__ by inspect's signat
https://bugs.python.org/issue37063  opened by bup

#37064: Feature request: option to keep/add flags to pathfix.
https://bugs.python.org/issue37064  opened by pkopkan

#37068: Emit SyntaxWarning for f-strings without expressions ?
https://bugs.python.org/issue37068  opened by mbussonn

#37069: regrtest: log unraisable exceptions and uncaught thread except
https://bugs.python.org/issue37069  opened by vstinner

#37071: HTMLParser mistakenly inventing new tags while parsing
https://bugs.python.org/issue37071  opened by htran

#37073: clarify functions docs in IO modules and Bytes Objects
https://bugs.python.org/issue37073  opened by Windson Yang

#37074: os.stat() does not work for NUL and CON
https://bugs.python.org/issue37074  opened by serhiy.storchaka

#37075: Error message improvement for AsyncMock
https://bugs.python.org/issue37075  opened by xtreak

#37077: Threading: add builtin TID attribute to Thread objects for AIX
https://bugs.python.org/issue37077  opened by Michael.Felt

#37080: Cannot compile Python3.7.3 on Alt-F (ARM)
https://bugs.python.org/issue37080  opened by Jarl Gullberg

#37081: Test with OpenSSL 1.1.1c
https://bugs.python.org/issue37081  opened by christian.heimes

#37082: Assignment expression operator (walrus) not in built-in help()
https://bugs.python.org/issue37082  opened by mbussonn

#37083: Document TYPE_COMMENT in documentation reference for compound 
https://bugs.python.org/issue37083  opened by pablogsal

#37084: _ctypes not failing, can't find reason
https://bugs.python.org/issue37084  opened by Brian Spratke

#37085: Expose additional socket constants for CAN_BCM flags
https://bugs.python.org/issue37085  opened by karlding

#37086: time.sleep error message misleading
https://bugs.python.org/issue37086  opened by Justin Fay

#37087: Adding native id support for openbsd
https://bugs.python.org/issue37087  opened by David Carlier

#37088: Add a way to schedule a function to be called from the main th
https://bugs.python.org/issue37088  opened by yselivanov

#37089: `import Lib.os` works on windows (but shouldn't)
https://bugs.python.org/issue37089  opened by Anthony Sottile

#37090: test_gdb's test_pycfunction should test all calling convention
https://bugs.python.org/issue37090  opened by petr.viktorin

#37091: subprocess - uncaught PermissionError in send_signal can cause
https://bugs.python.org/issue37091  opened by wesinator

#37093: http.client aborts header parsing upon encountering non-ASCII 
https://bugs.python.org/issue37093  opened by tburke

#37095: [Feature Request]: Add zstd support in tarfile
https://bugs.python.org/issue37095  opened 

Re: [Python-Dev] Should I postpone PEP 558 (locals() semantics) to Python 3.9?

[Terry Reedy]

On 5/31/2019 6:20 AM, Nick Coghlan wrote:
On Fri., 31 May 2019, 6:34 pm Nathaniel Smith, > wrote:


I wouldn't mind having a little more breathing room. It's frustrating
to miss the train, but these bugs are several decades old so I guess
nothing terrible will happen if their fixes get delayed to 3.9.


Agreed.

And I could put that extra time to good use, as starting to flesh out 
the proxy implementation showed that we're missing a lot of scaffolding 
to help make it easier to define new low level mapping types without 
duplicating a lot of code.


I'll update the PEP headers accordingly.


I believe some of your suggested doc change is true now, will remain 
true, and improves on the current locals entry.  No PEP approval is 
needed for this much.



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Expected stability of PyCode_New() and types.CodeType() signatures

[Terry Reedy]

On 5/31/2019 4:46 AM, Petr Viktorin wrote:

Hello,
PEP 570 (Positional-Only Parameters) changed the signatures of 
PyCode_New() and types.CodeType(), adding a new argument for "posargcount".
Our policy for such changes seems to be fragmented tribal knowledge. I'm 
writing to check if my understanding is reasonable, so I can apply it 
and document it explicitly.


There is a surprisingly large ecosystem of tools that create code objects.
The expectation seems to be that these tools will need to be adapted for 
each minor version of Python.


But that's not the same as saying that if you use types.CodeType(), 
you're on your own. To me [PR 13271], seems to go too far when it adds:

 > These types are not supposed to be instantiated outside of
 > CPython internals and constructor signatures will vary
 > between Python versions.

This kind of "washing our hands" doesn't do justice to tools like 
Cython, whose use of PyCode_New is (IMO) perfectly justified. As is 
adapting to Python minor versions.


I summarized the above on the issue
https://bugs.python.org/issue36896
and suggested specific revisions on the PR.
https://github.com/python/cpython/pull/13271


So, we should document the changes as any backwards-incompatible change.
Specifically, changes should be (or should have been, in hindsight) 
mentioned in:

* The "API changes" and/or "Porting" sections of the What's New document


The second PR for Document changes ... added something to What's New.
https://bugs.python.org/issue36886
https://github.com/python/cpython/commit/5d23e282af69d404a3430bb95aefe371112817b3

If you think something more is needed, say so on the issue.


* Version history of the documentation (e.g. versionchanged blocks)


We put version history of the code (but not of the docs) in the docs.
https://docs.python.org/3/reference/compound_stmts.html#function-definitions
should have a version-changed note for the added '/' syntax.


* Any relevant PEPs


If you mean PEPs other than 570, we don't do that.


Also, the expected level of API stability should be documented in the docs.


Yes.  I think that part of PR 13271, rewritten, is good.


PEP 570: https://www.python.org/dev/peps/pep-0570/
PyCode_New: https://docs.python.org/3.8/c-api/code.html#c.PyCode_New
[PR 13271]: https://github.com/python/cpython/pull/13271/files



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should I postpone PEP 558 (locals() semantics) to Python 3.9?

[Nick Coghlan]

On Fri., 31 May 2019, 6:34 pm Nathaniel Smith,  wrote:

> I wouldn't mind having a little more breathing room. It's frustrating
> to miss the train, but these bugs are several decades old so I guess
> nothing terrible will happen if their fixes get delayed to 3.9.
>

And I could put that extra time to good use, as starting to flesh out the
proxy implementation showed that we're missing a lot of scaffolding to help
make it easier to define new low level mapping types without duplicating a
lot of code.

I'll update the PEP headers accordingly.

Cheers,
Nick.


>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

[Inada Naoki]

> I'm keen to get feedback on this before merging the PR, because this
> case is so very much larger than anything I've ever tried that I'm
> wary that there may be more than one "surprise" lurking here.  The PR
> certainly addresses "an obvious" (with hindsight) problem - but is
> that the _only_ gross design flaw here?

I started r5a.4xlarge EC2 instance and started arena.py.
I will post the result in next 12 hours.

Regards,

-- 
Inada Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Expected stability of PyCode_New() and types.CodeType() signatures

[Petr Viktorin]

Hello,
PEP 570 (Positional-Only Parameters) changed the signatures of 
PyCode_New() and types.CodeType(), adding a new argument for "posargcount".
Our policy for such changes seems to be fragmented tribal knowledge. I'm 
writing to check if my understanding is reasonable, so I can apply it 
and document it explicitly.


There is a surprisingly large ecosystem of tools that create code objects.
The expectation seems to be that these tools will need to be adapted for 
each minor version of Python.


But that's not the same as saying that if you use types.CodeType(), 
you're on your own. To me [PR 13271], seems to go too far when it adds:

> These types are not supposed to be instantiated outside of
> CPython internals and constructor signatures will vary
> between Python versions.

This kind of "washing our hands" doesn't do justice to tools like 
Cython, whose use of PyCode_New is (IMO) perfectly justified. As is 
adapting to Python minor versions.


So, we should document the changes as any backwards-incompatible change.
Specifically, changes should be (or should have been, in hindsight) 
mentioned in:

* The "API changes" and/or "Porting" sections of the What's New document
* Version history of the documentation (e.g. versionchanged blocks)
* Any relevant PEPs

Also, the expected level of API stability should be documented in the docs.

Does that sound right?


PEP 570: https://www.python.org/dev/peps/pep-0570/
PyCode_New: https://docs.python.org/3.8/c-api/code.html#c.PyCode_New
[PR 13271]: https://github.com/python/cpython/pull/13271/files
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should I postpone PEP 558 (locals() semantics) to Python 3.9?

[Nathaniel Smith]

I wouldn't mind having a little more breathing room. It's frustrating
to miss the train, but these bugs are several decades old so I guess
nothing terrible will happen if their fixes get delayed to 3.9.

On Thu, May 30, 2019 at 4:23 PM Nick Coghlan  wrote:
>
> Hi folks,
>
> The reference implementation for PEP 558 (my attempt to fix the interaction 
> between tracing functions and closure variables) is currently segfaulting 
> somewhere deep in the garbage collector, and I've found that there's an issue 
> with the PyEval_GetLocals() API returning a borrowed reference that means I 
> need to tweak the proposed C API a bit such that PyEval_GetLocals() returns 
> the proxy at function scope, and we add a new PyEval_GetPyLocals() that 
> matches the locals() builtin.
>
> I don't *want* to postpone this to Python 3.9, but there turned out to be 
> more remaining work than I thought there was to get this ready for inclusion 
> in beta 1.
>
> I'll try to get the C API design details sorted today, but the segfault is 
> mystifying me, and prevents the option of putting the core implementation in 
> place for b1, and tidying up the documentation and comments for b2.
>
> Cheers,
> Nick.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/njs%40pobox.com



-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 595: Improving bugs.python.org

[Antoine Pitrou]

I second this.

There are currently ~7000 bugs open on bugs.python.org.  The Web UI
makes a good job of actually being able to navigate through these bugs,
search through them, etc.

Did the Steering Council conduct a usability study of Github Issues
with those ~7000 bugs open?  If not, then I think the acceptance of
migrating to Github is a rushed job.  Please reconsider.

Regards

Antoine.



On Thu, 23 May 2019 22:17:24 +0200
Ezio Melotti  wrote:
> Hello,
> Berker and I have been working on a PEP that suggests we keep using
> and improving bugs.python.org and Roundup instead of switching to
> GitHub Issues as proposed by PEP 581.
> 
> The PEP covers:
> * What are the advantages of Roundup over GitHub issues;
> * What features are missing in Roundup and how can we add them;
> * Issues with PEP 581;
> * Issues with the migration plan proposed by PEP 588;
> 
> The rendered version of PEP 595 is available at
> https://www.python.org/dev/peps/pep-0595/
> 
> For reference, you can consult PEP 581 and 588 at
> https://www.python.org/dev/peps/pep-0581/ and
> https://www.python.org/dev/peps/pep-0588/
> 
> The full text of the PEP is include below.  We are planning to update
> the PEP to include the feedback we receive and to update the status of
> features as we implement them (we also have a Google Summer of Code
> students working on it).
> 
> Best Regards,
> Ezio Melotti
> 
> 
> 
> PEP: 595
> Title: Improving bugs.python.org
> Author: Ezio Melotti , Berker Peksag
> 
> Status: Draft
> Type: Process
> Content-Type: text/x-rst
> Created: 12-May-2019
> 
> 
> Abstract
> 
> 
> This PEP proposes a list of improvements to make bugs.python.org
> more usable for contributors and core developers.  This PEP also
> discusses why remaining on Roundup should be preferred over
> switching to GitHub Issues, as proposed by :pep:`581`.
> 
> 
> Motivation
> ==
> 
> On May 14th, 2019 :pep:`581` has been accepted [#]_ without much
> public discussion and without a clear consensus [#]_.  The PEP
> contains factual errors and doesn't address some of the
> issues that the migration to GitHub Issues might present.
> 
> Given the scope of the migration, the amount of work required,
> and how it will negatively affect the workflow during the
> transition phase, this decision should be re-evaluated.
> 
> 
> Roundup advantages over GitHub Issues
> =
> 
> This section discusses reasons why Roundup should be preferred
> over GitHub Issues and Roundup features that are not available
> on GitHub Issues.
> 
> * **Roundup is the status quo.**  Roundup has been an integral
>   part of the CPython workflow for years.  It is a stable product
>   that has been tested and customized to adapt to our needs as the
>   workflow evolved.
> 
>   It is possible to gradually improve it and avoid the disruption
>   that a switch to a different system would inevitabily bring to
>   the workflow.
> 
> * **Open-source and Python powered.**  Roundup is an open-source
>   project and is written in Python.  By using it and supporting
>   it, we also support the Python ecosystem.  Several features
>   developed for bpo have also been ported to upstream Roundup
>   over the years.
> 
> * **Fully customizable.**  Roundup can be (and has been) fully
>   customized to fit our needs.
> 
> * **Finer-grained access control.**  Roundup allows the creation
>   of different roles with different permissions (e.g. create,
>   view, edit, etc.) for each individual property, and users can
>   have multiple roles.
> 
> * **Flexible UI.**  While Roundup UI might look dated, it is
>   convenient and flexible.
> 
>   For example, on the issue page, each field (e.g. title, type,
>   versions, status, linked files and PRs, etc.) have appropriate
>   UI elements (input boxes, dropdowns, tables, etc.) that are
>   easy to set and also provide a convenient way to get info about
>   the issue at a glance.  The number of fields, their values, and
>   the UI element they use is also fully customizable.
>   GitHub only provides labels.
> 
>   The issue list page presents the issues in a compact and easy
>   to read table with separate columns for different fields.  For
>   comparison, Roundup lists 50 issues in a screen, whereas GitHub
>   takes two screens to shows 25 issues.
> 
> * **Advanced search.**  Roundup provides an accurate way to search
>   and filter by using any combination of issue fields.
>   It is also possible to customize the number of results and the
>   fields displayed in the table, and the sorting and grouping
>   (up to two levels).
> 
>   bpo also provides predefined summaries (e.g. "Created by you",
>   "Assigned to you", etc.) and allows the creation of custom
>   search queries that can be conveniently accessed from the sidebar.
> 
> * **Nosy list autocomplete.**  The nosy list has an autocomplete
>   feature that suggests maintainers and experts.  The suggestions
>   are automatically updated wh