[Python-Dev] Re: GDB not breaking at the right place

[Guido van Rossum]

On Tue, May 25, 2021 at 7:42 PM Inada Naoki  wrote:

> On Tue, May 25, 2021 at 5:38 AM Guido van Rossum  wrote:
> >
> > To the contrary, I think if you want the CI jobs to be faster you should
> add the CFLAGS to the configure call used to run the CI jobs.
> >
>
> -Og makes it faster not only CI jobs, but also everyday "edit code and
> run `make test` with all assertions" cycles.
>

I usually edit and build many times before I'm happy enough with my work to
even try "make test" (or the Windows equivalent :-). Which of these options
*compiles* the fastest? Maybe that one should be the default. But perhaps
other peoples' workflow is different (in fact it would be surprising if we
all had the same workflow for writing C code :-).

In fact I'm guilty of not even building using --with-pydebug until I run
into a problem I can't debug with fprintf().  :-) But once I do, it's
likely that I want to aim a debugger at the code, *and* I'd be recompiling
repeatedly. The time to complete a full "make test" run on my local machine
doesn't bother me, it's too slow to have in my development loop either way.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SLEXGPM5U2F4S2QRNUEZ3HEPB27ZMJ4B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

[Guido van Rossum]

On Tue, May 25, 2021 at 7:56 PM Brett Cannon  wrote:

>
> On Tue., May 25, 2021, 12:58 Guido van Rossum,  wrote:
>
>> [...]
>> Or do you think the "Standards Track" PEP should just codify general
>> agreement that we're going to implement a specializing adaptive
>> interpreter, with the level of detail that's currently in the PEP?
>>
>
> This. Having this as an informational PEP that's already marked as Active
> seems off somehow to me. I guess it feels more "we're doing this" (which I
> know isn't intended) rather than "this is our plan, what do you all think?
> All good?"
>

Right. I have no power to unilaterally decide that "we're doing this", in
the sense of "we're definitely merging this", and neither do Eric and Mark.
But given the reactions during and since the Language Summit I had assumed
that there isn't much dissent, so that "it's okay to try this" seems a
reasonable conclusion. And given *that*, I'm not sure that there's much of
a difference between the two positions.

But I'm not trying to stifle discussion, and there is plenty of work that
we (the three authors) can do before we're at the point of no return. (In
fact, even if we were to get called back, all we'd need to do would be to
revert some commits -- this has happened before.)


> I don't recall other standards track PEPs that don't also spell out the
>> specification of the proposal in detail.
>>
>
> I also am not aware of a PEP that's proposed restructuring the eval loop
> like this either. 
>

Right. Usually we just discuss ideas to improve the eval loop on bpo or in
PRs, occasionally on python-dev, but not in PEPs. Over the years the eval
loop has become quite a bit more efficient than the loop I wrote three
decades ago, but also much more complex, and there's less and less
low-hanging fruit left.

In the end the proposal here could make it easier to reason about the
performance of the eval loop, because there will be fewer one-off hacks,
and instead a more systematic approach.


> I'm personally fine with the detail and saying details may shift as things
> move forward and lessons are learned based on the scope and updating the
> PEP accordingly. But that's just me and I don't know if others agree (hence
> the reason I'm suggesting this be Standards Track).
>

Sure. Do you have a specific text change in mind (or even just a suggestion
about where we should insert some language about details shifting and
learning lessons? Are there other things you'd like to see changed in the
PEP?

Also, can you outline a specific process that you would be comfortable with
here, given that we're breaking a certain amount of new ground here
process-wise? Or should we just change the PEP type to Standards Track and
submit it to the Steering Council for review?

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XJ562DTRNBCMAUUERVWQPAJDUS3JWLFK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: GDB not breaking at the right place

[Gregory P. Smith]

On Tue, May 25, 2021 at 7:49 PM Inada Naoki  wrote:

> On Tue, May 25, 2021 at 5:38 AM Guido van Rossum  wrote:
> >
> > To the contrary, I think if you want the CI jobs to be faster you should
> add the CFLAGS to the configure call used to run the CI jobs.
> >
>
> -Og makes it faster not only CI jobs, but also everyday "edit code and
> run `make test` with all assertions" cycles.
>
> I don't have opinion which should be default. (+0 for -O0).
> I use -Og by default and use -O0 only when I need anyway.
>

Agreed, what we do today is already fine.  -Og or -O1 are decent options
for fast unoptimized builds that lead to increased productivity in the
common case.
Actually firing up a debugger on CPython's C code is not the common thing
for a developer to do.
When someone wants to do that, they should build with the relevant compiler
for that purpose.  ie: Skip should do this.  If there is confusion about
the meaning of --with-pydebug, that's just a documentation/help-text update
to be made.

-gps

FWIW, we can disable optimization per-file basis during debugging.
>
>   // Put this line on files you want to debug.
>   #pragma GCC optimize ("O0")
>
> Regards,
>
> --
> Inada Naoki  
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OJJZKWS446PJPXHUBNUVIYE756D5HHP4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PMOP7CQHRHB7C5737E5BKRIBXHX5PBHX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

[Brett Cannon]

On Tue., May 25, 2021, 12:58 Guido van Rossum,  wrote:

> On Tue, May 25, 2021 at 12:34 PM Brett Cannon  wrote:
>
>>
>> I personally think it should be a Standards Track PEP. This PEP isn't
>> documenting some detail like PEP 13 or some release schedule, but is
>> instead proposing a rather major change to the interpreter which a lot of
>> us will need to understand in order to support the code (and I do realize
>> the entire area of "what requires a PEP and what doesn't" is very hazy).
>>
>
> Does that also mean you think the design should be completely hashed out
> and approved by the SC ahead of merging the implementation? Given the
> amount of work, that would run into another issue -- many of the details of
> the design can't be fixed until the implementation has proceeded, and we'd
> end up with a long-living fork of the implementation followed by a giant
> merge. My preference (and my promise at the Language Summit) is to avoid
> mega-PRs and instead work on this incrementally.
>
> Now, we've done similar things before (for example, the pattern matching
> implementation was a long-living branch), but the difference is that for
> pattern matching, the implementation followed the design, whereas for the
> changes to the bytecode interpreter that we're undertaking here, much of
> the architecture will be designed as the implementation proceeds, based on
> what we learn during the implementation.
>
> Or do you think the "Standards Track" PEP should just codify general
> agreement that we're going to implement a specializing adaptive
> interpreter, with the level of detail that's currently in the PEP?
>

This. Having this as an informational PEP that's already marked as Active
seems off somehow to me. I guess it feels more "we're doing this" (which I
know isn't intended) rather than "this is our plan, what do you all think?
All good?"


I don't recall other standards track PEPs that don't also spell out the
> specification of the proposal in detail.
>

I also am not aware of a PEP that's proposed restructuring the eval loop
like this either.  I'm personally fine with the detail and saying details
may shift as things move forward and lessons are learned based on the scope
and updating the PEP accordingly. But that's just me and I don't know if
others agree (hence the reason I'm suggesting this be Standards Track).

-Brett


> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JSKR5GS7OSXYKZFQAYZ4IVXZCC5NXIEX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: GDB not breaking at the right place

[Inada Naoki]

On Tue, May 25, 2021 at 5:38 AM Guido van Rossum  wrote:
>
> To the contrary, I think if you want the CI jobs to be faster you should add 
> the CFLAGS to the configure call used to run the CI jobs.
>

-Og makes it faster not only CI jobs, but also everyday "edit code and
run `make test` with all assertions" cycles.

I don't have opinion which should be default. (+0 for -O0).
I use -Og by default and use -O0 only when I need anyway.

FWIW, we can disable optimization per-file basis during debugging.

  // Put this line on files you want to debug.
  #pragma GCC optimize ("O0")

Regards,

-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OJJZKWS446PJPXHUBNUVIYE756D5HHP4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Difficulty of testing beta releases now available

[Pablo Galindo Salgado]

> - Cython doesn't work because of _PyGen_Send change [1]

I think this is fixed on the latest cython (0.29.23)


On Wed, 26 May 2021 at 01:05, Neil Schemenauer 
wrote:

> On 2021-05-04, Łukasz Langa wrote:
> > We strongly encourage maintainers of third-party Python projects
> > to test with 3.10 during the beta phase and report issues found to
> > the Python bug tracker  as soon as
> > possible.
>
> Testing with Python 3.10b1 is not easy, at least for me.  Here is a
> list of initial problems I ran into, from memory:
>
> - Cython doesn't work because of _PyGen_Send change [1]
>
> - scipy cannot be installed because it has requires_python =
>   ">=3.7,<3.10".  If you manually install from source, it seems to
>   work.
>
> - numpy cannot be installed because of _Py_HashDouble() change [2]
>
> - trio cannot be used because of TracebackException.__init__ changes [3]
>
> For the above problems, I would suggest the 3rd party package has
> the issue and it's not a problem with the Python release.  However,
> I guess that few people are using Python without 3rd party packages.
> So, it seems unsurprising that beta and RC releases are not well
> tested.  It has taken me quite a few hours to get a working version
> of Python 3.10 with all required dependancies such that I can run
> unit tests for some application code.
>
> Can we do any things to improve the situation?  Perhaps using the
> pre-release functionality of PyPI would help.  We would have to
> somehow encourage 3rd party packages to upload pre-releases that are
> compatible with our beta/RC releases.
>
>
> 1. https://github.com/cython/cython/issues/3876
> 2. https://github.com/numpy/numpy/issues/19033
> 3. https://github.com/python-trio/trio/issues/1899
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2Z23GRYT5VDHR4YJKS6YIQG7G46SC27T/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZJPJRK4OYCA7EOIMYRW4U5N6YPJVKVXF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Difficulty of testing beta releases now available

[Neil Schemenauer]

On 2021-05-04, Łukasz Langa wrote:
> We strongly encourage maintainers of third-party Python projects
> to test with 3.10 during the beta phase and report issues found to
> the Python bug tracker  as soon as
> possible.

Testing with Python 3.10b1 is not easy, at least for me.  Here is a
list of initial problems I ran into, from memory:

- Cython doesn't work because of _PyGen_Send change [1]

- scipy cannot be installed because it has requires_python =
  ">=3.7,<3.10".  If you manually install from source, it seems to
  work.

- numpy cannot be installed because of _Py_HashDouble() change [2]

- trio cannot be used because of TracebackException.__init__ changes [3]

For the above problems, I would suggest the 3rd party package has
the issue and it's not a problem with the Python release.  However,
I guess that few people are using Python without 3rd party packages.
So, it seems unsurprising that beta and RC releases are not well
tested.  It has taken me quite a few hours to get a working version
of Python 3.10 with all required dependancies such that I can run
unit tests for some application code.

Can we do any things to improve the situation?  Perhaps using the
pre-release functionality of PyPI would help.  We would have to
somehow encourage 3rd party packages to upload pre-releases that are
compatible with our beta/RC releases.


1. https://github.com/cython/cython/issues/3876
2. https://github.com/numpy/numpy/issues/19033
3. https://github.com/python-trio/trio/issues/1899
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2Z23GRYT5VDHR4YJKS6YIQG7G46SC27T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

[Dino]

To potentially help provide a little bit of additional detail around our
approach I've spent some time writing up our internal details of the shadow
byte code implementation, and landed that in our Cinder repo here:
https://github.com/facebookincubator/cinder/blob/cinder/3.8/CinderDoc/shadowcode.rst.
That might at least spark discussion or ideas about possible internal
implementation details or things which could be different/more efficient in
our implementation.

I've also had a version of it against 3.10 going for a while (as internally
we're still at 3.8) and I've updated it to a relatively recent merge of
3.11 main.  I've pushed the latest version of that here here:
https://github.com/DinoV/cpython/tree/shadowcode_rebase_2021_05_12.  The
3.11 version obviously isn't as battle tested as what we've been running in
production for some time now but it pretty much the same.  It is missing
our improved global caching which uses dictionary watches though.  And it
is a rather large PR (almost 7k lines) but over 1/3rd of that is the test
cases.

Also just to inform the discussion around potential performance benefits,
here's how that alone is currently benchmarking versus the base commit:

cpython_310_opt_rig.json


Performance version: 1.0.1
Report on
Linux-5.2.9-229_fbk15_hardened_4185_g357f49b36602-x86_64-with-glibc2.28
Number of logical CPUs: 48
Start date: 2021-05-17 21:57:08.095822
End date: 2021-05-17 22:40:33.374232

cpython_ghdino_opt_rig.json
===

Performance version: 1.0.1
Report on
Linux-5.2.9-229_fbk15_hardened_4185_g357f49b36602-x86_64-with-glibc2.28
Number of logical CPUs: 48
Start date: 2021-05-21 17:25:24.410644
End date: 2021-05-21 18:02:53.524314

+-+--+-+--+---+
| Benchmark   | cpython_310_opt_rig.json |
cpython_ghdino_opt_rig.json | Change   | Significance  |
+=+==+=+==+===+
| 2to3| 498 ms   | 459 ms
   | 1.09x faster | Significant (t=15.60) |
+-+--+-+--+---+
| chameleon   | 13.4 ms  | 12.6 ms
| 1.07x faster | Significant (t=11.10) |
+-+--+-+--+---+
| chaos   | 163 ms   | 135 ms
   | 1.21x faster | Significant (t=33.07) |
+-+--+-+--+---+
| crypto_pyaes| 171 ms   | 147 ms
   | 1.16x faster | Significant (t=24.93) |
+-+--+-+--+---+
| deltablue   | 11.7 ms  | 8.38 ms
| 1.40x faster | Significant (t=70.51) |
+-+--+-+--+---+
| django_template | 73.7 ms  | 68.1 ms
| 1.08x faster | Significant (t=13.12) |
+-+--+-+--+---+
| dulwich_log | 108 ms   | 98.6 ms
| 1.10x faster | Significant (t=18.11) |
+-+--+-+--+---+
| fannkuch| 734 ms   | 731 ms
   | 1.00x faster | Not significant   |
+-+--+-+--+---+
| float   | 166 ms   | 140 ms
   | 1.18x faster | Significant (t=29.38) |
+-+--+-+--+---+
| go  | 345 ms   | 305 ms
   | 1.13x faster | Significant (t=31.29) |
+-+--+-+--+---+
| hexiom  | 14.4 ms  | 13.1 ms
| 1.10x faster | Significant (t=15.95) |
+-+--+-+--+---+
| json_dumps  | 19.6 ms  | 18.1 ms
| 1.09x faster | Significant (t=13.85) |
+-+--+-+--+---+
| json_loads  | 37.5 us  | 34.8 us
  

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

[Guido van Rossum]

On Tue, May 25, 2021 at 1:50 PM Łukasz Langa  wrote:

>
> On 25 May 2021, at 21:57, Guido van Rossum  wrote:
>
> On Tue, May 25, 2021 at 12:34 PM Brett Cannon  wrote:
>
>>
>> I personally think it should be a Standards Track PEP. This PEP isn't
>> documenting some detail like PEP 13 or some release schedule, but is
>> instead proposing a rather major change to the interpreter which a lot of
>> us will need to understand in order to support the code (and I do realize
>> the entire area of "what requires a PEP and what doesn't" is very hazy).
>>
>
> Now, we've done similar things before (for example, the pattern matching
> implementation was a long-living branch), but the difference is that for
> pattern matching, the implementation followed the design, whereas for the
> changes to the bytecode interpreter that we're undertaking here, much of
> the architecture will be designed as the implementation proceeds, based on
> what we learn during the implementation.
>
>
> Good point. We've also done long-living branching during Gilectomy which
> saved a lot of pain when it turned out not to be worth pursuing after all.
> Do you think this case is qualitatively different?
>

I think it's different -- the problems with the Gilectomy were pretty
predictable (slower single-core perf due to way more locking calls), but it
was not predictable whether Larry would be able to overcome them (I was
rooting for him the whole time).

Here, we're looking at something where Mark has prototyped the proposed
approach extensively (HoyPy, HotPy2), and the question is more whether
Python 3.11 is going to be 15% faster or 50%. And some of the ideas have
also been prototyped by the existing inline caches (some of the proposal is
just to do more of those, and reducing the overhead by specializing
opcodes), and further validated by Dino's work at Facebook/Instagram on
Shadowcode (part of Cinder), which also specializes opcodes.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WOODDS3VR5AWKWXRZC4XU26F44H2CC4W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

[Łukasz Langa]

> On 25 May 2021, at 21:57, Guido van Rossum  wrote:
> 
> On Tue, May 25, 2021 at 12:34 PM Brett Cannon  > wrote:
> 
> I personally think it should be a Standards Track PEP. This PEP isn't 
> documenting some detail like PEP 13 or some release schedule, but is instead 
> proposing a rather major change to the interpreter which a lot of us will 
> need to understand in order to support the code (and I do realize the entire 
> area of "what requires a PEP and what doesn't" is very hazy).
> 
> Now, we've done similar things before (for example, the pattern matching 
> implementation was a long-living branch), but the difference is that for 
> pattern matching, the implementation followed the design, whereas for the 
> changes to the bytecode interpreter that we're undertaking here, much of the 
> architecture will be designed as the implementation proceeds, based on what 
> we learn during the implementation.

Good point. We've also done long-living branching during Gilectomy which saved 
a lot of pain when it turned out not to be worth pursuing after all. Do you 
think this case is qualitatively different?

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PUMUMMZDBXICCZ6VVKU43VQY32DIYQED/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

[Guido van Rossum]

On Tue, May 25, 2021 at 12:34 PM Brett Cannon  wrote:

>
> I personally think it should be a Standards Track PEP. This PEP isn't
> documenting some detail like PEP 13 or some release schedule, but is
> instead proposing a rather major change to the interpreter which a lot of
> us will need to understand in order to support the code (and I do realize
> the entire area of "what requires a PEP and what doesn't" is very hazy).
>

Does that also mean you think the design should be completely hashed out
and approved by the SC ahead of merging the implementation? Given the
amount of work, that would run into another issue -- many of the details of
the design can't be fixed until the implementation has proceeded, and we'd
end up with a long-living fork of the implementation followed by a giant
merge. My preference (and my promise at the Language Summit) is to avoid
mega-PRs and instead work on this incrementally.

Now, we've done similar things before (for example, the pattern matching
implementation was a long-living branch), but the difference is that for
pattern matching, the implementation followed the design, whereas for the
changes to the bytecode interpreter that we're undertaking here, much of
the architecture will be designed as the implementation proceeds, based on
what we learn during the implementation.

Or do you think the "Standards Track" PEP should just codify general
agreement that we're going to implement a specializing adaptive
interpreter, with the level of detail that's currently in the PEP? I don't
recall other standards track PEPs that don't also spell out the
specification of the proposal in detail.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7BDH52HDWQB4U7NSZ4CNPXIB2MTMD3XE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-iterators mailing list on SourceForge

[Julien Palard via Python-Dev]

Hi,

Le 5/25/21 à 9:18 PM, Brett Cannon a écrit :
> Is there something to do here? The python-iterators mailing list is
> already marked as public.

Looks like Guido is faster than you and set it public already. But looks
like the archives are corrupted or something, it's almost empty.

I sent a mail to SourceForge support but my hope is around 0. If it's
not fixed in a few months I'll just remove the links pointing to this
archive (there's some in the PEPs).

Thanks Brett and Guido,
--
[Julien Palard](https://mdk.fr)

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QD7SVDWBMAAV7DCVK5K7RPEWCZ6YJIKB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-iterators mailing list on SourceForge

[Guido van Rossum]

Sorry, that was me. :-)

On Tue, May 25, 2021 at 12:24 PM Brett Cannon  wrote:

>
>
> On Fri, May 21, 2021 at 9:25 AM Julien Palard via Python-Dev <
> python-dev@python.org> wrote:
>
>> Le 5/11/21 à 8:39 PM, Guido van Rossum a écrit :
>> > I doubt that anyone has the keys to the python project on sourceforge
>> > any more... :-( We've abandoned that platform nearly two decades ago.
>>
>
>> That's right…
>>
>> Sourceforge staff mentionned there's the list of current admins here:
>>
>> => https://sourceforge.net/p/python/_members/
>>
>>
> 1Password to the rescue as I still have my keys! 
>
>
>> so if someone see its login there, you can try logging in and unhide the
>> mailing list, else I'll try to have sourceforge unhide it by hand.
>>
>
> Is there something to do here? The python-iterators mailing list is
> already marked as public.
>
> -Brett
>
>
>>
>> --
>> [Julien Palard](https://mdk.fr)
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/DFE7G73N6LLSVYOZBZ6BHDAOGAGORQ6I/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/C66CRKE5PJJVU2EUB25UY6SBTDMEQE5T/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7KDP4WM2SUWHGYNJSZIOHDCBEMNLBRVT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

[Brett Cannon]

On Thu, May 13, 2021 at 1:38 AM Mark Shannon  wrote:

> Hi Terry,
>
> On 13/05/2021 8:20 am, Terry Reedy wrote:
> > On 5/12/2021 1:40 PM, Mark Shannon wrote:
> >
> >> This is an informational PEP about a key part of our plan to improve
> >> CPython performance for 3.11 and beyond.
> >
> > What is the purpose of this PEP?  It seems in part to be like a
> > Standards Track PEP in that it proposes a new (revised) implementation
> > idea for the CPython bycode interpreter.  Do you not intend this to not
> > constitute approval of even the principle?
>
> I will make it a standards PEP if anyone feels that would be better.
> We can implement PEP 659 incrementally, without any large changes to the
> implementation or any to the language or API/ABI, so a standards PEP
> didn't seem necessary to us.
>
> However, because it is a large change to the implementation, it seemed
> worth documenting and doing so in a clearly public fashion. Hence the
> informational PEP.
>

I personally think it should be a Standards Track PEP. This PEP isn't
documenting some detail like PEP 13 or some release schedule, but is
instead proposing a rather major change to the interpreter which a lot of
us will need to understand in order to support the code (and I do realize
the entire area of "what requires a PEP and what doesn't" is very hazy).

-Brett


>
> >
> > One of the issues in the new project gave formulas for the cost versus
> > benefit calculations underlying specialization.  Depending on your
> > purpose, it might be good to include them.  They certainly gave some
> > clarity to me.
> >
>
> Which ones in particular? I can add something like them to the PEP.
>
> Cheers,
> Mark.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VY3JB3XO4D2E65ZR5IZUDP7MFQJ3JXIF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SHR37RVTQGCTNK23ENVG3K57ZHY2K37X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The repr of a sentinel

[Eric Nieuwland]

To add to the suggestions already given in this thread I dug into code I wrote 
some time ago.
Offered as an inspiration.

=== missing.py ===

from typing import Any

def MISSING(klass: Any) -> Any:
"""
create a sentinel to indicate a missing instance of a class
:param klass: the class of which an instance is missing
:return: missing class object
"""
g = globals()
missing_klass_name = 
f"_MISSING_{klass.__module__}_{klass.__name__}_MISSING_"
if missing_klass_name not in g:
g[missing_klass_name] = type(
missing_klass_name,
(klass,),
{
"__repr__": lambda x: f"MISSING({klass.__name__})",
}
)
return g[missing_klass_name]()

===

and as a demo:

=== demo_missing.py ===

import pickle

from missing import MISSING

x = MISSING(str)
y = "bar"
print(f"{x!r} == {y!r}: {x == y}")
print(f"{x!r} is {y!r}: {x is y}")
# MISSING(str) == 'bar': False
# MISSING(str) is 'bar': False

with open("object.pickled", "wb") as f:
pickle.dump(x, f)
with open("object.pickled", "rb") as f:
y = pickle.load(f)
print(f"{x!r} == {y!r}: {x == y}")
print(f"{x!r} is {y!r}: {x is y}")
# MISSING(str) == MISSING(str): True
# MISSING(str) is MISSING(str): False

def foo(a: int = MISSING(int), b: int = MISSING(int)):
print(f"{a=} {isinstance(a, int)}")
print(f"{b=} {isinstance(b, int)}")
print(f"{a!r} == {b!r}: {a == b}")
print(f"{a!r} is {b!r}: {a is b}")

foo()
# a=MISSING(int) True
# b=MISSING(int) True
# MISSING(int) == MISSING(int): True
# MISSING(int) is MISSING(int): False

foo(1)
# a=1 True
# b=MISSING(int) True
# 1 == MISSING(int): False
# 1 is MISSING(int): False

class Test:
...

t = MISSING(Test)
print(f"{t=}")
# t=MISSING(Test)

===___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NABTH4G5IZZYCFSH2RBQNLQL3BKEUFKQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: GDB not breaking at the right place

[Steve Dower]

On 5/24/2021 9:38 PM, Guido van Rossum wrote:
To the contrary, I think if you want the CI jobs to be faster you should 
add the CFLAGS to the configure call used to run the CI jobs.


Big +1

We should have the most useful interactive development/debugging options 
set by default (or with an obvious option), and use the complex 
overrides in our own automated systems.


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TNHXYXH6A77J5Y43JBLVMXC3KF722H6T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-iterators mailing list on SourceForge

[Brett Cannon]

On Fri, May 21, 2021 at 9:25 AM Julien Palard via Python-Dev <
python-dev@python.org> wrote:

> Le 5/11/21 à 8:39 PM, Guido van Rossum a écrit :
> > I doubt that anyone has the keys to the python project on sourceforge
> > any more... :-( We've abandoned that platform nearly two decades ago.
>

> That's right…
>
> Sourceforge staff mentionned there's the list of current admins here:
>
> => https://sourceforge.net/p/python/_members/
>
>
1Password to the rescue as I still have my keys! 


> so if someone see its login there, you can try logging in and unhide the
> mailing list, else I'll try to have sourceforge unhide it by hand.
>

Is there something to do here? The python-iterators mailing list is already
marked as public.

-Brett


>
> --
> [Julien Palard](https://mdk.fr)
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/DFE7G73N6LLSVYOZBZ6BHDAOGAGORQ6I/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C66CRKE5PJJVU2EUB25UY6SBTDMEQE5T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The repr of a sentinel

[Luciano Ramalho]

On Tue, May 25, 2021 at 1:08 PM Pascal Chambon  wrote:
> I feel like the 3-lines declaration of a new sentinel would discourage a
> bit its adoption compared to just "sentinel = object()"

How about one line?

class Missing(Sentinel): pass

And avoiding a lot of complications?

https://github.com/fluentpython/example-code-2e/blob/master/25-class-metaprog/sentinel/sentinel.py

Cheers,

Luciano



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
|  Technical Principal at ThoughtWorks
|  Twitter: @ramalhoorg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZUKHRZWLQK45TV74NKG2FX4ILWI4YHXK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: GDB not breaking at the right place

[Skip Montanaro]

me> I'm having a hard time debugging some virtual machine code because GDB
won't break where it's supposed to.

Here's a quick follow-up. I tried a number of different values of OPT
during configuration and compilation, but nothing changed the result. I
could never (and still can't) get GDB to break at the line it announces
when a breakpoint is set using the TARGET_* labels generated for computed
gotos. I also backed away from my dev branch and switched to up-to-date
versions of main and 3.10. No difference...

So, I opened a bug against GDB:

https://sourceware.org/bugzilla/show_bug.cgi?id=27907

and ... wait for it ...

The person who responded (Keith Seitz @ RedHat) was unable to reproduce the
problem. He encouraged me to build GDB and try again, and with some effort
I was able to build an executable (wow, the GDB build process makes
building Python look like a piece of cake). Still, the difference between
the announced and actual line numbers of the breakpoint remains.

I disabled Python support in GDB by renaming my ~/.gdbinit file which
declares

add-auto-load-safe-path /home/skip/src/python/rvm

That had no effect either. I don't have any LD_*_PATH environment variables
set. I think I've run out of things to try.

I don't recall anyone here indicating they'd tried to replicate the
problem. Could I bother someone to give it a whirl? It's easy. Just run GDB
referring to a Python executable with computed gotos enabled and debug
symbols included. At the (gdb) prompt, execute:

b ceval.c:_PyEval_EvalFrameDefault:TARGET_LOAD_CONST
run

and compare the line number announced when the breakpoint is set with the
line number announced when execution stops. On my main branch (updated
yesterday), using

OPT='-O3 -g -Wall'

I get an absolutely bonkers break in the action:

% ~/src/binutils-gdb/gdb/gdb ./pythonild.sh
GNU gdb (GDB) 11.0.50.20210524-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./python...
(gdb) b ceval.c:_PyEval_EvalFrameDefault:TARGET_LOAD_CONST
*Breakpoint 1 at 0x5e934: file Python/ceval.c, line 1836.*
(gdb) r
Starting program: /home/skip/src/python/rvm/python
warning: the debug information found in "/lib64/ld-2.31.so" does not match
"/lib64/ld-linux-x86-64.so.2" (CRC mismatch).

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

*Breakpoint 1, 0x555b2934 in _PyEval_EvalFrameDefault
(tstate=, f=, throwflag=) at
Python/ceval.c:2958*
2958 DISPATCH();


LOAD_CONST is, in fact, defined at line ceval.c:1836. Line 2958 is the last
line of the implementation of LOAD_NAME, just a few lines away :-/.

If I get more detailed with the configure/compile options I can get the
difference down to a few lines, but I've yet to see it work correctly.

I'm currently offering

OPT='-g -O0 -Wall' --with-pydebug --with-trace-refs


to the configure script. In most any other program, breaking a few lines
ahead of where you wanted would just be an annoyance, but in the Python
virtual machine, it makes the breakpoint useless.

Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H5KAVDYA4BKRH6ZXIWTY246X662WOPW2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The repr of a sentinel

[Pascal Chambon]

Hello, and thanks for the PEP,

I feel like the 3-lines declaration of a new sentinel would discourage a 
bit its adoption compared to just "sentinel = object()"
From what I understand from the PEP, if new classes are defined inside 
the closure of a factory function, some Python implementations would 
have trouble copying/pickling them?


Would it be doable to have a single Sentinel class, whose instances 
store their representation and some autogenerated UUID, and which 
automatically return internally stored singletons (depending on this 
UUID) when called multiple times or unpickled ?
This would require some __new__() and unpickling magic, but nothing too 
CPython-specific (or am I missing something?).


regards,

Pascal


Le 24/05/2021 à 16:28, Tal Einat a écrit :

On Mon, May 24, 2021 at 3:30 AM Luciano Ramalho  wrote:

On Sun, May 23, 2021 at 3:37 AM Tal Einat  wrote:

I put up an early draft of a PEP on a branch in the PEPs repo:
https://github.com/python/peps/blob/sentinels/pep-.rst

Thanks for that PEP, Tal. Good ideas and recap there.

I think repr= should have a default: the name of the class within <>:
.

Sentinels don't have state or any other data besides a name, so I
would prefer not to force users to create a class just so they can
instantiate it.

Why not just this?

NotGiven = sentinel('')

I'm seriously considering that now. The issues I ran into with this
approach are perhaps not actually problematic.


On the other hand, if the user must create a class, the class itself
should be the sentinel. Class objects are already singletons, so that
makes sense.

Here is a possible class-based API:

class NotGiven(Sentinel):
 pass

That's it. Now I can use NotGiven as the sentinel, and its default
repr is .

Behind the scenes we can have a SentinelMeta metaclass with all the
magic that could be required--including the default __repr__ method.

What do you think?

One issue with that is that such sentinels don't have their own class,
so you can't write a strict type signature, such as `Union[str,
NotGivenType]`.

Another issue is that having these objects be classes, rather than
normal instances of classes, could be surprising and confusing.

For those two reasons, for now, I think generating a unique object
with its own unique class is preferable.


Sorry about my detour into the rejected idea of a factory function.

Please don't apologize! I put those ideas in the "Rejected Ideas"
section mostly to have them written down with a summary of the
considerations related to them. They shouldn't be considered finally
rejected unless and until the PEP is finished and accepted.

- Tal
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HL74JC3OF7Y3F5RDYVACAFODL4E3CBI6/
Code of Conduct: http://python.org/psf/codeofconduct/
.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JZEPSN4ZSAZ6QWXN75GFWQRJMJLPN37M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Implementing async enumerate

[Victor Stinner]

Hi,

I suggest to put such function in a Python module and publish it on
PyPI. I'm sure that the module will quickly grow with more async
flavor of existing functions.

Victor

On Tue, May 25, 2021 at 4:22 AM Filipe Alves Caixeta
 wrote:
>
> Hello,
>
> Python 3.10 comes with the built-in anext() but I think we may be missing 
> some other equivalent built-in functions like list and enumerate.
> I didn't discuss the idea of creating a PEP for this but I would like to get 
> some help with the implementation.
>
> The equivalent Python code would be
>
> async def aenumerate(sequence, start=0):
> n = start
> async for elem in sequence:
> yield n, elem
> n += 1
>
> The code for enumerate if here 
> https://github.com/python/cpython/blob/2f3a87856c7033227577b9ed0c77ed75311430b7/Objects/enumobject.c#L42
>
> The next function creates a tuple and return with (n, elem)
>
> result = PyTuple_New(2);
> PyTuple_SET_ITEM(result, 0, next_index);
> PyTuple_SET_ITEM(result, 1, next_item);
> return result;
>
> For the async implementation I would have to instead return an 
> async_generator_asend object
> and that would resolve to the tuple. Calling tp_as_async->am_anext and 
> getting the value doesn't seem like the solution because I should not get the 
> result at this stage yet.
>
> I would like to have a hint on how to implement this =D
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/7ZRSZAOM5NPF3DOMPS2XNS64L7XP5SBN/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TZBZXWHMZDQSW7BWXBJTNFKAJDAUCZLL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [python-committers] IMPORTANT: Python 3.10b2 release blockers

[Christian Heimes]

On 25/05/2021 00.45, Pablo Galindo Salgado wrote:
> Hi,
> 
> Tomorrow is the scheduled release of Python 3.10 beta 2 but
> unfortunately we have several release blockers:
>
> https://bugs.python.org/issue44043 :
> 3.10 b1 armhf Bus Error in hashlib test: test_gil

The problem is already fixed. I forgot to close the release blocker bug
after Greg and I took care of https://bugs.python.org/issue36515.

Christian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GSGRKOGDZICAX7RJRBWNVKHFQAE5EWVQ/
Code of Conduct: http://python.org/psf/codeofconduct/