date:20211220

18.12.21 23:07, Terry Reedy пише:
> Batuhan expresses my concerns better than I could, so I just add my
> agreement.
> 
> On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:
> 
>> tl;dr: I find it very troubling that we are going on a path where need
>> to increase the language complexity (syntax) only in the cause
>> 'easier' typing. So I am opposed to this change.

I concur.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DVREYSY6BIPCESCOT4QP2SV5CPG53LXB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Mark Shannon


OOI, of those 1577 Callable type hints, how many distinct Callable types?


On 20/12/2021 7:52 am, Andrew Svetlov wrote:

At my job, we have 1577 Callable type hints scattered in 1063 Python files.
For comparison, this codebase also has 2754 dict annotations and 1835 list ones.

On Mon, Dec 20, 2021 at 8:11 AM Christopher Barker mailto:python...@gmail.com>> wrote:

note: I wasn't thinking -- typeshed, of course, has a lot more than the 
standard lib.  But it's still a collection of widely used somewhat general 
purpose libraries. So I think my hypothesis is still valid.

-CHB


On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker mailto:python...@gmail.com>> wrote:

A question that came up for me is:

How common is it to need to use Callable for type hints? particularly 
complex versions, specifying what parameters the Callable takes? A more compact 
and easier to read syntax is nice, but not very important if it isn't used much.

My first thought on this was that I can't remember a single time that I 
wrote production code that took a Callable as a function parameter -- or 
returned one -- OK maybe a few times, but it's certainly rare in my production 
code.

So I looked in the PEP to see if that issue was addressed, and indeed 
it is:

"The Callable type is widely used. For example, as of October 2021 it was 
the fifth most common complex type in typeshed,"

That did surprise me, but on thinking about it, maybe not so much. It 
strikes me that Callable is most likely to be used in fairly low level, general 
purpose functions, like map(), sort(), various functions in itertools, etc. 
Just the sort of functions that are common in the standard library, but may not 
so much in production code.

I have no idea how to evaluate how common it is in production code -- 
maybe type hinting is common enough now  that PyPi could be searched -- but 
even PyPi is a biased sample, as it is full of, by definition, libraries for 
others' use -- i.e. general purpose tools (less general that the stad lib, but 
still not specialty production code, which I suspect is the majority of Python 
code out there).

Perhaps some folks that have been type=hinting their production code 
bases could provide anecdotal evidence.

Anyway, if my hypothesis is correct, then it's not so bad that 
not-so-nice syntax is required to type hint general purpose utilities.

-CHB

-- 
Christopher Barker, PhD (Chris)


Python Language Consulting
   - Teaching
   - Scientific Software Development
   - Desktop GUI and Web Development
   - wxPython, numpy, scipy, Cython



-- 
Christopher Barker, PhD (Chris)


Python Language Consulting
   - Teaching
   - Scientific Software Development
   - Desktop GUI and Web Development
   - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org 

To unsubscribe send an email to python-dev-le...@python.org 

https://mail.python.org/mailman3/lists/python-dev.python.org/ 

Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
 

Code of Conduct: http://python.org/psf/codeofconduct/ 




--
Thanks,
Andrew Svetlov

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JYZGIDEBV4R5E7XXT3KFS2O545TDTAGT/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RQ32VDXY7KUXHCUM6DASPVZ434PN5GLM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

20.12.21 13:42, Mark Shannon пише:
> OOI, of those 1577 Callable type hints, how many distinct Callable types?

Around 15-20%. Most of them are in tests which widely use pytest
fixtures, so functions taking and returning callables are common. There
are around 200 occurrences in non-test code, half of them are distinct
types.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Mark Shannon


Hi,

Why not make Callable usable as a function decorator?



The motivating example in the PEP is this:


def flat_map(
l: list[int],
func: Callable[[int], list[int]]
) -> list[int]:



Since, as the PEP claims, `Callable[[int], list[int]]` is hard to read, then 
give it a name and use regular function definition syntax.


@Callable
def IntToIntFunc(a:int)->int:
pass


def flat_map(
l: list[int],
func: IntToIntFunc
) -> list[int]:



To me, this seems much clearer than the proposed syntax and is more general.


It is a little longer, but unless you are playing code golf, that shouldn't 
matter.

Cheers,
Mark.



On 16/12/2021 5:57 pm, Steven Troxler wrote:

Hello all,

Thanks everyone for comments on our earlier thread [1] about callable type 
syntax. We now  have a draft PEP [2] proposing an arrow-based syntax for 
callable types, for example:

```
(int, str) -> bool # equivalent to Callable[[int, str], bool]
```

In support of the PEP we also have:
- a reference implementation of the parser [3] to ensure the grammar is correct 
 (tests [5], [6], [7])
- a detailed specification of planned runtime behavior [4], which is not yet in 
the reference implementation

We'd like to get your feedback about the PEP in general, and especially details 
and edge cases we need to consider regarding runtime behavior.

Cheers,
Steven Troxler

-
[1] Earlier python-dev thread 
https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/
[2] PEP 677: https://www.python.org/dev/peps/pep-0677/
[3] Reference implementation of Parser: 
https://github.com/stroxler/cpython/tree/callable-type-syntax--shorthand
[4] Details on the runtime behavior:  
https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG71Lk/edit

[5] Ast tests for parser changes:
https://github.com/stroxler/cpython/blob/20eb59fdca0d6d8dbe4efa3b04038c7c22024654/Lib/test/test_ast.py#L359-L392
[6] Easy-read tests of examples from the PEP: 
https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
[7] Test sanity checking hundreds of examples pulled from typeshed:
https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OGACYN2X7RX2GHAUP2AKRPT6DP432VCN/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2BKD5YBU7WJMUY3TSX34HX5IICT5UFRQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Andrew Svetlov

On Mon, Dec 20, 2021 at 1:42 PM Mark Shannon  wrote:

> OOI, of those 1577 Callable type hints, how many distinct Callable types?
>
>
Good question. About 30 callables for source code itself and an additional
60 for pytest factory fixtures.


> On 20/12/2021 7:52 am, Andrew Svetlov wrote:
> > At my job, we have 1577 Callable type hints scattered in 1063 Python
> files.
> > For comparison, this codebase also has 2754 dict annotations and 1835
> list ones.
> >
> > On Mon, Dec 20, 2021 at 8:11 AM Christopher Barker  > wrote:
> >
> > note: I wasn't thinking -- typeshed, of course, has a lot more than
> the standard lib.  But it's still a collection of widely used somewhat
> general purpose libraries. So I think my hypothesis is still valid.
> >
> > -CHB
> >
> >
> > On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker <
> python...@gmail.com > wrote:
> >
> > A question that came up for me is:
> >
> > How common is it to need to use Callable for type hints?
> particularly complex versions, specifying what parameters the Callable
> takes? A more compact and easier to read syntax is nice, but not very
> important if it isn't used much.
> >
> > My first thought on this was that I can't remember a single time
> that I wrote production code that took a Callable as a function parameter
> -- or returned one -- OK maybe a few times, but it's certainly rare in my
> production code.
> >
> > So I looked in the PEP to see if that issue was addressed, and
> indeed it is:
> >
> > "The Callable type is widely used. For example, as of October
> 2021 it was the fifth most common complex type in typeshed,"
> >
> > That did surprise me, but on thinking about it, maybe not so
> much. It strikes me that Callable is most likely to be used in fairly low
> level, general purpose functions, like map(), sort(), various functions in
> itertools, etc. Just the sort of functions that are common in the standard
> library, but may not so much in production code.
> >
> > I have no idea how to evaluate how common it is in production
> code -- maybe type hinting is common enough now  that PyPi could be
> searched -- but even PyPi is a biased sample, as it is full of, by
> definition, libraries for others' use -- i.e. general purpose tools (less
> general that the stad lib, but still not specialty production code, which I
> suspect is the majority of Python code out there).
> >
> > Perhaps some folks that have been type=hinting their production
> code bases could provide anecdotal evidence.
> >
> > Anyway, if my hypothesis is correct, then it's not so bad that
> not-so-nice syntax is required to type hint general purpose utilities.
> >
> > -CHB
> >
> > --
> > Christopher Barker, PhD (Chris)
> >
> > Python Language Consulting
> >- Teaching
> >- Scientific Software Development
> >- Desktop GUI and Web Development
> >- wxPython, numpy, scipy, Cython
> >
> >
> >
> > --
> > Christopher Barker, PhD (Chris)
> >
> > Python Language Consulting
> >- Teaching
> >- Scientific Software Development
> >- Desktop GUI and Web Development
> >- wxPython, numpy, scipy, Cython
> > ___
> > Python-Dev mailing list -- python-dev@python.org  python-dev@python.org>
> > To unsubscribe send an email to python-dev-le...@python.org  python-dev-le...@python.org>
> > https://mail.python.org/mailman3/lists/python-dev.python.org/ <
> https://mail.python.org/mailman3/lists/python-dev.python.org/>
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
> <
> https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
> >
> > Code of Conduct: http://python.org/psf/codeofconduct/ <
> http://python.org/psf/codeofconduct/>
> >
> >
> >
> > --
> > Thanks,
> > Andrew Svetlov
> >
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JYZGIDEBV4R5E7XXT3KFS2O545TDTAGT/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MG2PKYBIZHA3PPEVWJMSXR4WJ4TLBKVZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Andrew Svetlov

Perhaps Serhiy did more accurate counting, my estimate is very rough.

On Mon, Dec 20, 2021 at 2:15 PM Serhiy Storchaka 
wrote:

> 20.12.21 13:42, Mark Shannon пише:
> > OOI, of those 1577 Callable type hints, how many distinct Callable types?
>
> Around 15-20%. Most of them are in tests which widely use pytest
> fixtures, so functions taking and returning callables are common. There
> are around 200 occurrences in non-test code, half of them are distinct
> types.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q4A4ZTJHSS6ALDXHZJEERJBVGJNYC6KV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] About the new CFrame structure

2021-12-20 Thread Gabriele

Hi there

I hope you would indulge me in asking for some details about the new
CFrame structure, even in the form of existing literature (e.g. PEP)
where the idea behind it is explained.

Also, I'd like to a quick question, if I may. There now appear to be
two ways of unwinding the frame stack: either iterate over
CFrame.previous, or the more traditional PyFrameObject.f_back. I
suspect there are reasons why these are perhaps not actually
equivalent, and indeed this is mainly what I'd like to read in the
literature I've requested above.

Cheers,
Gabriele

-- 
"Egli è scritto in lingua matematica, e i caratteri son triangoli,
cerchi, ed altre figure
geometriche, senza i quali mezzi è impossibile a intenderne umanamente parola;
senza questi è un aggirarsi vanamente per un oscuro laberinto."

-- G. Galilei, Il saggiatore.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KQOQTLR5IXMJXYZGPDHWR32I2Z53UVBL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Steven Troxler

In most of our discussions of this idea, we've assumed that we'd adopt the same 
semantics that callback protocols use.

If we do that, then only `lambda a: 3` will type check. In order to type check 
both you'd have to make `a` positional-only:
```
def IntToIntFunc(a: int, /) -> int:
...
```

This is one of the reasons I think functions-as-types could be a great idea but 
not a good substitute for better callable syntax.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6EB54EZXUVFR3HPTGLQK4AGJ7UUR5K4F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The python command help is too long

2021-12-20 Thread Barry Warsaw

On Dec 20, 2021, at 12:22, Guido van Rossum  wrote:
> What do you think about -hh (and maybe —help-full) printing full help?
> 
> Is there enough of a use case for this to bother?

Maybe not.  I’d say if it was easy to implement, why not, but if it’s a pain, 
don't bother.

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T66U3KWST6OQQS37FAX3VXULC3O6GAVB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: About the new CFrame structure

2021-12-20 Thread Brandt Bucher

Hi Gabriele!

> I hope you would indulge me in asking for some details about the new CFrame 
> structure, even in the form of existing literature (e.g. PEP) where the idea 
> behind it is explained.

There isn't too much documentation on this, unfortunately (since these are all 
very unstable, low-level interpreter details), but a good place to start would 
be https://bugs.python.org/issue46090.

Based on my own understanding (from reading the source code):
- There are three relevant structures: CFrame, InterpreterFrame, and 
PyFrameObject.
  - PyFrameObjects are just PyObject wrappers around an InterpreterFrame, where 
all of the *actual* frame state for the Python stack is maintained.
   - They are created lazily (for example, when sys._getframe() is called).
 - See https://github.com/python/cpython/pull/27077.
  - InterpreterFrames live in a "datastack" for fast allocation and 
deallocation.
- This "datastack" lives on the PyThreadState.
- Because of how it is designed, InterpreterFrames must be 
allocated/deallocated "in order".
- If an InterpreterFrame is cleared, but still has a live PyFrameObject 
that points to it, it will copy itself *into* the PyFrameObject first (to 
guarantee that the PyFrameObject keeps working).
- See https://github.com/python/cpython/pull/26076.
  - A single CFrame is statically allocated inside of each 
_PyEval_EvalFrameDefault call, so it corresponds to the C stack, not the Python 
stack.
- It links to a chain of one or more InterpreterFrames.
- Multiple InterpreterFrames can correspond to a single CFrame!
  -  This is a performance optimization in 3.11: rather than enter a new 
call to _PyEval_EvalFrameDefault, calls to pure-Python code just create a new 
InterpreterFrame, set it as the current one, and continue execution.
  - You can see how many InterpreterFrames correspond to the current CFrame 
by reading the "depth" member of the current InterpreterFrame.
- A value of 0 indicates that this is the only InterpreterFrame for 
this CFrame.
- A value of 42 means that this optimization has been performed 42 
times (and there are currently 43 InterpreterFrames executing in this CFrame).

> Also, I'd like to a quick question, if I may. There now appear to be two ways 
> of unwinding the frame stack: either iterate over CFrame.previous, or the 
> more traditional PyFrameObject.f_back. I suspect there are reasons why these 
> are perhaps not actually equivalent, and indeed this is mainly what I'd like 
> to read in the literature I've requested above.

The above outline probably makes the differences clear:
- PyFrameObject.f_back just gives you a dummy wrapper around the previous frame 
object.
  - It's not really useful for unwinding anything.
- InterpreterFrame.previous gives you the previous interpreter frame (duh!).
  - This is probably what you want.
- CFrame.previous gives you the previous call to _PyEval_EvalFrameDefault.
  - It's not really useful for unwinding anything.
  - This is only really useful to maintain the current tracing state when 
returning. 

Hopefully this helps! Somebody (Pablo or Mark?) will probably jump in here if I 
got anything wrong.

Brandt
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GDCPNESE2BWJUNPYFANCZVZK4EZNTKAF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The python command help is too long

On Sat, Dec 18, 2021 at 4:00 PM Guido van Rossum  wrote:

> On Sat, Dec 18, 2021 at 2:16 PM Serhiy Storchaka 
> wrote:
>
>> The output of "python -h" is 104 lines long now. It was only 51 lines in
>> 3.6. 35% of it is about the -X option, and 30% about environment
>> variables. Also some lines in the -X option description are too long
>> (102 columns). Both topics are "advanced" and mostly interested for
>> debugging. I suggest to move them out of the main help output.
>>
>
> Sounds like a good plan.
>
>
>> The are two options:
>>
>> 1. Move it to pydoc topics. The advantage is that it is a standard way,
>> there are already 88 topics. The disadvantage is that this information
>> will be not available in "minimal" installations of Python which do not
>> include docs.
>>
>> 2. Add command-line options like -hX and -henv. The information will
>> always be available with the interpreter, but the interface is special.
>>
>
> For -X, I suggest we could output the following line:
>
> -X opt : implementation-specific option; use -X help to list options.
>

+1 from me.


>
> We could also see if we can put the help text for each of the supported -X
> flags in the table defining these flags (sorry, I can't recall where it
> lives, but I'm pretty sure I've seen such a table.)
>
> For env vars I think moving this to pydoc is fine; I don't think we have a
> process or mechanism that ensures the docs for env vars are even complete.
> (We don't have one for the flags either, but somehow I find it hard to
> conceive of someone adding a new command line flag without them or someone
> else involved in the code review thinking of updating the help text. But I
> find it quite conceivable that someone adds a new env var used only in a
> corner case without anyone thinking to update the docs. And I presume that
> the -E flag isn't honored 100% of the time either.)
>
> But I wouldn't object to a -h sub-option that lists environment vars
> either.
>

I think an option to `-h` for environment variables is also good as that
command can be the generic `-h` output as well without resorting to needing
to run Python code like pydoc to get at more help.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BSXEPG6ERPYCPPQ6WVAF3NJU5CVFFJ4Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

On Sun, Dec 19, 2021 at 8:26 PM Christopher Barker 
wrote:

>
> On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:
>>
>> > tl;dr: I find it very troubling that we are going on a path where need
>> > to increase the language complexity (syntax) only in the cause
>> > 'easier' typing.
>
>
> Which brings up the question is whether it's worth adding  syntax for
> typing, but only in the context of typing.
>

The SC has already said we don't like that idea as that bifurcates the
knowledge one needs in order to even have a chance at comprehending a type
hint. Plus PEP 649 wouldn't be possible in that instance unless we ship a
second parser just for type hints in order to translate the type-specific
syntax to type-related objects.

-Brett


> As of right now, typing.get_type_hints() will evaluate a string
> annotation, e.g.
>
> In [62]: def f(x:"int"):
> ...: pass
> ...:
>
> In [63]: typing.get_type_hints(f)
> Out[63]: {'x': int}
>
> so get_type_hints could extend its acceptable syntax with this new use of
> -> -- and it could get used by wrapping it in quotes. And depending on
> how PEP 563 gets resolved, the quotes may not be necessary in the future.
>
> And this could open up some other nifty things, like extending what's
> allowable inside [] -- there was a discussion a while back on python-ideas
> about extending the __getitem__ protocol, partly motivated by type hints.
>
> -CHB
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JWPMHLSPFTYAYSMWCX7LOXWBEU4FA6DC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N7HF4FDSPEPJDWFD56NFUPHOEBU4AS7D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: About the new CFrame structure

Hi Gabriele,

I think the code is currently the only documentation, since this is
considered a CPython internal issue. I'm CC'ing Mark Shannon, since he
designed this for 3.11.

For a bit of background, see this issue in the Faster CPython tracker:
https://github.com/faster-cpython/ideas/issues/31 -- maybe this will help
you understand the background (though what was implemented doesn't match
the discussion there precisely).

Since you seem to be the author of Austin, a sampling profiler for Python
code (which I presume itself is written in C or C++), I'm guessing you're
asking because you're looking to make Austin work with 3.11. (Actually, the
CFrame structure appears in 3.10, but it's not usable for unwinding stack
frames in 3.10, since the 3.10 version doesn't have a pointer to the
interpreter frame -- it's only used to speed up access to the `use-tracing`
flag. So I'm assuming you're asking about 3.10.)

One of the goals for the Faster CPython project that has only become in
focus for the team recently is helping tools like Austin (and others like
Cython, PyDev and Greenlets) that use internal interpreter state figure out
how to get their code working in 3.11, without making too many commitments
to stable APIs -- we're already thinking about more performance
improvements in 3.12 and beyond, which most likely will require us to
change the internal data structures again. (We do commit to leaving these
internal data structures unchanged for bugfix release cycles, so that once
you've got it working for 3.11.0, it should work without changes for
3.11.1, 3.11.2, etc. In fact we hope to stabilize everything starting with
the release of 3.11b1, May 2020.)

Oh, I see that Brandt has already given a better overview of how this stuff
actually works (he's a faster typist than me :-), so I'll just sign off
here. If you have more specific questions, you can continue this thread, or
you can create a new Discussion item in the
https://github.com/faster-cpython/ideas repo.

--Guido

On Mon, Dec 20, 2021 at 10:25 AM Gabriele  wrote:

> Hi there
>
> I hope you would indulge me in asking for some details about the new
> CFrame structure, even in the form of existing literature (e.g. PEP)
> where the idea behind it is explained.
>
> Also, I'd like to a quick question, if I may. There now appear to be
> two ways of unwinding the frame stack: either iterate over
> CFrame.previous, or the more traditional PyFrameObject.f_back. I
> suspect there are reasons why these are perhaps not actually
> equivalent, and indeed this is mainly what I'd like to read in the
> literature I've requested above.
>
> Cheers,
> Gabriele
>
> --
> "Egli è scritto in lingua matematica, e i caratteri son triangoli,
> cerchi, ed altre figure
> geometriche, senza i quali mezzi è impossibile a intenderne umanamente
> parola;
> senza questi è un aggirarsi vanamente per un oscuro laberinto."
>
> -- G. Galilei, Il saggiatore.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/KQOQTLR5IXMJXYZGPDHWR32I2Z53UVBL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5MUUF4FPKW5PANCUQ55C2ZYHSSRVLSFW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: About the new CFrame structure

2021-12-20 Thread Pablo Galindo Salgado

Hi Gabriele,

In addition to what Guido and Brandt have already said, I can help to you
adapting Austin to 3.11 as I reviewed or authored some of these changes and
I have already been helping some projects do the relevant changes as well
as in my own tools.

What you want to do si the following for unwinding:

* Go from _PyRuntime -> PyThreadState -> CFrame -> current_frame

This will lead you to s PyInterpreterFrame that you should use for
unwinding the entire thread stack. The difference is that cframe->previous
will skip you several frames as it points to the previous CFrame, but there
are a one to many relationships between CFrame and interpreter frames
because several python functions can now reuse the same evaluation loop.

Also, I would recommend waiting until beta freeze to start adapting
anything as things can still massively change until then for 3.11.

If you have any questions or you need help, feel free to ping me in GitHub
if you want.

Regards from rainy London,
Pablo Galindo Salgado

On Mon, 20 Dec 2021, 18:27 Gabriele,  wrote:

> Hi there
>
> I hope you would indulge me in asking for some details about the new
> CFrame structure, even in the form of existing literature (e.g. PEP)
> where the idea behind it is explained.
>
> Also, I'd like to a quick question, if I may. There now appear to be
> two ways of unwinding the frame stack: either iterate over
> CFrame.previous, or the more traditional PyFrameObject.f_back. I
> suspect there are reasons why these are perhaps not actually
> equivalent, and indeed this is mainly what I'd like to read in the
> literature I've requested above.
>
> Cheers,
> Gabriele
>
> --
> "Egli è scritto in lingua matematica, e i caratteri son triangoli,
> cerchi, ed altre figure
> geometriche, senza i quali mezzi è impossibile a intenderne umanamente
> parola;
> senza questi è un aggirarsi vanamente per un oscuro laberinto."
>
> -- G. Galilei, Il saggiatore.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/KQOQTLR5IXMJXYZGPDHWR32I2Z53UVBL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F4QZ3UXCW3ZLTUUKD3L4XFHHDWPZGJV4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

On Mon, Dec 20, 2021 at 12:44 PM  wrote:

> This is such a great idea that I think it deserves its own PEP (to compete
> with this one?) Let me explain. PEP 677 was created for the sole purpose of
> replacing typing.Callable, but there are still some other areas where
> function metadata is required. What if we instead introduced a function
> prototype that allows you to "declare" a "function" without a body.
>
> typing example:
>
> import typing
>
> @typing.Callable
> def IntToIntFunc(a: int) -> int
>
> def flat_map(
> l: list[int],
> func: IntToIntFunc
> ) -> list[int]:
> ...
>
> ctypes example:
>
> import ctypes
>
> @ctypes.CFUNCTYPE
> def f(x: int) -> bool
>
> But of course this comes with a few issues: should it be an expression and
> if so, should the name be optional? How can ParamSpec be handled?
>

Allowing 'def' without a body based on the presence or absence of a
decorator sounds like it will just confuse people and cause bizarre errors
if people accidentally leave out a body. Let's not go there.

However, Lukasz has already proposed a very similar mechanism (with dummy
body), without needing a decorator. His proposal is simply:

def IntToIntFunc(a: int) -> int: ...

def flat_map(l: list[int], func: IntToIntFunc) -> list[int]:
# body

No need for a `@Callable` decorator!

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/64RWUSM5IR7ZLDPGIMVWEWXC4GVKUPW4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

On Mon, Dec 20, 2021 at 3:44 AM Mark Shannon  wrote:

> Hi,
>
> Why not make Callable usable as a function decorator?
>
>
>
> The motivating example in the PEP is this:
>
>
> def flat_map(
>  l: list[int],
>  func: Callable[[int], list[int]]
> ) -> list[int]:
>  
>
>
> Since, as the PEP claims, `Callable[[int], list[int]]` is hard to read,
> then give it a name and use regular function definition syntax.
>
>
> @Callable
> def IntToIntFunc(a:int)->int:
>  pass
>
>
> def flat_map(
>  l: list[int],
>  func: IntToIntFunc
> ) -> list[int]:
>  
>
>
> To me, this seems much clearer than the proposed syntax and is more
> general.
>
>
> It is a little longer, but unless you are playing code golf, that
> shouldn't matter.
>

It's an interesting idea! Both `@overload` and `@final` show there is
precedence for having decorators have special meanings to static type
checkers:
https://docs.python.org/3/library/typing.html#functions-and-decorators.


>
> Cheers,
> Mark.
>
>
>
> On 16/12/2021 5:57 pm, Steven Troxler wrote:
> > Hello all,
> >
> > Thanks everyone for comments on our earlier thread [1] about callable
> type syntax. We now  have a draft PEP [2] proposing an arrow-based syntax
> for callable types, for example:
> >
> > ```
> > (int, str) -> bool # equivalent to Callable[[int, str], bool]
> > ```
> >
> > In support of the PEP we also have:
> > - a reference implementation of the parser [3] to ensure the grammar is
> correct  (tests [5], [6], [7])
> > - a detailed specification of planned runtime behavior [4], which is not
> yet in the reference implementation
> >
> > We'd like to get your feedback about the PEP in general, and especially
> details and edge cases we need to consider regarding runtime behavior.
> >
> > Cheers,
> > Steven Troxler
> >
> > -
> > [1] Earlier python-dev thread
> https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/
> > [2] PEP 677: https://www.python.org/dev/peps/pep-0677/
> > [3] Reference implementation of Parser:
> https://github.com/stroxler/cpython/tree/callable-type-syntax--shorthand
> > [4] Details on the runtime behavior:
> https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG71Lk/edit
> >
> > [5] Ast tests for parser changes:
> >
> https://github.com/stroxler/cpython/blob/20eb59fdca0d6d8dbe4efa3b04038c7c22024654/Lib/test/test_ast.py#L359-L392
> > [6] Easy-read tests of examples from the PEP:
> https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
> > [7] Test sanity checking hundreds of examples pulled from typeshed:
> >
> https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OGACYN2X7RX2GHAUP2AKRPT6DP432VCN/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2BKD5YBU7WJMUY3TSX34HX5IICT5UFRQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OCLGF6IXGHVBCCA24ZFJMUEFJ2DJVAQX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The python command help is too long

On Mon, Dec 20, 2021 at 11:15 AM Brett Cannon  wrote:

>
>
> On Sat, Dec 18, 2021 at 4:00 PM Guido van Rossum  wrote:
>
>> On Sat, Dec 18, 2021 at 2:16 PM Serhiy Storchaka 
>> wrote:
>>
>>> The output of "python -h" is 104 lines long now. It was only 51 lines in
>>> 3.6. 35% of it is about the -X option, and 30% about environment
>>> variables. Also some lines in the -X option description are too long
>>> (102 columns). Both topics are "advanced" and mostly interested for
>>> debugging. I suggest to move them out of the main help output.
>>>
>>
>> Sounds like a good plan.
>>
>>
>>> The are two options:
>>>
>>> 1. Move it to pydoc topics. The advantage is that it is a standard way,
>>> there are already 88 topics. The disadvantage is that this information
>>> will be not available in "minimal" installations of Python which do not
>>> include docs.
>>>
>>> 2. Add command-line options like -hX and -henv. The information will
>>> always be available with the interpreter, but the interface is special.
>>>
>>
>> For -X, I suggest we could output the following line:
>>
>> -X opt : implementation-specific option; use -X help to list options.
>>
>
> +1 from me.
>
>
>>
>> We could also see if we can put the help text for each of the supported
>> -X flags in the table defining these flags (sorry, I can't recall where it
>> lives, but I'm pretty sure I've seen such a table.)
>>
>> For env vars I think moving this to pydoc is fine; I don't think we have
>> a process or mechanism that ensures the docs for env vars are even
>> complete. (We don't have one for the flags either, but somehow I find it
>> hard to conceive of someone adding a new command line flag without them or
>> someone else involved in the code review thinking of updating the help
>> text. But I find it quite conceivable that someone adds a new env var used
>> only in a corner case without anyone thinking to update the docs. And I
>> presume that the -E flag isn't honored 100% of the time either.)
>>
>> But I wouldn't object to a -h sub-option that lists environment vars
>> either.
>>
>
> I think an option to `-h` for environment variables is also good as that
> command can be the generic `-h` output as well without resorting to needing
> to run Python code like pydoc to get at more help.
>

Fair enough. Quick design proposal:

  -h and --help print info about flags (existing flags)
  --help-env (or --env-help?) prints info about env vars (new flag)
  -X help prints info about -X options (new -X option)

Two lines printed at the end of the -h/--help output would refer users to
--help-env and -Xhelp.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NF4LTZWAMOATEA4ZZLIGVIJOATVIYMF2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: About the new CFrame structure

2021-12-20 Thread Brandt Bucher

Just to clear up a quick point I made:

> - PyFrameObject.f_back just gives you a dummy wrapper around the previous 
> frame object.
>   - It's not really useful for unwinding anything.

That should read "previous InterpreterFrame", rather than "previous frame 
object".

Also, everything I wrote above is in the context of 3.11. InterpreterFrames 
don't exist in 3.10 and below, so in those versions PyFrameObject.f_back is 
indeed what you probably want.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TKPWWP33QJJEVEIP63C4SIEMVBY44LCW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread asleep . cult

This is such a great idea that I think it deserves its own PEP (to compete with 
this one?) Let me explain. PEP 677 was created for the sole purpose of 
replacing typing.Callable, but there are still some other areas where function 
metadata is required. What if we instead introduced a function prototype that 
allows you to "declare" a "function" without a body.

typing example:

import typing

@typing.Callable
def IntToIntFunc(a: int) -> int

def flat_map(
l: list[int],
func: IntToIntFunc
) -> list[int]:
...

ctypes example:

import ctypes

@ctypes.CFUNCTYPE
def f(x: int) -> bool

But of course this comes with a few issues: should it be an expression and if 
so, should the name be optional? How can ParamSpec be handled?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XPPZZZWVVW6KRBKYXJKXHPTECRDIOFUE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

As someone with use of this, would you find this useful (i.e. +1, +0)?
Serhiy already said "no" in another thread.

On Mon, Dec 20, 2021 at 4:38 AM Andrew Svetlov 
wrote:

> Perhaps Serhiy did more accurate counting, my estimate is very rough.
>
> On Mon, Dec 20, 2021 at 2:15 PM Serhiy Storchaka 
> wrote:
>
>> 20.12.21 13:42, Mark Shannon пише:
>> > OOI, of those 1577 Callable type hints, how many distinct Callable
>> types?
>>
>> Around 15-20%. Most of them are in tests which widely use pytest
>> fixtures, so functions taking and returning callables are common. There
>> are around 200 occurrences in non-test code, half of them are distinct
>> types.
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Thanks,
> Andrew Svetlov
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/Q4A4ZTJHSS6ALDXHZJEERJBVGJNYC6KV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2OPXHGWUDCFTDIYEARIRJFIGWLM6JTLR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The python command help is too long

2021-12-20 Thread Barry Warsaw

On Dec 20, 2021, at 11:34, Guido van Rossum  wrote:
> 
> Fair enough. Quick design proposal:
> 
>   -h and --help print info about flags (existing flags)
>   --help-env (or --env-help?) prints info about env vars (new flag)
>   -X help prints info about -X options (new -X option)
> 
> Two lines printed at the end of the -h/--help output would refer users to 
> --help-env and -Xhelp.

+1, and —help-env seems better.

What do you think about -hh (and maybe —help-full) printing full help?

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BJVQBZSBRJU74NWUYPQCHVGL42MT6W7Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The python command help is too long

On Mon, Dec 20, 2021 at 12:16 PM Barry Warsaw  wrote:

> On Dec 20, 2021, at 11:34, Guido van Rossum  wrote:
> >
> > Fair enough. Quick design proposal:
> >
> >   -h and --help print info about flags (existing flags)
> >   --help-env (or --env-help?) prints info about env vars (new flag)
> >   -X help prints info about -X options (new -X option)
> >
> > Two lines printed at the end of the -h/--help output would refer users
> to --help-env and -Xhelp.
>
> +1, and —help-env seems better.
>

Sure.

What do you think about -hh (and maybe —help-full) printing full help?
>

Is there enough of a use case for this to bother?

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BSCJIDR4BJYB2OZH7EU4NOKBFYFZIFBT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The python command help is too long

20.12.21 21:34, Guido van Rossum пише:
> Fair enough. Quick design proposal:
> 
>   -h and --help print info about flags (existing flags)
>   --help-env (or --env-help?) prints info about env vars (new flag)
>   -X help prints info about -X options (new -X option)
> 
> Two lines printed at the end of the -h/--help output would refer users
> to --help-env and -Xhelp.

Thank you Guido and Barry for your suggestions. It looks great!

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7PL3GAK26HXX3PLFU6MSHNUHKOYE3D7G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Patrick Reader

On 20/12/2021 22:34, Serhiy Storchaka wrote:
> 20.12.21 21:28, Brett Cannon пише:
>> As someone with use of this, would you find this useful (i.e. +1, +0)?
>> Serhiy already said "no" in another thread.
> In every file we import 5-10 or more names from the typing module. We
> still does not use PEP 585 and PEP 604 syntax, but are going to do this
> in near future. It could save as from importing List, Tuple, Dict, Set,
> Union, Optional, but we still need to import Any, Sequence, Iterable,
> Iterator, AsyncIterator, Awaitable, TypeVar, and sometimes
> AsyncContextManager, NewType, cast, overload. Special syntax for
> callable type hints will not save game.
Sequence, Iterable, Iterator, AsyncIterator, Awaitable, AsyncContextManager are 
all in collections.abc not typing now. Of course it doesn't reduce total 
imports but it makes it less likely that you'll need to import typing.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PVF4G57RJ634QYFJ2HGI3QQ5RXB6J455/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Gregory Beauregard via Python-Dev

Does `lambda b: 3` type check with `IntToIntFunc` or only `lambda a: 3`? The 
intent seems to be it's more like `Callable` (where the argument name is not 
important), but maybe both could be supported? I wonder about making more use 
of the `_` soft keyword where calling a function with multiple `_` is a runtime 
error. Maybe it would make too much of a mess.

After some testing evidently mypy only applies its knowledge sometimes anyway: 
https://github.com/python/mypy/issues/11807
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MDR2TCLEN7YKNUHKID5D6KMBJL73UBOZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Mehdi2277

I don’t see how this helps much in situations with nested callables which are
very common for decorators. I’m also unsure how this will work with situations
where input and output are both callables and the signature is modified using
paramspec. An example would be a decorator that adds/removes one argument. Is
Concatenate[int, InttoIntFunction] a thing?

I also would ideally want a very light weight way to handle simple callables
like (int) -> bool. This decorator solution of @Callable seems roughly as
readable as existing solution of using callback protocols or a type alias.

I took a quick scan of my team’s codebase and see about 150 callable vs like
1.5k optional. Optional is fairly high as a lot of our code has many arguments
that can be disabled which I think is normal for ml/numerical libraries with
tons of tuning settings. The most complex callable signatures tend to be
decorators/decorator factories for us.

Main place I find this useful is skimming other libraries type signatures and
hovering in ides. Nice hover type hints is helpful over large blocks of
callable brackets.

I do find this syntactic sugar nice. I’m +0.5, mainly as while I think current
callable generic is messy, I also think most of the people that write those
type annotations are the ones that write a lot of them. Maybe that would change
with better readability but right now heavy callable type usage feel like an
advanced, uncommon thing. Part of it is also heavy functional style code isn’t
that common in python.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/EVE274LSWE5CDPJYW3VVVPJENVQIQMVM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: About the new CFrame structure

2021-12-20 Thread Gabriele

Brandt, Guido, Pablo

Many thanks for your helpful answers. Indeed I'm asking because I just
finished working on some improvements to Austin and got back to looking
into what was coming in order to add support for 3.11 (plus make use of
some of the changes that I recently contributed, like
PyThreadState.native_thread_id, Py_Version and code.co_qualname). Pablo's
suggestion of waiting until 3.11 becomes more stable is a sensible one, but
in the meantime I wanted to understand where the frame stack management is
heading to at least have an idea of what's in store for Austin :). I'm very
much satisfied by all the details in your replies as they answer all my
questions, for now, so thanks a lot for that.

A final thought: I have discussed some of the technical details of the
recent improvements to Austin in a blog post (
https://p403n1x87.github.io/increasing-austin-accuracy-with-a-dobule-heap-trick.html).
My experiments seem to suggest that, the less sparse the frame objects are
in memory, the more accurate tools like Austin can be. So this makes me
wonder if it would make sense for CPython to ensure frame objects are
created in a contiguous block of memory (perhaps there could be benefits
from the locality of reference, although it's not obvious to me why this
would be the case at the moment).

Best,
Gabriele

On Mon, 20 Dec 2021 at 20:16, Pablo Galindo Salgado 
wrote:

> Hi Gabriele,
>
> In addition to what Guido and Brandt have already said, I can help to you
> adapting Austin to 3.11 as I reviewed or authored some of these changes and
> I have already been helping some projects do the relevant changes as well
> as in my own tools.
>
> What you want to do si the following for unwinding:
>
> * Go from _PyRuntime -> PyThreadState -> CFrame -> current_frame
>
> This will lead you to s PyInterpreterFrame that you should use for
> unwinding the entire thread stack. The difference is that cframe->previous
> will skip you several frames as it points to the previous CFrame, but there
> are a one to many relationships between CFrame and interpreter frames
> because several python functions can now reuse the same evaluation loop.
>
> Also, I would recommend waiting until beta freeze to start adapting
> anything as things can still massively change until then for 3.11.
>
> If you have any questions or you need help, feel free to ping me in GitHub
> if you want.
>
> Regards from rainy London,
> Pablo Galindo Salgado
>
>
>
> On Mon, 20 Dec 2021, 18:27 Gabriele,  wrote:
>
>> Hi there
>>
>> I hope you would indulge me in asking for some details about the new
>> CFrame structure, even in the form of existing literature (e.g. PEP)
>> where the idea behind it is explained.
>>
>> Also, I'd like to a quick question, if I may. There now appear to be
>> two ways of unwinding the frame stack: either iterate over
>> CFrame.previous, or the more traditional PyFrameObject.f_back. I
>> suspect there are reasons why these are perhaps not actually
>> equivalent, and indeed this is mainly what I'd like to read in the
>> literature I've requested above.
>>
>> Cheers,
>> Gabriele
>>
>> --
>> "Egli è scritto in lingua matematica, e i caratteri son triangoli,
>> cerchi, ed altre figure
>> geometriche, senza i quali mezzi è impossibile a intenderne umanamente
>> parola;
>> senza questi è un aggirarsi vanamente per un oscuro laberinto."
>>
>> -- G. Galilei, Il saggiatore.
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/KQOQTLR5IXMJXYZGPDHWR32I2Z53UVBL/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>

-- 

*"Egli è scritto in lingua matematica, e i caratteri son triangoli, cerchi,
ed altre figuregeometriche, senza i quali mezzi è impossibile a intenderne
umanamente parola;senza questi è un aggirarsi vanamente per un oscuro
laberinto."*

*-- *G. Galilei*, Il saggiatore.*
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YBAMU2YESAL66DB7CPG6JXJWQMFO2HNN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: About the new CFrame structure

2021-12-20 Thread Pablo Galindo Salgado

Hi Gabriele,

>> So this makes me wonder if it would make sense for CPython to ensure
frame objects are created in a contiguous block of memory (perhaps there
could be benefits from the locality of reference, although it's not obvious
to me why this would be the case at the moment).


That's already the case for 3.11: we currently allocate frames in a
continuous, per thread stack (it includes some of the frame memory and
arrays apart from the frames themselves).

Check this for more details:
https://github.com/python/cpython/pull/27077

Also, don't rely on this on any way or form as this code can and will
likely change a lot (even between patch versions if we found bugs).

Regards from rainy London,
Pablo Galindo Salgado



On Mon, 20 Dec 2021, 21:43 Gabriele,  wrote:

> Brandt, Guido, Pablo
>
> Many thanks for your helpful answers. Indeed I'm asking because I just
> finished working on some improvements to Austin and got back to looking
> into what was coming in order to add support for 3.11 (plus make use of
> some of the changes that I recently contributed, like
> PyThreadState.native_thread_id, Py_Version and code.co_qualname). Pablo's
> suggestion of waiting until 3.11 becomes more stable is a sensible one, but
> in the meantime I wanted to understand where the frame stack management is
> heading to at least have an idea of what's in store for Austin :). I'm very
> much satisfied by all the details in your replies as they answer all my
> questions, for now, so thanks a lot for that.
>
> A final thought: I have discussed some of the technical details of the
> recent improvements to Austin in a blog post (
> https://p403n1x87.github.io/increasing-austin-accuracy-with-a-dobule-heap-trick.html).
> My experiments seem to suggest that, the less sparse the frame objects are
> in memory, the more accurate tools like Austin can be. So this makes me
> wonder if it would make sense for CPython to ensure frame objects are
> created in a contiguous block of memory (perhaps there could be benefits
> from the locality of reference, although it's not obvious to me why this
> would be the case at the moment).
>
> Best,
> Gabriele
>
>
> On Mon, 20 Dec 2021 at 20:16, Pablo Galindo Salgado 
> wrote:
>
>> Hi Gabriele,
>>
>> In addition to what Guido and Brandt have already said, I can help to you
>> adapting Austin to 3.11 as I reviewed or authored some of these changes and
>> I have already been helping some projects do the relevant changes as well
>> as in my own tools.
>>
>> What you want to do si the following for unwinding:
>>
>> * Go from _PyRuntime -> PyThreadState -> CFrame -> current_frame
>>
>> This will lead you to s PyInterpreterFrame that you should use for
>> unwinding the entire thread stack. The difference is that cframe->previous
>> will skip you several frames as it points to the previous CFrame, but there
>> are a one to many relationships between CFrame and interpreter frames
>> because several python functions can now reuse the same evaluation loop.
>>
>> Also, I would recommend waiting until beta freeze to start adapting
>> anything as things can still massively change until then for 3.11.
>>
>> If you have any questions or you need help, feel free to ping me in
>> GitHub if you want.
>>
>> Regards from rainy London,
>> Pablo Galindo Salgado
>>
>>
>>
>> On Mon, 20 Dec 2021, 18:27 Gabriele,  wrote:
>>
>>> Hi there
>>>
>>> I hope you would indulge me in asking for some details about the new
>>> CFrame structure, even in the form of existing literature (e.g. PEP)
>>> where the idea behind it is explained.
>>>
>>> Also, I'd like to a quick question, if I may. There now appear to be
>>> two ways of unwinding the frame stack: either iterate over
>>> CFrame.previous, or the more traditional PyFrameObject.f_back. I
>>> suspect there are reasons why these are perhaps not actually
>>> equivalent, and indeed this is mainly what I'd like to read in the
>>> literature I've requested above.
>>>
>>> Cheers,
>>> Gabriele
>>>
>>> --
>>> "Egli è scritto in lingua matematica, e i caratteri son triangoli,
>>> cerchi, ed altre figure
>>> geometriche, senza i quali mezzi è impossibile a intenderne umanamente
>>> parola;
>>> senza questi è un aggirarsi vanamente per un oscuro laberinto."
>>>
>>> -- G. Galilei, Il saggiatore.
>>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/KQOQTLR5IXMJXYZGPDHWR32I2Z53UVBL/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>
> --
>
>
> *"Egli è scritto in lingua matematica, e i caratteri son triangoli,
> cerchi, ed altre figuregeometriche, senza i quali mezzi è impossibile a
> intenderne umanamente parola;senza questi è un aggirarsi vanamente per un
> oscuro laberinto."*
>
> *-- *G. Galilei*, Il

[Python-Dev] Re: RFC on Callable Syntax PEP

20.12.21 21:28, Brett Cannon пише:
> As someone with use of this, would you find this useful (i.e. +1, +0)?
> Serhiy already said "no" in another thread.

In every file we import 5-10 or more names from the typing module. We
still does not use PEP 585 and PEP 604 syntax, but are going to do this
in near future. It could save as from importing List, Tuple, Dict, Set,
Union, Optional, but we still need to import Any, Sequence, Iterable,
Iterator, AsyncIterator, Awaitable, TypeVar, and sometimes
AsyncContextManager, NewType, cast, overload. Special syntax for
callable type hints will not save game.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NLLN4IGOKCQZP7LYM7VBQSGGGWPW3WKZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

2021-12-20 Thread Steven Troxler

Thanks for the feedback. I have a few thoughts.


(1) Concerns about complexity of the syntax make sense to me, it's definitely 
possible to write confusing types with this syntax. Readability would be a good 
reason to reject this idea, but it does cut both ways because `Callable` can be 
hard to read today.

Typing-sig as a whole is confident that an arrow syntax would be an improvement 
but I think the submission process is a good time for wider opinions, I can see 
that it might not be a good idea overall even if it's a better type syntax.


(2) I do like the idea of allowing parameters in a tuple in the existing 
Callable type, that seems like a very clear easy win if we *don't* accept 
callable syntax. I agree that tuple-for-parameters combined with making 
`builtins.callable` subscriptable would address a chunk of the concerns.



I can look into getting more stats on async callables, I don't think they are 
terribly common in most projects, but in certain contexts like async webservers 
they can come up quite a lot.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BXX3IHTGXCBBL6XKSEGK366CFGFDNLHX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP