[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Stefano Borini]

On Mon, 20 Jul 2020 at 03:26, Jonathan Goble  wrote:
>> One use case that comes up in xarray and pandas is support for indicating 
>> indexing "modes". For example, when indexing with floating point numbers 
>> it's convenient to be able to opt-in to approximate indexing, e.g., 
>> something like:
>> array.loc[longitude, latitude, method='nearest', tolerance=0.001]
>
>
> I had to stare at this for a good 30 seconds before I realized that this 
> wasn't a function/method call. Except for the square brackets instead of 
> parentheses, it would be.
>
> Honestly, this whole idea smells to me like just wanting another type of 
> function call with different semantics.

I agree and in fact it was a very weak point. The main point, in my
opinion, would be axis naming. Whatever the term "axis" means. In the
case of an array, it can be obvious. In the case of the starting mail,
axis is basically the degree of freedom of customisation of a typing
class.



-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IGFJK7S5YGD22Z2SQV3OZXYIETM6VDBH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Stefano Borini]

On Fri, 17 Jul 2020 at 20:58, Christopher Barker  wrote:

> So what would the “key object” be I the proposed case:
>
>  d2[a=1, b=2]
>
> A namedtuple? Or namedtuple-like object?

This was already discussed in PEP-472 as the "namedtuple" strategy,
but among the various negative points were the relative lack of
maturity of the namedtuple and the _n magic field that we assigned to
it.
I kind of feel they are still relevant, but I might be wrong. The PEP
is 5 years old.


-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EK6LXJ453JCU3HUAMDNPZSWGBOAJFATV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

As I said I would, I've now published a package on PyPI to support our
exploration and development of this idea.

You'll find more information on this new thread:
Subject: Package kwkey and PEP 472 -- Support for indexing with keyword
arguments
URL:
https://mail.python.org/archives/list/python-ideas@python.org/thread/ZAELJR4TFJ327CATJP467F5D2C2UG33T/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SB3ONYMVABT4GSQH4C3GPMYD5CCDUYTO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Todd]

On Tue, Jul 21, 2020 at 2:05 PM David Mertz  wrote:

> On Tue, Jul 21, 2020, 12:14 PM Sebastian Berg
>
>> First, using it for named dimensions, means you don't actually need to
>> mix it with normal tuple indexing, mixing both seems rather confusing?
>>
>> temperature.loc(method="nearest")[longitude=longs, latitude=lats]
>>
>
> [*] In my opinion, passing the keyword argument using anything other than
> the standard **kws style sounds crazy. We have perfectly good ways to do
> that for every other function or method. Don't invent something new and
> different that only works with .__getitem__().
>

 It would also be possible to pass __getitem__ exactly one other argument,
a dict with all the named arguments.

So instead of __getitem__(inds, **kwargs) simply __getitem__(inds, kwargs)

"kwargs" would be the same in both cases, a dict with the key/value pairs.
In most cases a dict is what people are going to want anyway.  It avoids
having to unpack then repack the dict.  But it would make it slightly
harder for adding parameters to indexing, although I am skeptical of that
use-case to begin with.  It would also make it feasible to use labels that
aren't valid variable names, although it may not be desirable to support
that.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WUS2RRY73GOZVUC4BUP7WF6WAXERDDAS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Todd]

On Sat, Jul 18, 2020, 00:21 Ricky Teachey  wrote:

>
> This raises a question that needs to be answered, then: what would be the
> utility of mixing together positional and kwd arguments in this way?
>
> Even the xarray examples given so far don't seem to make use of this
> mixture. From my knowledge of pandas I am not sure what the meaning of
> this would be, either.
>

For xarray an index can have a position, a label, or both.  So being able
to mix positional and label-based indexing is important.  I just didn't
include that in the examples because I was focusing on the advantage of
label-based indexing.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T6K73JPTCUCRMBDRMAQUOMI5LGJSYJK6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

Relevant to Jonathan's suggestion for a "key-object", Marco has been 
performing some experiments on an immutable dict:

https://mail.python.org/archives/list/python-ideas@python.org/message/K7CRVW6O7RO6DT3JIG3OAJCAVCA5CNTN/


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HWX7WMO2F5JL25K2D47BF5OMCUPIV2YP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Stephan Hoyer]

On Tue, Jul 21, 2020 at 9:15 AM Sebastian Berg 
wrote:

> On Mon, 2020-07-20 at 22:27 -0700, Christopher Barker wrote:
> > On Mon, Jul 20, 2020 at 3:17 AM Rhodri James 
> > wrote:
> >
> > > Ironically that example pushes me back to -1.  It may look a lot
> > > like
> > > xarray and pandas working, but that just means it should be in
> > > xarray
> > > and/or pandas.
> >
> > after following most of this discussion, I'm still not sure what we'd
> > get
> > with keywords in indexing.
> >
> > But I do think it would be nice of we could use slice syntax in other
> > places. That would allow things like xarray and pandas to use slices
> > in
> > regular function calls. here's an example from the xarray docs:
> >
> >  da.isel(space=0, time=slice(None, 2))
> >
> > wouldn't that be nice as:
> >
> > da.isel(space=0, time=:2)
> >
> > or:
> >
> > da.sel(time=slice("2000-01-01", "2000-01-02"))
> >
> > could be:
> >
> > da.sel(time="2000-01-01":"2000-01-02")
> >
> > As far as I can tell, slicing syntax is currently a syntax error in
> > these
> > cases, and any others I thought to test.
> >
> > Is there a reason to not allow syntax for creating a slice object to
> > be
> > used anywhere (Or more places, anyway)?
> >
> > By the way, I just noticed this note in the xarray docs:
> >
> > """Note: We would love to be able to do indexing with labeled
> > dimension
> > names inside brackets, but unfortunately, Python does yet not support
> > indexing with keyword arguments like da[space=0]
> > """
>
> This would be the thing I would think of first when indexing with
> keywords.  But, there are a few points about named dimensions:
>
> First, using it for named dimensions, means you don't actually need to
> mix it with normal tuple indexing, mixing both seems rather confusing?
> (I cannot think of how to interpret it)
>
> Second,  using keyword arguments for indexing `mode=` or `method=`
> switches as Stephan Hoyer mentioned as well seems neat.  But I am
> worried that the two potential uses clash way too much and my gut
> feeling is to prefer the labeled use (which is why I would be extremely
> hesitant to add mode-switching things to NumPy or pandas).  I might
> rather prefer mode switching to be spelled as::
>
> temperature.loc(method="nearest")[longitude=longs, latitude=lats]
>
> even if that has to create an intermediate indexer object (twice, since
> `.loc` here is also an index helper object).
> (This means axis labels must be strings, that is likely no issue, but
> should maybe be mentioned.)
>
> Thus, for most containers, my knee jerk reaction would be to discourage
> the use of keywords in indexing for mode switching.  But some of the
> use-cases seemed more like class factories, for which there is no clash
> of these two concepts/applications.
>
> That said, labeled dimensions/axis do seem like nice syntax with quite
> a bit of potential to me, even with 3 dimensions, remembering whether
> your coordinate order was x,y,z or z,x,y or z,y,x can be annoying
> (especially if you mix in a 1-D dataset with only a z axis).
>

For what it's worth, I (as the original author of xarray) totally agree
with both Sebastian and Christopher. For indexing labeled arrays, the most
compelling use-case is cleaner syntax for creating slice() objects along
with keyword arguments for dimension names.

I don't particularly care whether that's spelled with [] or (), e.g.,
da.sel(time="2000-01-01":"2000-01-02")
or
da.loc[time="2000-01-01":"2000-01-02"]
neither of which is currently valid syntax.

The further advantages of supporting keyword arguments in
__getitem__/__setitem__ would be:
1. We wouldn't need separate methods for positional vs keyword argument
indexing. Currently, xarray has both .loc[] and .sel().
2. We could support matching syntax with keyword arguments in assignment.
This is mostly relevant for inexperienced Python users, who will try
something like "da.sel(x=0) = value" and encounter a SyntaxError. (This
does come up with some regularity, because xarray's target audience
includes scientists who often aren't experienced programmers.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/P3OYHPIML7EO7JTE4T4MQ6QF36F4VA36/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Tue, Jul 21, 2020, 12:14 PM Sebastian Berg

> First, using it for named dimensions, means you don't actually need to
> mix it with normal tuple indexing, mixing both seems rather confusing?
>
> temperature.loc(method="nearest")[longitude=longs, latitude=lats]
>

I probably don't disagree on the API for xarray or similar library. But
that choice would be up to the library, not Python developers. Whatever the
way keyword argument are passed to .__getitem__()[*], they are available
for the class to do whatever it likes with them.

[*] In my opinion, passing the keyword argument using anything other than
the standard **kws style sounds crazy. We have perfectly good ways to do
that for every other function or method. Don't invent something new and
different that only works with .__getitem__().
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SA5FXO3R5LC52EKKABFH7PBXZEKP53CD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Sebastian Berg]

On Mon, 2020-07-20 at 22:27 -0700, Christopher Barker wrote:
> On Mon, Jul 20, 2020 at 3:17 AM Rhodri James 
> wrote:
> 
> > Ironically that example pushes me back to -1.  It may look a lot
> > like
> > xarray and pandas working, but that just means it should be in
> > xarray
> > and/or pandas.
> 
> after following most of this discussion, I'm still not sure what we'd
> get
> with keywords in indexing.
> 
> But I do think it would be nice of we could use slice syntax in other
> places. That would allow things like xarray and pandas to use slices
> in
> regular function calls. here's an example from the xarray docs:
> 
>  da.isel(space=0, time=slice(None, 2))
> 
> wouldn't that be nice as:
> 
> da.isel(space=0, time=:2)
> 
> or:
> 
> da.sel(time=slice("2000-01-01", "2000-01-02"))
> 
> could be:
> 
> da.sel(time="2000-01-01":"2000-01-02")
> 
> As far as I can tell, slicing syntax is currently a syntax error in
> these
> cases, and any others I thought to test.
> 
> Is there a reason to not allow syntax for creating a slice object to
> be
> used anywhere (Or more places, anyway)?
> 
> By the way, I just noticed this note in the xarray docs:
> 
> """Note: We would love to be able to do indexing with labeled
> dimension
> names inside brackets, but unfortunately, Python does yet not support
> indexing with keyword arguments like da[space=0]
> """

This would be the thing I would think of first when indexing with
keywords.  But, there are a few points about named dimensions:

First, using it for named dimensions, means you don't actually need to
mix it with normal tuple indexing, mixing both seems rather confusing?
(I cannot think of how to interpret it)

Second,  using keyword arguments for indexing `mode=` or `method=`
switches as Stephan Hoyer mentioned as well seems neat.  But I am
worried that the two potential uses clash way too much and my gut
feeling is to prefer the labeled use (which is why I would be extremely
hesitant to add mode-switching things to NumPy or pandas).  I might
rather prefer mode switching to be spelled as::

temperature.loc(method="nearest")[longitude=longs, latitude=lats]

even if that has to create an intermediate indexer object (twice, since
`.loc` here is also an index helper object).
(This means axis labels must be strings, that is likely no issue, but
should maybe be mentioned.)

Thus, for most containers, my knee jerk reaction would be to discourage
the use of keywords in indexing for mode switching.  But some of the
use-cases seemed more like class factories, for which there is no clash
of these two concepts/applications.

That said, labeled dimensions/axis do seem like nice syntax with quite
a bit of potential to me, even with 3 dimensions, remembering whether
your coordinate order was x,y,z or z,x,y or z,y,x can be annoying
(especially if you mix in a 1-D dataset with only a z axis).

Cheers,

Sebastian


> So they would like it :-)
> 
> -CHB
> 
> 
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/VSQO7A6K2SA7KI25U25CHROPEHCZFEG2/
> Code of Conduct: http://python.org/psf/codeofconduct/



signature.asc
Description: This is a digitally signed message part
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QNEFKJPSAIXE2D467MJCYKSHIR7H6TQ4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Guido van Rossum]

You have to find a core dev who is willing to act as a Sponsor. I recommend
asking Steven d’Aprano (but I do not know if he’s interested). Until then,
hash out the precise spec for the idea here. Coming up with a solid
motivation is also important.

On Tue, Jul 21, 2020 at 01:15 Stefano Borini 
wrote:

> I am unsure of the process if there's interest. Should I revise the
> PEP and create a new one?
>
> On Tue, 21 Jul 2020 at 06:29, Christopher Barker 
> wrote:
> >
> > On Mon, Jul 20, 2020 at 3:17 AM Rhodri James 
> wrote:
> >>
> >> Ironically that example pushes me back to -1.  It may look a lot like
> >> xarray and pandas working, but that just means it should be in xarray
> >> and/or pandas.
> >
> >
> > after following most of this discussion, I'm still not sure what we'd
> get with keywords in indexing.
> >
> > But I do think it would be nice of we could use slice syntax in other
> places. That would allow things like xarray and pandas to use slices in
> regular function calls. here's an example from the xarray docs:
> >
> >  da.isel(space=0, time=slice(None, 2))
> >
> > wouldn't that be nice as:
> >
> > da.isel(space=0, time=:2)
> >
> > or:
> >
> > da.sel(time=slice("2000-01-01", "2000-01-02"))
> >
> > could be:
> >
> > da.sel(time="2000-01-01":"2000-01-02")
> >
> > As far as I can tell, slicing syntax is currently a syntax error in
> these cases, and any others I thought to test.
> >
> > Is there a reason to not allow syntax for creating a slice object to be
> used anywhere (Or more places, anyway)?
> >
> > By the way, I just noticed this note in the xarray docs:
> >
> > """Note: We would love to be able to do indexing with labeled dimension
> names inside brackets, but unfortunately, Python does yet not support
> indexing with keyword arguments like da[space=0]
> > """
> > So they would like it :-)
> >
> > -CHB
> >
> >
> >
> > --
> > Christopher Barker, PhD
> >
> > Python Language Consulting
> >   - Teaching
> >   - Scientific Software Development
> >   - Desktop GUI and Web Development
> >   - wxPython, numpy, scipy, Cython
> > ___
> > Python-ideas mailing list -- python-ideas@python.org
> > To unsubscribe send an email to python-ideas-le...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/VSQO7A6K2SA7KI25U25CHROPEHCZFEG2/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Kind regards,
>
> Stefano Borini
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/FVBXHEKTOTRXBBQ2AVUHTF72YFD4HCCH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/URJK4PLF7XCMRD7PGRKP7BSVGQAVSEZ2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

On Tue, Jul 21, 2020, 5:48 AM Gerrit Holl  wrote:

> On Sat, 18 Jul 2020 at 18:31, MRAB  wrote:
> > [snip]
> > I haven't followed this thread for a while, but, to me, it seems that
> > the simplest option would be to pass the keyword arguments as a dict:
> >
> >  obj[a, b:c, x=1] does obj.__getitem__((a, slice(b, c)), dict(x=1))
> >
> > If there are no keyword arguments, then there's no dict.
>
> Could the entire argument be turned into a namedtuple?
>

The original rejected PEP had some good reasons for not going with a
namedtuple. I haven't read it in a while but I suggest considering them.
The biggest one being, a new class would have to be created and
instantiated for every indexing operation.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/K25KTQ73HFTBCWJF6WOB7OTWCEOVP2UE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Gerrit Holl]

On Sat, 18 Jul 2020 at 18:31, MRAB  wrote:
> [snip]
> I haven't followed this thread for a while, but, to me, it seems that
> the simplest option would be to pass the keyword arguments as a dict:
>
>  obj[a, b:c, x=1] does obj.__getitem__((a, slice(b, c)), dict(x=1))
>
> If there are no keyword arguments, then there's no dict.

Could the entire argument be turned into a namedtuple?

obj[a, b:c] does obj.__getitem__((a, slice(b, c)) or
obj.__getitem__(namedtuple(_0=a, _1=slice(b, c)))

obj[a, b:c, d=e:f] does obj.__getitem__(namedtuple(_0=a, _1=slice(b,
c), d=slice(e:f)))

it would seem a namedtuple is a more natural extension of the current
tuple, fully backward compatible (for sane code), while allowing for
keywords that are valid identifiers (which should be an acceptable
limitation).  The other restriction would be that the keyword-indexing
cannot use all-numeric identifiers prepended by an underscore.

Gerrit.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZZDIVLUFIM2B45BRVQAZ5EMFKJQ6IJTL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Gerrit Holl]

On Mon, 20 Jul 2020 at 04:27, Jonathan Goble  wrote:
>> One use case that comes up in xarray and pandas is support for indicating 
>> indexing "modes". For example, when indexing with floating point numbers 
>> it's convenient to be able to opt-in to approximate indexing, e.g., 
>> something like:
>> array.loc[longitude, latitude, method='nearest', tolerance=0.001]
>
> I had to stare at this for a good 30 seconds before I realized that this 
> wasn't a function/method call. Except for the square brackets instead of 
> parentheses, it would be.
>
> Honestly, this whole idea smells to me like just wanting another type of 
> function call with different semantics.
>
> IMHO the above example would be better spelled as:
>
> array.loc.get(longitude, latitude, method='nearest', tolerance=0.001)
>
> Pros: Much more obvious, perfectly legal today, zero backward compatibility 
> issues, probably the way many libraries with such functionality are doing it 
> now.
> Cons: A few extra characters (depending on the method name; here it's only 
> four) and a couple of taps on the Shift key (but you're already used to that).

Cons

- cannot assign to method call
- cannot use slicing syntax

With PEP:

array[lon=0:10, lat=0:10, method="nearest", tolerance=0.001] = 42

Without PEP (syntactically valid, although not valid xarray API):

array[K(lon=slice(0, 10), lat=slice(0, 10), method="nearest",
tolerance=0.001)] = 42

Gerrit.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JF5SSNHIETRJKBM2ADEOCFGTJVD3XMWW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Stefano Borini]

I am unsure of the process if there's interest. Should I revise the
PEP and create a new one?

On Tue, 21 Jul 2020 at 06:29, Christopher Barker  wrote:
>
> On Mon, Jul 20, 2020 at 3:17 AM Rhodri James  wrote:
>>
>> Ironically that example pushes me back to -1.  It may look a lot like
>> xarray and pandas working, but that just means it should be in xarray
>> and/or pandas.
>
>
> after following most of this discussion, I'm still not sure what we'd get 
> with keywords in indexing.
>
> But I do think it would be nice of we could use slice syntax in other places. 
> That would allow things like xarray and pandas to use slices in regular 
> function calls. here's an example from the xarray docs:
>
>  da.isel(space=0, time=slice(None, 2))
>
> wouldn't that be nice as:
>
> da.isel(space=0, time=:2)
>
> or:
>
> da.sel(time=slice("2000-01-01", "2000-01-02"))
>
> could be:
>
> da.sel(time="2000-01-01":"2000-01-02")
>
> As far as I can tell, slicing syntax is currently a syntax error in these 
> cases, and any others I thought to test.
>
> Is there a reason to not allow syntax for creating a slice object to be used 
> anywhere (Or more places, anyway)?
>
> By the way, I just noticed this note in the xarray docs:
>
> """Note: We would love to be able to do indexing with labeled dimension names 
> inside brackets, but unfortunately, Python does yet not support indexing with 
> keyword arguments like da[space=0]
> """
> So they would like it :-)
>
> -CHB
>
>
>
> --
> Christopher Barker, PhD
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/VSQO7A6K2SA7KI25U25CHROPEHCZFEG2/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FVBXHEKTOTRXBBQ2AVUHTF72YFD4HCCH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Christopher Barker]

On Mon, Jul 20, 2020 at 3:17 AM Rhodri James  wrote:

> Ironically that example pushes me back to -1.  It may look a lot like
> xarray and pandas working, but that just means it should be in xarray
> and/or pandas.


after following most of this discussion, I'm still not sure what we'd get
with keywords in indexing.

But I do think it would be nice of we could use slice syntax in other
places. That would allow things like xarray and pandas to use slices in
regular function calls. here's an example from the xarray docs:

 da.isel(space=0, time=slice(None, 2))

wouldn't that be nice as:

da.isel(space=0, time=:2)

or:

da.sel(time=slice("2000-01-01", "2000-01-02"))

could be:

da.sel(time="2000-01-01":"2000-01-02")

As far as I can tell, slicing syntax is currently a syntax error in these
cases, and any others I thought to test.

Is there a reason to not allow syntax for creating a slice object to be
used anywhere (Or more places, anyway)?

By the way, I just noticed this note in the xarray docs:

"""Note: We would love to be able to do indexing with labeled dimension
names inside brackets, but unfortunately, Python does yet not support
indexing with keyword arguments like da[space=0]
"""
So they would like it :-)

-CHB



-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VSQO7A6K2SA7KI25U25CHROPEHCZFEG2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Rhodri James]

On 20/07/2020 06:11, Ricky Teachey wrote:

On Sun, Jul 19, 2020 at 9:53 PM Stephan Hoyer  wrote:

One use case that comes up in xarray and pandas is support for indicating
indexing "modes". For example, when indexing with floating point numbers
it's convenient to be able to opt-in to approximate indexing, e.g.,
something like:
array.loc[longitude, latitude, method='nearest', tolerance=0.001]



Thanks to those who pointed out that using kwargs to specify modes of key
management is an obvious use case.


Ironically that example pushes me back to -1.  It may look a lot like 
xarray and pandas working, but that just means it should be in xarray 
and/or pandas.  It doesn't feel at all right for regular Python to be 
muddling indices and modal parameters to something that isn't a function 
call like that.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q6ZF746AJLZURKSLUGTIXWEQLUDMEQ2D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

On Sun, Jul 19, 2020 at 9:53 PM Stephan Hoyer  wrote:

> On Fri, Jul 17, 2020 at 9:22 PM Ricky Teachey  wrote:
>
>> >>> # The positional arguments aren't part of the KeyObject
>> >>> d[a, b:c, d, e=5, f=6] == d.__getitem__((a, b:c, d), KeyObject(e=5,
>> f=6))
>>
>> This raises a question that needs to be answered, then: what would be the
>> utility of mixing together positional and kwd arguments in this way?
>>
>> Even the xarray examples given so far don't seem to make use of this
>> mixture. From my knowledge of pandas I am not sure what the meaning of
>> this would be, either.
>>
>
> One use case that comes up in xarray and pandas is support for indicating
> indexing "modes". For example, when indexing with floating point numbers
> it's convenient to be able to opt-in to approximate indexing, e.g.,
> something like:
> array.loc[longitude, latitude, method='nearest', tolerance=0.001]
>

Thanks to those who pointed out that using kwargs to specify modes of key
management is an obvious use case.

Additionally I agree it is extremely likely that most use cases will
involve specific kwd args, as Steven argues:

On Sat, Jul 18, 2020 at 1:49 PM Steven D'Aprano  wrote:

> ...
>
> Most use-cases I can think of will have to unpack the dict into named
> parameters. (Just as the interpreter does for us, in function calls.)
>
> When I'm writing functions, for every one use of `**kwargs`, I have
> about a hundred uses of named parameters. I'm pretty sure most people
> are similar. I don't think that keyword args in subscripts will be
> different. I'm pretty sure that nearly everyone will want to unpack the
> `**kwargs` into named parameters nearly all of the time.
>
> So why force them to do the unpacking themselves when the interpreter
> already has all the machinery to do it?


And Guido and Christopher Barker agree:

On Sun, Jul 19, 2020 at 12:46 AM Guido van Rossum  wrote:

> On Sat, Jul 18, 2020 at 9:11 PM Christopher Barker 
> wrote:
>
>>
>>
>> On Sat, Jul 18, 2020 at 1:43 PM Guido van Rossum 
>> wrote:
>>
>>> Yes please.
>>>
>>
>> Yes to what, exactly?
>>
>> -CHB
>>
>>
> Not to this...
>
> I don't think that is either simpler or more useful than a straight-
 forward binding of arguments to parameters, just as function calls
 already do:

 obj[a, b:c, x=1] ==> obj.__getitem__((a, slice(b, c)), x=1)

>>>
> But to this.
>
> It's unfortunate that positional subscripts are bundled together into a
 tuple. I have resisted calling that design a "mistake" because I don't
 know the reason for the design. There was probably a good reason for
 it,
 back in the ancient history of Python when getslice and getitem were
 unified. But I am sure that it will be a horrible mistake to emulate
 that decision for keyword arguments.

 If anyone wants or needs their keyword arguments to be bundled into a
 single kwargs parameter, you can have it. All you need do is declare
 your method with a `**kwargs` parameter, and the interpreter will do
 the
 rest.

>>>
> These two paragraphs make it clear what Steven was proposing. I am
> supporting him in this.
>

So it's a very good point and I support the idea.

Steven replied to my concern about the inconsistent mental model:

On Sat, Jul 18, 2020 at 5:09 AM Steven D'Aprano  wrote:

> On Sat, Jul 18, 2020 at 12:18:38AM -0400, Ricky Teachey wrote:
> > This strikes me as problematic for having a consistent mental model of
> how
> > stuff works in python. I think that for many the difference in the
> meaning
> > of the syntax between item-getting/setting and function-calling would
> be...
> > glaring.
>
> Yes, but what are we going to do about it?
>
> ...
>
> We have a few choices:
>
> ...
>
> (3) Reinforce that inconsistency, and continue to obfuscate the
> similarities, by handling keyword arguments in the same fashion as
> comma-separated subscripts. This will require a new builtin "key-object"
> class, and it will require every class that cares about keyword
> arguments in their subscripts to parse them themselves.
>
> We'll also need to decide how to combine subscripts and keywords:
>
> obj[a, b:c, x=1]
> # is this a tuple argument (a, slice(b, c), key(x=1))
> # or key argument key(a, slice(b, c), x=1)
>

So it's worth reiterating that parsing of the keyword arguments can be
accomplished relatively easily using a decorator that groups the arguments
into some sort of key object, as Jonathan Fine is suggesting.

On Sat, Jul 18, 2020 at 7:50 AM Jonathan Fine  wrote:

> ...
>
> In some other situations we wish for
> >>> d[1, 2, x=3, y=4] = 5
> to result in a call such as
>  __setitem__(d, (1, 2), 5, x=3, y=4)
> where __setitem__ is a function defined in the implementation of D =
> type(d).
>
> I fully support this goal, although not the implementation details in the
> example above. It is my opinion that this goal is best achieved by making
> easier TRANSPARENT use of
> k = K(1, 2, x=3, y=4

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Sun, Jul 19, 2020 at 10:27 PM Jonathan Goble  wrote:

> One use case that comes up in xarray and pandas is support for indicating
>> indexing "modes". For example, when indexing with floating point numbers
>> it's convenient to be able to opt-in to approximate indexing, e.g.,
>> something like:
>> array.loc[longitude, latitude, method='nearest', tolerance=0.001]
>>
>
> I had to stare at this for a good 30 seconds before I realized that this
> wasn't a function/method call. Except for the square brackets instead of
> parentheses, it would be.
> Honestly, this whole idea smells to me like just wanting another type of
> function call with different semantics.
> IMHO the above example would be better spelled as:
> array.loc.get(longitude, latitude, method='nearest', tolerance=0.001)
>

The problem is that with Pandas, Xarray, and other data frame/data array
libraries, using slices is typical.  Continuing with the example:

arr.loc[45:46, 69:70, altitude=300:400, tolerance=0.0001, projection="
WGS1984"]

There's no reason to tack a function onto the .loc accessor, it could just
be a method on the array/data frame itself.  So there are no extra
characters.  But doing this with parentheses would need a different new
feature of allowing slices directly as arguments.

That said, pandas.IndexSlice and numpy.s_ both provide accessors to allow
passing slices more easily.  So this is possible now:

arr.get(I[45:46], I[69:70], altitude=I[300:400], tolerance=0.0001,
projection="WGS1984")

... I mean, assuming someone writes an appropriate .get() method for their
favorite data array/frame library.  But Python itself has everything needed.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W6UH7CWZK5BELTR6JB2B5XHZLAPS4N74/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

On Sun, Jul 19, 2020 at 10:25 PM Jonathan Goble  wrote:

> On Sun, Jul 19, 2020 at 9:53 PM Stephan Hoyer  wrote:
>
>> On Fri, Jul 17, 2020 at 9:22 PM Ricky Teachey  wrote:
>>
>>> >>> # The positional arguments aren't part of the KeyObject
>>> >>> d[a, b:c, d, e=5, f=6] == d.__getitem__((a, b:c, d), KeyObject(e=5,
>>> f=6))
>>>
>>> This raises a question that needs to be answered, then: what would be
>>> the utility of mixing together positional and kwd arguments in this way?
>>>
>>> Even the xarray examples given so far don't seem to make use of this
>>> mixture. From my knowledge of pandas I am not sure what the meaning of
>>> this would be, either.
>>>
>>
>> One use case that comes up in xarray and pandas is support for indicating
>> indexing "modes". For example, when indexing with floating point numbers
>> it's convenient to be able to opt-in to approximate indexing, e.g.,
>> something like:
>> array.loc[longitude, latitude, method='nearest', tolerance=0.001]
>>
>
> I had to stare at this for a good 30 seconds before I realized that this
> wasn't a function/method call. Except for the square brackets instead of
> parentheses, it would be.
>
> Honestly, this whole idea smells to me like just wanting another type of
> function call with different semantics.
>
> IMHO the above example would be better spelled as:
>
> array.loc.get(longitude, latitude, method='nearest', tolerance=0.001)
>
> Pros: Much more obvious, perfectly legal today, zero backward
> compatibility issues, probably the way many libraries with such
> functionality are doing it now.
> Cons: A few extra characters (depending on the method name; here it's only
> four) and a couple of taps on the Shift key (but you're already used to
> that).
>
> Whereas for the indexing example:
>
> Pros: Explicit about indexing a collection and returning an item (but a
> good method name like "get" also has this quality).
> Cons: Not immediately obvious to an uninitiated reader of the code, new
> syntax, code using it is not compatible with Python 3.9 or earlier.
>
> As for setting a value, the indexing approach has a bit more value (since
> a function call can't be an lvalue), but you can just change the method
> name from "get" to "set" and add an extra parameter to be the value to set,
> and you're done. The pros and cons still apply.
>

> TL;DR: -1 all the way. Just use ordinary methods.
>

Another Con for the function call approach, you don't get to use the nice
slicing syntax. Compare:

>>> array.loc.get(slice(None), slice(None), method='nearest',
tolerance=0.001)
>>> array.loc.get[:, :, method='nearest', tolerance=0.001]

You also can't delete using a function call:

>>> del array.loc.get[:, :, method='nearest', tolerance=0.001]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EVP5ZCBMS3ZYORMCI7LHSFX3CEMZTBKW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Sun, Jul 19, 2020 at 10:24:42PM -0400, Jonathan Goble wrote:

> IMHO the above example would be better spelled as:
> 
> array.loc.get(longitude, latitude, method='nearest', tolerance=0.001)

One of the use-cases for this is for type hints, where you cannot use 
method calls. So that's right out.

Minor issues with "use a method call" include that it makes using slices 
difficult, as slice syntax is not legal in a method call, and that item 
assignment and deletion are awkward (you have to use a setter and 
deleter method, like some sort of ~~caveman~~ Java developer, instead of 
writing what you mean).



-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZODIBT7NCVXXVJP4P4Y3WSORERU7W3C2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Goble]

On Sun, Jul 19, 2020 at 9:53 PM Stephan Hoyer  wrote:

> On Fri, Jul 17, 2020 at 9:22 PM Ricky Teachey  wrote:
>
>> >>> # The positional arguments aren't part of the KeyObject
>> >>> d[a, b:c, d, e=5, f=6] == d.__getitem__((a, b:c, d), KeyObject(e=5,
>> f=6))
>>
>> This raises a question that needs to be answered, then: what would be the
>> utility of mixing together positional and kwd arguments in this way?
>>
>> Even the xarray examples given so far don't seem to make use of this
>> mixture. From my knowledge of pandas I am not sure what the meaning of
>> this would be, either.
>>
>
> One use case that comes up in xarray and pandas is support for indicating
> indexing "modes". For example, when indexing with floating point numbers
> it's convenient to be able to opt-in to approximate indexing, e.g.,
> something like:
> array.loc[longitude, latitude, method='nearest', tolerance=0.001]
>

I had to stare at this for a good 30 seconds before I realized that this
wasn't a function/method call. Except for the square brackets instead of
parentheses, it would be.

Honestly, this whole idea smells to me like just wanting another type of
function call with different semantics.

IMHO the above example would be better spelled as:

array.loc.get(longitude, latitude, method='nearest', tolerance=0.001)

Pros: Much more obvious, perfectly legal today, zero backward compatibility
issues, probably the way many libraries with such functionality are doing
it now.
Cons: A few extra characters (depending on the method name; here it's only
four) and a couple of taps on the Shift key (but you're already used to
that).

Whereas for the indexing example:

Pros: Explicit about indexing a collection and returning an item (but a
good method name like "get" also has this quality).
Cons: Not immediately obvious to an uninitiated reader of the code, new
syntax, code using it is not compatible with Python 3.9 or earlier.

As for setting a value, the indexing approach has a bit more value (since a
function call can't be an lvalue), but you can just change the method name
from "get" to "set" and add an extra parameter to be the value to set, and
you're done. The pros and cons still apply.

TL;DR: -1 all the way. Just use ordinary methods.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IGFUWPYAXKNMIB3RV3XPCJSSMAEH6OHO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Stephan Hoyer]

On Fri, Jul 17, 2020 at 9:22 PM Ricky Teachey  wrote:

> >>> # The positional arguments aren't part of the KeyObject
> >>> d[a, b:c, d, e=5, f=6] == d.__getitem__((a, b:c, d), KeyObject(e=5,
> f=6))
>
> This raises a question that needs to be answered, then: what would be the
> utility of mixing together positional and kwd arguments in this way?
>
> Even the xarray examples given so far don't seem to make use of this
> mixture. From my knowledge of pandas I am not sure what the meaning of
> this would be, either.
>

One use case that comes up in xarray and pandas is support for indicating
indexing "modes". For example, when indexing with floating point numbers
it's convenient to be able to opt-in to approximate indexing, e.g.,
something like:
array.loc[longitude, latitude, method='nearest', tolerance=0.001]

(This use case is actually already mentioned in PEP 472, as "an optional
contextual to the indexing":
https://www.python.org/dev/peps/pep-0472/#use-cases)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XC6XYIUHDYFR43CEV3QR67JK7UALYACL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Eric V. Smith]

On 7/19/2020 6:49 PM, David Mertz wrote:
On Sun, Jul 19, 2020 at 6:35 PM Dominik Vilsmeier 
mailto:dominik.vilsme...@gmx.de>> wrote:


But this looks unnecessarily complicated. Why can't xarray allow
the following:

    ds["empty"]["lon", 1:5, "lat", 3:] = 10

which looks very close to the proposed syntax below. Not that I'm
against the proposal but I think that any use case involving
*only* keyword arguments isn't a very strong one, because it can
easily be solved that way without a change to existing syntax.

Xarray already allows positional slices in multiple dimensions.  The 
existing dict weirdness is to have a way to introduce the named 
dimensions.  But suppose that the array in question had a first listed 
'altitude'.  I think nowadays, we can write:


    arr.loc[50:60, 1:5, 3:]

If we only reference dimensions by number not by name. Under the 
"commas separate keys from values" this would be difficult:


    arr.loc[50:60, "lon", 1:5, "lat", 3:]

Yes, I can imagine a rule like "If it is a slice that wasn't preceded 
by a string, treat it as positional, otherwise if it is a string treat 
it as a key, but treat the next thing after a string as the slice 
value corresponding to that key."


That seems more error prone and harder to grok than the potential:

    arr.loc[50:60, lon=1:5, lat=3:]

Where you'd still just have to know that "axis 0 is altitude" of your 
particular array.


In addition, what if you actually wanted:

arr.loc["lon", "lon", 1:5, "lat", 3:]

That is: what if you have a string argument whose value happens to be 
name of one of your "named" parameters?


Eric

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XCB5JTANORJQEJTBJNCJAAWSZHL5UU5X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Sun, Jul 19, 2020 at 6:35 PM Dominik Vilsmeier 
wrote:

> But this looks unnecessarily complicated. Why can't xarray allow the
> following:
>
> ds["empty"]["lon", 1:5, "lat", 3:] = 10
>
> which looks very close to the proposed syntax below. Not that I'm against
> the proposal but I think that any use case involving *only* keyword
> arguments isn't a very strong one, because it can easily be solved that way
> without a change to existing syntax.
>
Xarray already allows positional slices in multiple dimensions.  The
existing dict weirdness is to have a way to introduce the named
dimensions.  But suppose that the array in question had a first listed
'altitude'.  I think nowadays, we can write:

arr.loc[50:60, 1:5, 3:]

If we only reference dimensions by number not by name.  Under the "commas
separate keys from values" this would be difficult:

arr.loc[50:60, "lon", 1:5, "lat", 3:]

Yes, I can imagine a rule like "If it is a slice that wasn't preceded by a
string, treat it as positional, otherwise if it is a string treat it as a
key, but treat the next thing after a string as the slice value
corresponding to that key."

That seems more error prone and harder to grok than the potential:

arr.loc[50:60, lon=1:5, lat=3:]

Where you'd still just have to know that "axis 0 is altitude" of your
particular array.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XNJL45PE6DFYTMB6SUYESPG5HNY62A4I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Dominik Vilsmeier]

On 17.07.20 22:11, Todd wrote:


On Fri, Jul 17, 2020 at 12:19 PM David Mertz mailto:me...@gnosis.cx>> wrote:

Fwiw, I'm probably -0 on the feature itself. Someone suggested it
could be useful for xarray, but I'm not sure now what that would
look like. If someone had an example, I could easily be moved.


Here is what it currently looks like to assign values to indices in
xarray (adapted from a tutorial):

ds["empty"].loc[dict(lon=5, lat=6)] = 10

This could be changed to:

ds["empty"][lon=5, lat=6] = 10

This becomes even a bigger advantage if we include slicing, which I
think we should:

ds["empty"].loc[dict(lon=slice(1, 5), lat=slice(3, None))] = 10


But this looks unnecessarily complicated. Why can't xarray allow the
following:

    ds["empty"]["lon", 1:5, "lat", 3:] = 10

which looks very close to the proposed syntax below. Not that I'm
against the proposal but I think that any use case involving *only*
keyword arguments isn't a very strong one, because it can easily be
solved that way without a change to existing syntax. Only when
positional and keyword arguments are mixed, it becomes difficult to
distinguish (for both the reader and the method).



to

ds["empty"][lon=1:5, lat=6:] = 10

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H33KPR66M7DQJAGBNMH7HCJEIM7JKZ7V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Guido van Rossum]

On Sat, Jul 18, 2020 at 9:11 PM Christopher Barker 
wrote:

>
>
> On Sat, Jul 18, 2020 at 1:43 PM Guido van Rossum  wrote:
>
>> Yes please.
>>
>
> Yes to what, exactly?
>
> -CHB
>
> FWIW, IIRC the “bundle values in a single parameter” predates the demise
>> of __getslice__. It probably was meant for dict keys primarily (no surprise
>> there). The bundling would have been easier for the C API — __getitem__ is
>> as old as Python there.
>>
>> On Sat, Jul 18, 2020 at 10:49 Steven D'Aprano 
>> wrote:
>>
>>> On Sat, Jul 18, 2020 at 05:30:40PM +0100, MRAB wrote:
>>>
>>> > I haven't followed this thread for a while, but, to me, it seems that
>>> > the simplest option would be to pass the keyword arguments as a dict:
>>>
>>> What are you going to do with that keyword argument dict?
>>>
>>> Most use-cases I can think of will have to unpack the dict into named
>>> parameters. (Just as the interpreter does for us, in function calls.)
>>>
>>> When I'm writing functions, for every one use of `**kwargs`, I have
>>> about a hundred uses of named parameters. I'm pretty sure most people
>>> are similar. I don't think that keyword args in subscripts will be
>>> different. I'm pretty sure that nearly everyone will want to unpack the
>>> `**kwargs` into named parameters nearly all of the time.
>>>
>>> So why force them to do the unpacking themselves when the interpreter
>>> already has all the machinery to do it?
>>>
>>> If you want kwargs to collect arbitrary keyword arguments, you can just
>>> declare your getitem method (and setitem and delitem if needed) to take
>>> `**kwargs`, and the interpreter will oblige.
>>>
>>> If you want no keyword arguments at all, you don't have to change a
>>> thing. Your getitem (etc) methods have no keyword parameters, so using
>>> keywords in the subscript will fail with TypeError.
>>>
>>>
>>> > obj[a, b:c, x=1] does obj.__getitem__((a, slice(b, c)), dict(x=1))
>>>
>>
Not to this...

I don't think that is either simpler or more useful than a straight-
>>> forward binding of arguments to parameters, just as function calls
>>> already do:
>>>
>>> obj[a, b:c, x=1] ==> obj.__getitem__((a, slice(b, c)), x=1)
>>>
>>
But to this.

It's unfortunate that positional subscripts are bundled together into a
>>> tuple. I have resisted calling that design a "mistake" because I don't
>>> know the reason for the design. There was probably a good reason for it,
>>> back in the ancient history of Python when getslice and getitem were
>>> unified. But I am sure that it will be a horrible mistake to emulate
>>> that decision for keyword arguments.
>>>
>>> If anyone wants or needs their keyword arguments to be bundled into a
>>> single kwargs parameter, you can have it. All you need do is declare
>>> your method with a `**kwargs` parameter, and the interpreter will do the
>>> rest.
>>>
>>
These two paragraphs make it clear what Steven was proposing. I am
supporting him in this.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q3THDB35R4AQIK4JSNYBM5XFSOPQQZ66/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Christopher Barker]

On Sat, Jul 18, 2020 at 1:43 PM Guido van Rossum  wrote:

> Yes please.
>

Yes to what, exactly?

-CHB

FWIW, IIRC the “bundle values in a single parameter” predates the demise of
> __getslice__. It probably was meant for dict keys primarily (no surprise
> there). The bundling would have been easier for the C API — __getitem__ is
> as old as Python there.
>
> On Sat, Jul 18, 2020 at 10:49 Steven D'Aprano  wrote:
>
>> On Sat, Jul 18, 2020 at 05:30:40PM +0100, MRAB wrote:
>>
>> > I haven't followed this thread for a while, but, to me, it seems that
>> > the simplest option would be to pass the keyword arguments as a dict:
>>
>> What are you going to do with that keyword argument dict?
>>
>> Most use-cases I can think of will have to unpack the dict into named
>> parameters. (Just as the interpreter does for us, in function calls.)
>>
>> When I'm writing functions, for every one use of `**kwargs`, I have
>> about a hundred uses of named parameters. I'm pretty sure most people
>> are similar. I don't think that keyword args in subscripts will be
>> different. I'm pretty sure that nearly everyone will want to unpack the
>> `**kwargs` into named parameters nearly all of the time.
>>
>> So why force them to do the unpacking themselves when the interpreter
>> already has all the machinery to do it?
>>
>> If you want kwargs to collect arbitrary keyword arguments, you can just
>> declare your getitem method (and setitem and delitem if needed) to take
>> `**kwargs`, and the interpreter will oblige.
>>
>> If you want no keyword arguments at all, you don't have to change a
>> thing. Your getitem (etc) methods have no keyword parameters, so using
>> keywords in the subscript will fail with TypeError.
>>
>>
>> > obj[a, b:c, x=1] does obj.__getitem__((a, slice(b, c)), dict(x=1))
>>
>> I don't think that is either simpler or more useful than a straight-
>> forward binding of arguments to parameters, just as function calls
>> already do:
>>
>> obj[a, b:c, x=1] ==> obj.__getitem__((a, slice(b, c)), x=1)
>>
>> It's unfortunate that positional subscripts are bundled together into a
>> tuple. I have resisted calling that design a "mistake" because I don't
>> know the reason for the design. There was probably a good reason for it,
>> back in the ancient history of Python when getslice and getitem were
>> unified. But I am sure that it will be a horrible mistake to emulate
>> that decision for keyword arguments.
>>
>> If anyone wants or needs their keyword arguments to be bundled into a
>> single kwargs parameter, you can have it. All you need do is declare
>> your method with a `**kwargs` parameter, and the interpreter will do the
>> rest.
>>
>>
>>
>> --
>> Steven
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/5MAWLGDJCVM62N7CKKY3C5DBYKVTWZHC/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> --
> --Guido (mobile)
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/NAWEMP4FPLZPP7X4A4IND7IQEMB62CFL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NKBXBORVUA6TF5FZ3P72DPSAPPAJLMAE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Guido van Rossum]

Yes please.

FWIW, IIRC the “bundle values in a single parameter” predates the demise of
__getslice__. It probably was meant for dict keys primarily (no surprise
there). The bundling would have been easier for the C API — __getitem__ is
as old as Python there.

On Sat, Jul 18, 2020 at 10:49 Steven D'Aprano  wrote:

> On Sat, Jul 18, 2020 at 05:30:40PM +0100, MRAB wrote:
>
> > I haven't followed this thread for a while, but, to me, it seems that
> > the simplest option would be to pass the keyword arguments as a dict:
>
> What are you going to do with that keyword argument dict?
>
> Most use-cases I can think of will have to unpack the dict into named
> parameters. (Just as the interpreter does for us, in function calls.)
>
> When I'm writing functions, for every one use of `**kwargs`, I have
> about a hundred uses of named parameters. I'm pretty sure most people
> are similar. I don't think that keyword args in subscripts will be
> different. I'm pretty sure that nearly everyone will want to unpack the
> `**kwargs` into named parameters nearly all of the time.
>
> So why force them to do the unpacking themselves when the interpreter
> already has all the machinery to do it?
>
> If you want kwargs to collect arbitrary keyword arguments, you can just
> declare your getitem method (and setitem and delitem if needed) to take
> `**kwargs`, and the interpreter will oblige.
>
> If you want no keyword arguments at all, you don't have to change a
> thing. Your getitem (etc) methods have no keyword parameters, so using
> keywords in the subscript will fail with TypeError.
>
>
> > obj[a, b:c, x=1] does obj.__getitem__((a, slice(b, c)), dict(x=1))
>
> I don't think that is either simpler or more useful than a straight-
> forward binding of arguments to parameters, just as function calls
> already do:
>
> obj[a, b:c, x=1] ==> obj.__getitem__((a, slice(b, c)), x=1)
>
> It's unfortunate that positional subscripts are bundled together into a
> tuple. I have resisted calling that design a "mistake" because I don't
> know the reason for the design. There was probably a good reason for it,
> back in the ancient history of Python when getslice and getitem were
> unified. But I am sure that it will be a horrible mistake to emulate
> that decision for keyword arguments.
>
> If anyone wants or needs their keyword arguments to be bundled into a
> single kwargs parameter, you can have it. All you need do is declare
> your method with a `**kwargs` parameter, and the interpreter will do the
> rest.
>
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/5MAWLGDJCVM62N7CKKY3C5DBYKVTWZHC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NAWEMP4FPLZPP7X4A4IND7IQEMB62CFL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Sat, Jul 18, 2020 at 05:30:40PM +0100, MRAB wrote:

> I haven't followed this thread for a while, but, to me, it seems that 
> the simplest option would be to pass the keyword arguments as a dict:

What are you going to do with that keyword argument dict?

Most use-cases I can think of will have to unpack the dict into named 
parameters. (Just as the interpreter does for us, in function calls.)

When I'm writing functions, for every one use of `**kwargs`, I have 
about a hundred uses of named parameters. I'm pretty sure most people 
are similar. I don't think that keyword args in subscripts will be 
different. I'm pretty sure that nearly everyone will want to unpack the 
`**kwargs` into named parameters nearly all of the time.

So why force them to do the unpacking themselves when the interpreter 
already has all the machinery to do it?

If you want kwargs to collect arbitrary keyword arguments, you can just 
declare your getitem method (and setitem and delitem if needed) to take 
`**kwargs`, and the interpreter will oblige.

If you want no keyword arguments at all, you don't have to change a 
thing. Your getitem (etc) methods have no keyword parameters, so using 
keywords in the subscript will fail with TypeError.


> obj[a, b:c, x=1] does obj.__getitem__((a, slice(b, c)), dict(x=1))

I don't think that is either simpler or more useful than a straight- 
forward binding of arguments to parameters, just as function calls 
already do:

obj[a, b:c, x=1] ==> obj.__getitem__((a, slice(b, c)), x=1)

It's unfortunate that positional subscripts are bundled together into a 
tuple. I have resisted calling that design a "mistake" because I don't 
know the reason for the design. There was probably a good reason for it, 
back in the ancient history of Python when getslice and getitem were 
unified. But I am sure that it will be a horrible mistake to emulate 
that decision for keyword arguments.

If anyone wants or needs their keyword arguments to be bundled into a 
single kwargs parameter, you can have it. All you need do is declare 
your method with a `**kwargs` parameter, and the interpreter will do the 
rest.



-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5MAWLGDJCVM62N7CKKY3C5DBYKVTWZHC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[MRAB]

On 2020-07-18 10:04, Steven D'Aprano wrote:

On Sat, Jul 18, 2020 at 12:18:38AM -0400, Ricky Teachey wrote:

On Fri, Jul 17, 2020 at 7:21 PM Steven D'Aprano  wrote:

> On Fri, Jul 17, 2020 at 11:11:17AM -0400, Ricky Teachey wrote:
>
> ...
>
> For backwards-compatibility, there will only ever be a single positional
> argument passed into the method. That's because comma-separated values
> in a subscript are already passed as a tuple:
>
> # this calls __getitem__ with a single tuple argument
> obj[a,b:c,d]  ==> (1, slice(2, 3), 4)
>
> So that's not going to change (at least not without a long and painful
> deprecation process). But adding support for keyword arguments requires
> no changes to any existing class or a new builtin "key object" type.
>

This strikes me as problematic for having a consistent mental model of how
stuff works in python. I think that for many the difference in the meaning
of the syntax between item-getting/setting and function-calling would be...
glaring.


Yes, but what are we going to do about it?

Break a million existing scripts, applications and libraries that rely
on `__getitem__` only receiving a single tuple argument when passed
comma-separated values? I don't think the core devs will accept that, I
think the numpy devs will object strongly, and I'm pretty sure that the
Steering Council will say no.

But if you disagree, then feel free to start writing a PEP.

The fact that multiple comma-separated subscripts are passed to the
method as a single tuple argument is a historical fact we (almost
certainly) cannot change now. But that is orthogonal to how we choose to
proceed with keyword arguments. We aren't obliged to repeat the same
design.

We have a few choices:

(1) There is a minor inconsistency between subscripts and function
calls, so let's just forget all about the whole idea. If we cannot agree
on a decision, this is the default. (Status quo wins a stalement.)

(2) Let the Perfect be the enemy of the Good. No compromises! Insist on
breaking the entire Python ecosystem for the sake of fixing this minor
inconsistency between subscripting and function calls.

(3) Reinforce that inconsistency, and continue to obfuscate the
similarities, by handling keyword arguments in the same fashion as
comma-separated subscripts. This will require a new builtin "key-object"
class, and it will require every class that cares about keyword
arguments in their subscripts to parse them themselves.

We'll also need to decide how to combine subscripts and keywords:

 obj[a, b:c, x=1]
 # is this a tuple argument (a, slice(b, c), key(x=1))
 # or key argument key(a, slice(b, c), x=1)

(4) Or keep the subscript processing as-is, for backwards-compatibility,
but pass keyword arguments as normal for functions.

Both (3) and (4) would get the job done, but (3) requires everyone who
needs keyword arguments to parse the tuple and/or key object by hand to
extract them. Having done something similiar in the past (emulating
keyword-only arguments in Python 2), I can tell you this is painful.

With (4), the interpreter automatically matches up passed keyword
arguments to my `__getitem__` parameters, filling in defaults if needed,
and I can concentrate on using the arguments, not parsing them.


[snip]
I haven't followed this thread for a while, but, to me, it seems that 
the simplest option would be to pass the keyword arguments as a dict:


obj[a, b:c, x=1] does obj.__getitem__((a, slice(b, c)), dict(x=1))

If there are no keyword arguments, then there's no dict.

This does mean that __getitem__ could be called with 1 or 2 arguments 
and __setitem__ could be called with 2 or 3 arguments, so it would be 
advisable to make the additional argument optional:


def __getitem__(self, args, kwargs=None):

Additionally, __setitem__ would look a little odd:

def __setitem__(self, args, value, kwargs=None):

It would raise a TypeError if there were keyword arguments, but 
__getitem__, etc, didn't accept any.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6C4GJS5RJQ3I3E76CRIA2QDG4BXMS4AD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Andras Tantos wrote
>
> Thanks for picking up on this thread, I almost given up hope that
> anyone would be interested. Then it suddenly blew up :)!

to thank all who expressed interest and support. (Andras means 'blew up' in
a good way!)

And I thank you, Andras. It's nice to be appreciated. I now have more
confidence that this discussion will result in changes to Python that will
help some users, and hinder none.

I'll now turn to next steps. First, something that I have a special
responsibility for. Then some technical remarks, on opaque and transparent
use of keys. Finally, some suggestions for others.

MY SPECIAL RESPONSIBILITY
=
On 16 July I wrote:

> I'll now state some goals.
>


> 1. Define 'o' and Protocol so that NOW gives the semantics you wish for.
> 2. Extend Python so that FUTURE give the semantics you wish for.
> 3. And the NOW syntax continues to work as expected (without changing 'o'
> and Protocol).
> 4. And all current use of container[key] continues to work as before.
>


> I believe that it is possible to achieve these goals. My previous posts to
> this discussion outline some of the key ideas. My next step, when I have
> time, is to implement and publish general purpose code for the NOW part of
> this list of goals.


I hope to have this done by the end of this month.  It won't be my
immediate priority. I'm attending the online TeX Users Group conference
next week, and haven't written my talks yet.
http://tug.org/tug2020/

TECHNICAL REMARKS
===
Here are some remarks related to this goal which, I hope, will help you. I
hope it helps both in your discussions, and in any responsibilities you
choose to take on. My remarks concern the coexistence of what I call the
opaque and transparent points of view, regarding the syntax
>>> d[1, 2, x=3, y=4]

A dict accepts any hashable object as a key.
>>> class A: __hash__ = lambda self: 0
>>> {A(): 1}
{<__main__.A object at 0x7f221087fa90>: 1}

My proposal is that
>>> d[1, 2, x=3, y=4] = 5
results in the call
>>> dict.__setitem__(d, k, 5)
where k is a hashable object, perhaps K(1, 2, x=3, y=4) for some new class
K.

Let's call this an OPAQUE use of k = K(1, 2, x=3, y=4). We don't look into
k, and the dict class checks that it is hashable.

In some other situations we wish for
>>> d[1, 2, x=3, y=4] = 5
to result in a call such as
 __setitem__(d, (1, 2), 5, x=3, y=4)
where __setitem__ is a function defined in the implementation of D =
type(d).

I fully support this goal, although not the implementation details in the
example above. It is my opinion that this goal is best achieved by making
easier TRANSPARENT use of
k = K(1, 2, x=3, y=4)

Here's how it goes. First we write
class D:
@wibble
def __setitem__(self, val, u, v, x, y):
pass  # Or do something.

Next, we define wibble. It will be a SIGNATURE CHANGING ADAPTER.  Those who
know how to make decorators will, I hope, have little difficulty in
defining wibble to do what is required. For this exercise, assume that
k.argv = (1, 2), and k.kwargs = dict(x=3, y=4).

The main idea is that each class will make an opaque use of the key, unless
it uses a signature changing adapter to enable a transparent use of the
key.  Thus, by default key use is opaque, but if a class wishes it can make
transparent use.

Without examples and working code (which I've promised for the end of the
month), this might be hard to understand. However this is I hope clear
enough for now.

SUGGESTIONS FOR OTHERS

Arising out of the discussion, it seems to me there are several different
mental models for
>>> d[...] = x
>>> x = d[...]
>>> del d[...]
particularly when keyword arguments are allowed in k[...].

I think clarifying the various mental models would be very helpful. By the
way, I don't think any one model will be better than all the others, for
all purposes.  This is particularly true for beginners and experts. And
also when there is already an existing mental model, such as in xarray.

The implementation of the resulting PEP would require change to user
documentation, and in particular to the tutorial.
https://docs.python.org/3/
https://docs.python.org/3/tutorial/index.html

I think it would be helpful to review the docs, to find some of the places
where the docs will need change. Some say that to really understand
something, you have to explain it to someone else, who doesn't share your
background.

I'm sure there are other useful things to do, such as exploring how all
this might help users of xarray. But I'm even less qualified to make
suggestions in that area.

I hope this post is helpful, particularly as I spent an hour writing it,
and perhaps a similar time thinking about these matters.

I think that's enough for now. When I have some code for you to look at,
I'll let you know.
-- 
Jonathan
___
Python-ideas mailing list -- python

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Sat, Jul 18, 2020 at 12:18:38AM -0400, Ricky Teachey wrote:
> On Fri, Jul 17, 2020 at 7:21 PM Steven D'Aprano  wrote:
> 
> > On Fri, Jul 17, 2020 at 11:11:17AM -0400, Ricky Teachey wrote:
> >
> > ...
> >
> > For backwards-compatibility, there will only ever be a single positional
> > argument passed into the method. That's because comma-separated values
> > in a subscript are already passed as a tuple:
> >
> > # this calls __getitem__ with a single tuple argument
> > obj[a,b:c,d]  ==> (1, slice(2, 3), 4)
> >
> > So that's not going to change (at least not without a long and painful
> > deprecation process). But adding support for keyword arguments requires
> > no changes to any existing class or a new builtin "key object" type.
> >
> 
> This strikes me as problematic for having a consistent mental model of how
> stuff works in python. I think that for many the difference in the meaning
> of the syntax between item-getting/setting and function-calling would be...
> glaring.

Yes, but what are we going to do about it?

Break a million existing scripts, applications and libraries that rely 
on `__getitem__` only receiving a single tuple argument when passed 
comma-separated values? I don't think the core devs will accept that, I 
think the numpy devs will object strongly, and I'm pretty sure that the 
Steering Council will say no.

But if you disagree, then feel free to start writing a PEP.

The fact that multiple comma-separated subscripts are passed to the 
method as a single tuple argument is a historical fact we (almost 
certainly) cannot change now. But that is orthogonal to how we choose to 
proceed with keyword arguments. We aren't obliged to repeat the same 
design.

We have a few choices:

(1) There is a minor inconsistency between subscripts and function 
calls, so let's just forget all about the whole idea. If we cannot agree 
on a decision, this is the default. (Status quo wins a stalement.)

(2) Let the Perfect be the enemy of the Good. No compromises! Insist on 
breaking the entire Python ecosystem for the sake of fixing this minor 
inconsistency between subscripting and function calls.

(3) Reinforce that inconsistency, and continue to obfuscate the 
similarities, by handling keyword arguments in the same fashion as 
comma-separated subscripts. This will require a new builtin "key-object" 
class, and it will require every class that cares about keyword 
arguments in their subscripts to parse them themselves.

We'll also need to decide how to combine subscripts and keywords:

obj[a, b:c, x=1]
# is this a tuple argument (a, slice(b, c), key(x=1))
# or key argument key(a, slice(b, c), x=1)

(4) Or keep the subscript processing as-is, for backwards-compatibility, 
but pass keyword arguments as normal for functions.

Both (3) and (4) would get the job done, but (3) requires everyone who 
needs keyword arguments to parse the tuple and/or key object by hand to 
extract them. Having done something similiar in the past (emulating 
keyword-only arguments in Python 2), I can tell you this is painful.

With (4), the interpreter automatically matches up passed keyword 
arguments to my `__getitem__` parameters, filling in defaults if needed, 
and I can concentrate on using the arguments, not parsing them.


> On the one hand, a fairly experienced person (who is familiar with the
> history of the item dunders, and a preexisting mental model that they are
> always being supplied a single positional argument ) should not have too
> much of a problem understanding WHY these would behave differently.
> 
> But on the other hand, even as an experienced person, this really messes
> with my brain, looking at it. It's hard for me to believe this isn't going
> to be a painful distinction for a large number of people to hold in their
> head-- especially beginners (but not only beginners).

I think you are overthinking this.

Inside a subscript, multiple positional arguments are collected into a 
tuple and passed as a single argument. (Vaguely similar to the way 
`*args` positional arguments are collected.)

Why? Because of historical reasons and backwards compatibility. If 
someone wants to trawl the archives looking for a discussion, I look 
forward to hearing the result, but we don't need to care about the past 
reason to learn it.

If you define your getitem like this:

def __getitem__(self, item, more):

then you'll get a TypeError when you try to subscript, because `more` 
doesn't get a value. This is already the case! So anyone writing getitem 
methods already knows that positional arguments aren't handled the same 
way as function calls.

If you give `more` a default, then you won't get an error... but even 
the tiniest bit of testing will reveal that `item` receives a tuple, and 
`more` always gets the default.

In practice, anyone writing getitem methods only ever gives it a single 
argument (aside from self of course :-) so if they add keyword 
arguments, the most natural way to do s

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

On Fri, Jul 17, 2020 at 7:21 PM Steven D'Aprano  wrote:

> On Fri, Jul 17, 2020 at 11:11:17AM -0400, Ricky Teachey wrote:
>
> ...
>
> For backwards-compatibility, there will only ever be a single positional
> argument passed into the method. That's because comma-separated values
> in a subscript are already passed as a tuple:
>
> # this calls __getitem__ with a single tuple argument
> obj[a,b:c,d]  ==> (1, slice(2, 3), 4)
>
> So that's not going to change (at least not without a long and painful
> deprecation process). But adding support for keyword arguments requires
> no changes to any existing class or a new builtin "key object" type.
>

This strikes me as problematic for having a consistent mental model of how
stuff works in python. I think that for many the difference in the meaning
of the syntax between item-getting/setting and function-calling would be...
glaring.

The left-hand-side syntax below looks pretty much identical in both cases
(except for the sugar of the slice object), but it feels
surprisingly different meanings for the positional arguments:

>>> a, b, c, d = (1, 2, 3, 4)
>>> args = a, slice(b, c), d
>>> #LEFT HAND SIDE  RIGHT HAND SIDE
>>> obj[a, b:c, d, e=5, f=6]  == obj.__getitem__(args, e=5, f=6)
>>> f(a, slice(b,c), d, e=5, f=6)  != f(args, d, e=5, f=6)

On the one hand, a fairly experienced person (who is familiar with the
history of the item dunders, and a preexisting mental model that they are
always being supplied a single positional argument ) should not have too
much of a problem understanding WHY these would behave differently.

But on the other hand, even as an experienced person, this really messes
with my brain, looking at it. It's hard for me to believe this isn't going
to be a painful distinction for a large number of people to hold in their
head-- especially beginners (but not only beginners).

A potentially elegant way around this glaring difference in the meaning of
the syntax might be the key-object paradigm Jonathan Fine has suggested.
However, that only works if you *disallow mixing together* positional
arguments and kwd args inside the [ ]:

>>> d[a="foo", b="bar"] == d[KeyObject(a="foo", b="bar")]

If you specifically desire to mix together positional and kwd arguments,
the key-object paradigm isn't nearly as elegant... there's still magical
stuff happening to "split off" the positional arguments from the keyword
arguments:

>>> # The positional arguments aren't part of the KeyObject
>>> d[a, b:c, d, e=5, f=6] == d.__getitem__((a, b:c, d), KeyObject(e=5,
f=6))

This raises a question that needs to be answered, then: what would be the
utility of mixing together positional and kwd arguments in this way?

Even the xarray examples given so far don't seem to make use of this
mixture. From my knowledge of pandas I am not sure what the meaning of
this would be, either.

> But! On the other hand, the huge difference here is that currently, the
> > signature of the get/set item dunders only accepts a single key argument.
>
> To be precise, you can put any signature you like into a `__getitem__`
> method, but only the first positional argument will be called from
> subscripting syntax.


I appreciate you pointing that out, it's a meaningful distinction because
it could mean that rewriting the default __getitem__ and __setitem__
signatures-- as you seem to be arguing for over the key-object paradigm--
won't be as nearly a painful/breaking change as it would be otherwise.


> Even if you pass comma-separated values, including slices, they all get
> packed into a tuple and passed as the first parameter `item`.
>
> But there is no need to emulate that for keyword args. They can, and I
> think should, simply be unpacked into keyword parameters exactly the
> same as function call syntax does.
>

Yup. Personally I really like the idea that unpacking some dict x will work
the same as unpacking in a function call:

>>> # Pretty!
>>> d[**x] == d[a=1, b=2]
>>> f(**x] == f(a=1, b=2)

Getting to the end here, I guess I'm really just wondering whether mixing
positional and kwd args is worth doing. If it isn't, then the key-object
paradigm seems like might be a nicer solution to me for the sole reason
that the mental model gets confused otherwise.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YNUSCJULZJJOK2P6QVTXTDHINM3RVSSV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Fri, Jul 17, 2020 at 11:11:17AM -0400, Ricky Teachey wrote:

> It seems to me that the validity of this key-object paradigm is directly
> tied to the decision of whether or not to change the get/set item dunder
> signatures.

But note that even if we allow keyword args in subscripts, we still 
don't have to change the signatures of existing objects. If keyword 
args aren't meaningful to a specific class, it will just change from a 
SyntaxError to a runtime TypeError.


> *If it turns out* the desired way forward is to not change the signature
> for __getitem__ and __setitem__, only then does a key object exists. If
> not, I suggest it means the key-object does not exist.
> 
> On the other hand, if it were determined we'd rather have **kwargs become
> part of these signatures, there is no "key object". There is more of an
> unpacked kwargs, or unpacked namespace, object.
>
> So it isn't obvious to me that this object *must be* some key object. It
> certainly CAN be. But the syntax looks so much like kwargs being supplied
> to a function, a (currently ephemeral) namespace object, or unpacked kwargs
> object, might be more of a valid way to think about it:

Indeed. And we need not require the method to unpack a kwargs dict 
themselves. We can allow (but not require!) keyword parameters in the 
method signature. Now the interpreter will handle the hard work of 
matching up keyword arguments to parameters.


class MyClass:
def __getitem__(self, item=None, *, x, y=0, **kw):
   # requires keyword arg x
   # optional keyword arg y
   # permits any other arbitrary keyword args

instance = MyClass()
instance[2:3, x=1, y=2, z=3]
# receives arguments item=slice(2, 3), x=1, y=2, kw={'z': 3}


For backwards-compatibility, there will only ever be a single positional 
argument passed into the method. That's because comma-separated values 
in a subscript are already passed as a tuple:

# this calls __getitem__ with a single tuple argument
obj[a,b:c,d]  ==> (1, slice(2, 3), 4)

So that's not going to change (at least not without a long and painful 
deprecation process). But adding support for keyword arguments requires 
no changes to any existing class or a new builtin "key object" type.


> But! On the other hand, the huge difference here is that currently, the
> signature of the get/set item dunders only accepts a single key argument.

To be precise, you can put any signature you like into a `__getitem__` 
method, but only the first positional argument will be called from 
subscripting syntax.

py> class Demo:
... def __getitem__(self, item, more=None, *args, x=0, y=1, **kw):
... return (item, more, args, x, y, kw)
...
py> Demo()['item']
('item', None, (), 0, 1, {})

Even if you pass comma-separated values, including slices, they all get 
packed into a tuple and passed as the first parameter `item`.

But there is no need to emulate that for keyword args. They can, and I 
think should, simply be unpacked into keyword parameters exactly the 
same as function call syntax does.

If, by chance, some class wants Jonathan's "keyword object" semantics, 
it's easy to get:

def __getitem__(self, **kwargs):

All the keyword arguments will be handed to you as a dict. You can then 
convert it to whatever special keyword object that suits your purposes.



-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KEEC3QI4H5AU754BCYSO75D6NCZ6UETS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Andras Tantos]

Jonathan and all!

Thanks for picking up on this thread, I almost given up hope that anyone 
would be interested. Then it suddenly blew up :)!


Jonathan, your suggestion makes sense as a stop-gap measure for current 
Python, but I'm unclear on the way forward to the new syntax:


When you say we introduce a new syntax where:

>>> value = d[K(1, 2, 3, a=4, b=5)]
>>> d[K(1, 2, 3, a=4, b=5)] = value

becomes:

>>> value = d[1, 2, 3, a=4, b=5]
>>> d[1, 2, 3, a=4, b=5] = value

somehow the interpreter would need to know that the intent is to create 
an instance of 'K'. How is that association expressed? Or is the idea 
that - the same was as slices - there would be a predefined object 
within the language that is always the thing that gets created in these 
instances?


It appears to me that the current behavior is:

    d[32] becomes d.__getitem__(32)

    d[2,4,"boo"] becomes d.__getitem__( (2,4,"boo") )

    d[2:3:"boo"] becomes d.__getitem__( slice(2,3,"boo") )

So then maybe

    d[2,4,scare="boo"] could become d.__getitem__( {0:2, 1:4, 
"scare":"boo"} )?


That is of course quite a bit different from the args, kwargs syntax, 
but I don't think it's invalid as positional arguments would get an 
integer key while keyword arguments would get a string key.


With this change, of course your previous (current python) syntax would 
make K into a function as opposed to a class:


def K(*args, **kwargs):
    ret_val = {}
    for idx, value in enumerate(args):
    ret_val[idx] = value
    for key, value in kwargs.items():
    assert isinstance(key, str)
    ret_val[key] = value
    return ret_val

>>> value = d[K(1, 2, 3, a=4, b=5)]
>>> d[K(1, 2, 3, a=4, b=5)] = value

Thanks again,
Andras
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/V7P2CRWGIAUCDR7S7A4ZJV75Q7WCDRND/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Rhodri James]

On 17/07/2020 21:11, Todd wrote:

On Fri, Jul 17, 2020 at 12:19 PM David Mertz  wrote:


Fwiw, I'm probably -0 on the feature itself. Someone suggested it could be
useful for xarray, but I'm not sure now what that would look like. If
someone had an example, I could easily be moved.



Here is what it currently looks like to assign values to indices in xarray
(adapted from a tutorial):

ds["empty"].loc[dict(lon=5, lat=6)] = 10

This could be changed to:

ds["empty"][lon=5, lat=6] = 10

This becomes even a bigger advantage if we include slicing, which I think
we should:

ds["empty"].loc[dict(lon=slice(1, 5), lat=slice(3, None))] = 10

to

ds["empty"][lon=1:5, lat=6:] = 10


Thanks for posting this.  I had been really struggling to see a use case 
that made any kind of sense to me, possibly because I am not a data 
scientist and have no interest in becoming one.  This helped a lot.  I 
particularly like the slice notation, there is a clear win there.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4M6DCHAEPFXDLDULALZQZCHOFL2ILDJM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz via Python-ideas]

On Fri, Jul 17, 2020 at 4:12 PM Todd  wrote:

> ds["empty"][lon=1:5, lat=6:] = 10
>

I agree that looks really nice. I think this is the first suggestion to
allow slices on the RHS of keyword indexing. That's the part that was
missing in me understanding how this would help xarray greatly. It's also
somewhat of an additional novelty added to the proposal that is somewhat
independent.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E4RTBDZL6CHMZAMCNWLMJ3RHZILDM5CT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Todd]

On Fri, Jul 17, 2020 at 12:19 PM David Mertz  wrote:

> Fwiw, I'm probably -0 on the feature itself. Someone suggested it could be
> useful for xarray, but I'm not sure now what that would look like. If
> someone had an example, I could easily be moved.
>

Here is what it currently looks like to assign values to indices in xarray
(adapted from a tutorial):

ds["empty"].loc[dict(lon=5, lat=6)] = 10

This could be changed to:

ds["empty"][lon=5, lat=6] = 10

This becomes even a bigger advantage if we include slicing, which I think
we should:

ds["empty"].loc[dict(lon=slice(1, 5), lat=slice(3, None))] = 10

to

ds["empty"][lon=1:5, lat=6:] = 10
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RZLZOUW5GUQDBBJCSOANGLTDTJCOJVGF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Todd]

On Fri, Jul 17, 2020, 12:10 David Mertz  wrote:

> On Fri, Jul 17, 2020, 8:16 AM Jonathan Fine  wrote:
>
>> Steve and I have different opinions, as to what the new behaviour of:
>> >>> d = dict()
>> >>> d[x=1, y=2] = 3
>> should be.
>>
>> He prefers that the assignment fail with
>> TypeError: dict subscripting takes no keyword arguments
>>
>> I prefer that the assignment succeed (and hence a new key-value pair is
>> added to 'd').
>>
>
> I definitely agree with Steven. It is obviously *possible* to create some
> brand new type of object that is "multi-assignment fragment." But why?!
>
> No clearly useful semantics comes to mind for this new object. Well, it
> would need to be hashable. Lots of things are though, so it's not like we
> have nothing to use as dict keys now.
>
> We don't lose anything if we add the feature but intake don't support it
> for dictionaries. If someone comes up with a really useful reason to have
> that MultiAssignmentType, is not usually considered a breaking change to go
> from "this raises am exception" to "this does something worthwhile" (but
> obviously, in all such cases, you can artificially construct code that will
> break without a certain exception).
>

I think there are a few different options, with different advantages and
disadvantages.  Some have been touched on, but I don't think they are the
only, or necessarily even best, options.  I think it would be worthwhile
looking at them all explicitly since I think they have different advantages
and disadvantages:

1. Use keyword args.  This has the advantage that it automatically handles
putting things into a dict or limiting the potential keys, and will fail
reliably if keyword indices aren't handled.  But it has a bunch of issues.
First, it is inconsistent with the existing indexing, which is a single
tuple.  Second, it only allows valid identifiers, which may be too
limiting.  Third, the position of positional and keyword indices are
independent.

2.  A single additional argument, containing a tuple of name/value tuples
or a dict.  This is consistent with the existing indexing, allows arbitrary
identifiers, and will fail reliably if not handled.  It still has the issue
with positional and keyword indices having independent orders.

3. Use the existing tuple, but put keyword indices inside it as single-item
dicts or named tuples.  This has the advantage of keeping track of all the
positions together, but you can already use these as indices so it has more
potential for backwards compatibility issues.

4. Use the existing tuple, but put keyword indices in a new class.  This
keeps track of positions and doesn't have the same backwards compatibility
issue, but would probably need either a new built-in or at least something
in a stdlib module, and I am not sure where that would go.  Care also needs
to be made that it will fail properly in all cases where keyword indices
aren't supported.

5. Use an entire new class for all indices when keyword indices are
provided. This still requires a new class, and care has to be made to make
sure it isn't accidentally handled improperly by existing classes.

Overall I think option 2 is the best.  It has pretty much no possibility of
backwards-incompatibility, can handle the widest variety of values, and
doesn't require any new classes or new behavior on the implementation side.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EJSWZMAN3RVG4GAPQOKFTCX63BOU7DB4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Christopher Barker]

On Fri, Jul 17, 2020 at 11:49 AM Jonathan Fine  wrote:

> I am able today to respond to one question. Ricky asked if
>>>> key_object = K(a=1, b=2)  # where K is some new key object type
>>>> d1 = {key_object: 3}
>>>> d2 = {}
>>>> d2[a=1, b=2] = 3
>>>> assert d1==d2
> was what I had in mind. The answer is YES, and in some ways better
> expressed. (And the new key object type is introduced only if necessary for
> the desired user experience.)
>

Maybe I’m being dense here, but I need more explainatikn of what this “key
object” is, how it works, or what its purpose is.

The use of dict literals makes the role of the key_object even clearer.
>

Not to me :-(

For clarity, when we write
> >>> d[1, 2, 3] = 4
> the associated key_object, for backwards compatibility, must be (1, 2, 3),
> which is a tuple.
>

Yes, and tuples are perfectly valid dict keys, so nothing new here.

>
> Again for clarity, in today's Python the statements
> >>> d[1, 2, 3] = 4
> >>> d[(1, 2, 3)] = 4
> are equivalent. I willingly accept this as a constraint. In my own mind, I
> require it.
>

Which is due to tuples being created by the commas ...

So what would the “key object” be I the proposed case:

 d2[a=1, b=2]

A namedtuple? Or namedtuple-like object?

The other trick here is that currently in indexes, the comma currently
creates a tuple in the general case, for instance:

Something[1:2, 3:4] creates  a tuple with two slice objects in it. (Heavily
used by numpy.

So if d2[a=1, b=2] creates single object (which should not be
dict-specific, so not a key per se), how would you created a tuple of more
than one such object? Maybe:

Something[(a=5, b=6), (c=7, d=8)]

Anyway, I think a prototype is in order.

-CHB



Once again, thank you all, and I'll say more tomorrow (about 18 hours from
> now).
> --
> Jonathan
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UFXWGFAUUQGMA7DBNYAK53THIWQOL632/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MPAGYF33Q3JA4JTEYAHJCGQ6LWPIKN2L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Thank you all, for your useful contributions. I particularly value the
insight you've given me regarding the experience that underlies your views.
I'll respond to your comments tomorrow.

I am able today to respond to one question. Ricky asked if
   >>> key_object = K(a=1, b=2)  # where K is some new key object type
   >>> d1 = {key_object: 3}
   >>> d2 = {}
   >>> d2[a=1, b=2] = 3
   >>> assert d1==d2
was what I had in mind. The answer is YES, and in some ways better
expressed. (And the new key object type is introduced only if necessary for
the desired user experience.)

The use of dict literals makes the role of the key_object even clearer. For
clarity, when we write
>>> d[1, 2, 3] = 4
the associated key_object, for backwards compatibility, must be (1, 2, 3),
which is a tuple.

Again for clarity, in today's Python the statements
>>> d[1, 2, 3] = 4
>>> d[(1, 2, 3)] = 4
are equivalent. I willingly accept this as a constraint. In my own mind, I
require it.

Once again, thank you all, and I'll say more tomorrow (about 18 hours from
now).
-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UFXWGFAUUQGMA7DBNYAK53THIWQOL632/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

On Fri, Jul 17, 2020 at 12:30 PM Christopher Barker 
wrote:

> On Fri, Jul 17, 2020 at 9:20 AM David Mertz  wrote:
>
>> Fwiw, I'm probably -0 on the feature itself. Someone suggested it could
>> be useful for xarray, but I'm not sure now what that would look like. If
>> someone had an example, I could easily be moved.
>>
>
> agreed -- I can imagine the use case, but am not enough of an xarray user
> to com eup with a compelling example.
>
> But if it IS useful for xarray (and maybe Pandas, and who knows what other
> nifty packages -- maybe a database wrapper? -- that doesn't mean it has to
> be used for Mappings.
>
> Jonathan -- I'm still quite confused about what your proposed syntax with
> dicts means, could you maybe both:
>
> describe it in words
>
> and
>
> Give complete examples -- that is, what the dict looks like before an
> after the operation, and what is returned.
>

It seemed clear to me that it would assign some key object key to the
supplied value:

>>> key_object = K(a=1, b=2)  # where K is some new key object type
>>> d1 = {key_object: 3}
>>> d2 = {}
>>> d2[a=1, b=2] = 3
>>> assert d1==d2

It will be interesting to see if my intuition on it lines up with what
Jonathan had in mind.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/M3I2HXPRRIYNDH7SSZFLPEIM7ZZVMR7Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Christopher Barker]

On Fri, Jul 17, 2020 at 9:20 AM David Mertz  wrote:

> Fwiw, I'm probably -0 on the feature itself. Someone suggested it could be
> useful for xarray, but I'm not sure now what that would look like. If
> someone had an example, I could easily be moved.
>

agreed -- I can imagine the use case, but am not enough of an xarray user
to com eup with a compelling example.

But if it IS useful for xarray (and maybe Pandas, and who knows what other
nifty packages -- maybe a database wrapper? -- that doesn't mean it has to
be used for Mappings.

Jonathan -- I'm still quite confused about what your proposed syntax with
dicts means, could you maybe both:

describe it in words

and

Give complete examples -- that is, what the dict looks like before an after
the operation, and what is returned.

>
> I'm not against the original suggested use with type annotations, but I
> also don't really care about it.
>

me neither, and I'm a little against it -- I really don't ike all the
typing creeping into python these days ...

-CHB



-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NEROTSFPISDXMQSUTK3SBBJPA65H4WCN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

Fwiw, I'm probably -0 on the feature itself. Someone suggested it could be
useful for xarray, but I'm not sure now what that would look like. If
someone had an example, I could easily be moved.

I'm not against the original suggested use with type annotations, but I
also don't really care about it. I don't think 'd[K(1, 2, 3, a=4, b=5)]' is
bad as an existing spelling.

On Fri, Jul 17, 2020, 12:08 PM David Mertz  wrote:

> On Fri, Jul 17, 2020, 8:16 AM Jonathan Fine  wrote:
>
>> Steve and I have different opinions, as to what the new behaviour of:
>> >>> d = dict()
>> >>> d[x=1, y=2] = 3
>> should be.
>>
>> He prefers that the assignment fail with
>> TypeError: dict subscripting takes no keyword arguments
>>
>> I prefer that the assignment succeed (and hence a new key-value pair is
>> added to 'd').
>>
>
> I definitely agree with Steven. It is obviously *possible* to create some
> brand new type of object that is "multi-assignment fragment." But why?!
>
> No clearly useful semantics comes to mind for this new object. Well, it
> would need to be hashable. Lots of things are though, so it's not like we
> have nothing to use as dict keys now.
>
> We don't lose anything if we add the feature but intake don't support it
> for dictionaries. If someone comes up with a really useful reason to have
> that MultiAssignmentType, is not usually considered a breaking change to go
> from "this raises am exception" to "this does something worthwhile" (but
> obviously, in all such cases, you can artificially construct code that will
> break without a certain exception).
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/62Y7TQXAMH2HCJPRCZOBHVHQJ76IBG4O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Fri, Jul 17, 2020, 8:16 AM Jonathan Fine  wrote:

> Steve and I have different opinions, as to what the new behaviour of:
> >>> d = dict()
> >>> d[x=1, y=2] = 3
> should be.
>
> He prefers that the assignment fail with
> TypeError: dict subscripting takes no keyword arguments
>
> I prefer that the assignment succeed (and hence a new key-value pair is
> added to 'd').
>

I definitely agree with Steven. It is obviously *possible* to create some
brand new type of object that is "multi-assignment fragment." But why?!

No clearly useful semantics comes to mind for this new object. Well, it
would need to be hashable. Lots of things are though, so it's not like we
have nothing to use as dict keys now.

We don't lose anything if we add the feature but intake don't support it
for dictionaries. If someone comes up with a really useful reason to have
that MultiAssignmentType, is not usually considered a breaking change to go
from "this raises am exception" to "this does something worthwhile" (but
obviously, in all such cases, you can artificially construct code that will
break without a certain exception).
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MUR2S5NGWCNT52Q2HXLEKSVXCVA3RCE6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

On Fri, Jul 17, 2020, 6:17 AM Jonathan Fine  wrote:

> Guido wrote:
>
>> I think it’s a reasonable idea and encourage you to start working on a
>> design for the API and then a PRP.
>>
>
> In this post, I explore how the new API might interact with dict objects.
> (I think PRP is a typo for PEP.)
>
>  ...
>
> To summarize and slightly extend the above. If
> >>> d = dict()
> >>> d[x=1, y=2] = 3
> executes without error, then there is an object 'k' such that
> >>> d[x=1, y=2] = 5
> >>> d[k] = 5
> are equivalent. (This object 'k' is the key in the collection of (key,
> value) pairs that are the entries in the dict.)
>
> Aside: The key object, if it exists, has a type. Whether this type is a
> new type or an existing type is an implementation question. This message is
> focussed on the user experience.
>

It seems to me that the validity of this key-object paradigm is directly
tied to the decision of whether or not to change the get/set item dunder
signatures.

*If it turns out* the desired way forward is to not change the signature
for __getitem__ and __setitem__, only then does a key object exists. If
not, I suggest it means the key-object does not exist.

On the other hand, if it were determined we'd rather have **kwargs become
part of these signatures, there is no "key object". There is more of an
unpacked kwargs, or unpacked namespace, object.

So it isn't obvious to me that this object *must be* some key object. It
certainly CAN be. But the syntax looks so much like kwargs being supplied
to a function, a (currently ephemeral) namespace object, or unpacked kwargs
object, might be more of a valid way to think about it:

>>> x = dict(a="foo", b="bar")
>>> d[**x]  # unpack kwargs here, rather than d[x]
>>> d[a="foo", b="bar"]

If one approaches it this way, then just as there isn't some namespace
object that you can supply to a function that becomes kwd args in all
situations, like:

>>> f(x) == f(a="foo", b="bar")  # this obviously doesn't make sense, since
a and b are kwargs

...I'd be more than a little surprised if this worked:

>>> d[x] == d[a="foo", b="bar"]  # a and b certainly LOOK LIKE kwargs,
don't they?

But! On the other hand, the huge difference here is that currently, the
signature of the get/set item dunders only accepts a single key argument.
This is a very big difference from arbitrary function signatures, which CAN
accept kwd args. So maybe this *key object paradigm* does make sense, but
what that REALLY means is we are committing ourselves to maintaining
the __getitem__ and __setitem__ signatures and not, for example, letting
them accept **kwargs.

On Fri, Jul 17, 2020, 6:49 AM Steven D'Aprano  wrote:
>
> On Fri, Jul 17, 2020 at 11:14:21AM +0100, Jonathan Fine wrote:
>
> > So what should be the new behaviour of:
> > >>> d = dict()
> > >>> d[x=1, y=2] = 3
>
> TypeError: dict subscripting takes no keyword arguments
>
>
>
> Just because something is syntactically allowed doesn't mean it has to be
> given a meaning in all circumstances.
>

Another way forward could be to expand the syntax, but delay the decision
on what it means. It would be a multi-step process:

   1. allow usage of Dict[x=str, y=int] (including a TypeError on
   assignment, and TypeError for dictionaries, and TypeError for all other
   mappings)
   2. delay the decision of which paradigm (key-object, or
   unpacked/namespace) makes sense (a TypeError everywhere else  in the
   meantime)
   3. once it is seen the new syntax is allowed but produces TypeError in
   all situations other than on `type` objects, let the people tell the
   steering council what they want the syntax to do in other situations

So INITIALLY you only teach`type` objects-- and NOT regular dicts or
mappings-- how to interpret this syntax:

>>> Dict[x=str, y=int]  # k, cool
>>> d[a="foo", b="bar"]  # TypeError

If people later demand that you be able to do this (with real world code
examples):

>>> xy = dict(x=str, y=int)
>>> Dict[**xy] == Dict[x=str, y=int]

...that tells you people want the namespace/unpacked paradigm. On the other
hand, if people start asking (with real world code examples) for some kind
of key object so they can do this:

>>> d[x] == d[a="foo", b="bar"]

...that tells you people are thinking about it more as a key-object
paradigm.

Rick.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XQNMC6VC7GKIDTQEF45GXWYYRGEWMCG7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Steve and I have different opinions, as to what the new behaviour of:
>>> d = dict()
>>> d[x=1, y=2] = 3
should be.

He prefers that the assignment fail with
TypeError: dict subscripting takes no keyword arguments

I prefer that the assignment succeed (and hence a new key-value pair is
added to 'd').

I think this is a important question to get right. And a pressing question.
If we choose unwisely now, it might be difficult to change our mind later
(particularly after the feature has been released).

To support his opinion, Steven wrote:

> Just because something is syntactically allowed doesn't mean it has to be
> given a meaning in all circumstances.


I completely agree with this statement. And I hope that Steven agrees with
the statement: Just because something can be semantically forbidden doesn't
mean that it should to be.

We have an opportunity to improve Python for the benefit of its users. I
think it would be helpful to have some concise statements on the benefits of
>>> d = dict()
>>> d[x=1, y=2] = 3
TypeError: dict subscripting takes no keyword arguments

And similarly, concise statements on the benefits of
>>> d = dict()
>>> d[x=1, y=2] = 3
>>> d[x=1, y=2]
3

By the way, I'd prefer that a problem seen in the one choice is instead
expressed as a benefit of the other.

To conclude, I'm pleased that this difference of opinion has emerged sooner
rather than later. I hope we have a discussion that leads to an improved
shared understanding.

I hope this message helps.

-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2MTOTJK3UGRAW3ANJTUT27PBLTBDHQ36/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Fri, Jul 17, 2020 at 11:14:21AM +0100, Jonathan Fine wrote:

> So what should be the new behaviour of:
> >>> d = dict()
> >>> d[x=1, y=2] = 3

TypeError: dict subscripting takes no keyword arguments


Just because something is syntactically allowed doesn't mean it has to 
be given a meaning in all circumstances. Just as not all functions 
accept keyword arguments:

py> len(x=1)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: len() takes no keyword arguments

so not all subscriptable objects will accept keywords.


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6IM6CTISEJUCNKHUXBXFAFMI4FQEBGRX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Guido wrote:

> I think it’s a reasonable idea and encourage you to start working on a
> design for the API and then a PRP.
>

In this post, I explore how the new API might interact with dict objects.
(I think PRP is a typo for PEP.)

 Here is an example of the present behaviour
>>> d = dict()
>>> d[x=1, y=2] = 3
SyntaxError: invalid syntax
with the syntax error occurring at the first '=' symbol.

So what should be the new behaviour of:
>>> d = dict()
>>> d[x=1, y=2] = 3

To me, it is clear that we won't get a syntax error. This is because the
proposal implies that
>>> d[x=1, y=2] = 3
executes without error, for some values of 'd'.

If these two lines
>>> d = dict()
>>> d[x=1, y=2] = 3
execute without error, then I would the expect
>>> len(d)
1
>>> d[x=1, y=2]
3

I would further expect
>>> k = list(d.keys())[0]
to give an object 'k', with the property
>>> d[k]
3

This object 'k' corresponds to the source code fragment
x=1, y=2
in
d[x=1, y=2]
and I would expect 'k' to work as a key in any instance of dict.

Here is something that might help the reader. It is that current Python
gives
>>> d = dict()
>>> d['x=1', 'y=2'] = 3
>>> list(d.keys())[0]
('x=1', 'y=2')
and we are looking for an analogous object for
>>> d[x=1, y=2] = 3

To summarize and slightly extend the above. If
>>> d = dict()
>>> d[x=1, y=2] = 3
executes without error, then there is an object 'k' such that
>>> d[x=1, y=2] = 5
>>> d[k] = 5
are equivalent. (This object 'k' is the key in the collection of (key,
value) pairs that are the entries in the dict.)

Aside: The key object, if it exists, has a type. Whether this type is a new
type or an existing type is an implementation question. This message is
focussed on the user experience.

To conclude: If d[x=1, y=2] is allowed for 'd' a dict then consistency with
the current dict behaviour requires the existence of a key object that
corresponds to the fragment 'x=1, y=2'.

Finally, I have tried to be both clear and concise. I hope my contribution
helps.
-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JZAY2VWTGYSAG6LILU57LUEPM6DUYEYZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Guido van Rossum]

I think it’s a reasonable idea and encourage you to start working on a
design for the API and then a PRP. It would help if someone looked into a
prototype implementation as well (once a design has been settled on).

On Thu, Jul 16, 2020 at 03:31 Steven D'Aprano  wrote:

> On Wed, Jul 15, 2020 at 11:09:42PM -0700, Caleb Donovick wrote:
>
> > I have wanted this and suggested it before for use with typing.
>
> Was Guido interested or did he reject the idea?
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/4OJ7OXKG75IXT2KX3UQJSQYEAQ52VAUM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7MHQYEOGFF764ZSME76UCMAFGPB2PR22/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Todd]

On Fri, Jul 10, 2020, 12:44 Steven D'Aprano  wrote:

> On Fri, Jul 10, 2020 at 02:48:31PM +0200, Alex Hall wrote:
>
> > I believe he was saying it would be weird if `d[1, 2]` called
> > `d.__getitem__((1, 2))` but `d[1, 2, x=3]` called `d.__getitem__(1, 2,
> > x=3)`. More generally, if `__getitem__` always receives a single
> positional
> > argument now, it should probably stay that way for consistency.
>
> Right -- I already said as much too :-)
>
> I think Jonathan's K class is a red herring (except to the degree that
> it could be used for experimentation). For backwards compatibility, it
> would be difficult or impossible to support multiple positional
> arguments. But we could add keyword only args to subscripting (if there
> is a compelling need for them) easily.
>
> Existing signatures would stay the same:
>
>  def __getitem__(self, item)
>  def __setitem__(self, item, value)
>
> but those who wanted keyword args could add them, with or without
> defaults:
>
>  def __getitem__(self, item, *, spam=None)
>  def __setitem__(self, item, value, *, spam, eggs=None)
>
> I trust the expected behaviour is obvious, but in case it's not:
>
> myobj[1, 2, spam=3]  # calls __getitem__((1, 2), spam=3)
>
> myobj[1, 2, spam=3]  = 999
> # calls __setitem__((1, 2), 999, spam=3, eggs=None)
>
> (And similarly for delitem of course.)
>
>
> I must admit I like the look of this, but I don't know what I would use
> it for.
>

This would be EXTREMELY useful for xarray, which uses labelled dimensions.
So you can index and slice based on dimension position, but also based on
dimension name.  It is currently very awkward to index or slice by
dimension name, though (particularly for slicing).

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7N3QRDKLDR2234I54QCU4USFLWHTXEGE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Wed, Jul 15, 2020 at 11:09:42PM -0700, Caleb Donovick wrote:

> I have wanted this and suggested it before for use with typing.

Was Guido interested or did he reject the idea?


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4OJ7OXKG75IXT2KX3UQJSQYEAQ52VAUM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Paul Sokolovsky]

Hello,

On Wed, 15 Jul 2020 23:09:42 -0700
Caleb Donovick  wrote:

> I have wanted this and suggested it before for use with typing.
> 
> Defining  protocols is obnoxiously verbose for "struct" like data and
> keyword
> arguments to subscript could help alleviate that.
> I often want to write type hint like this:
> 
> ```
> def foo(x: Protocol[id=int, name=str]):
>   bar(x)
>   baz(x)


Just write them as:

---
from __future__ import annotations

def foo(x: Protocol(id=int, name=str)):
---

So, if using in annotations is the usecase, no new language syntax is
required, just update your "Protocol" definition to
https://www.python.org/dev/peps/pep-0563/

[]

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/O2CCKMRHWUMGGC2YEHKXXL4ZN3DFYFHJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Hi Caleb

You wrote (call it FUTURE)

> def foo(x: Protocol[id=int, name=str]):
>   bar(x)
>   baz(x)
>

 As you know, at present this causes a SyntaxError.

However (call it NOW)
def foo(x: Protocol[o(id=int, name=str])):
  bar(x)
  baz(x)
has no syntax error.

I'll now state some goals.

1. Define 'o' and Protocol so that NOW gives the semantics you wish for.
2. Extend Python so that FUTURE give the semantics you wish for.
3. And the NOW syntax continues to work as expected (without changing 'o'
and Protocol).
4. And all current use of container[key] continues to work as before.

I believe that it is possible to achieve these goals. My previous posts to
this discussion outline some of the key ideas. My next step, when I have
time, is to implement and publish general purpose code for the NOW part of
this list of goals.

I hope this will help you and others.

with best regards

Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3IWL42XYUVODDVTPXF7EKXF6NE4JCRTV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Caleb Donovick]

I have wanted this and suggested it before for use with typing.

Defining  protocols is obnoxiously verbose for "struct" like data and
keyword
arguments to subscript could help alleviate that.
I often want to write type hint like this:

```
def foo(x: Protocol[id=int, name=str]):
  bar(x)
  baz(x)

def bar(x: Protocol[name=str]): ...

def baz(x: Protocol[id=int]): ...
```

So  I either need to specify more restrictive types than necessary (which
often
is not possible because I reuse my functions), or generate a combinatorial
number of Protocols.  Beyond the obvious annoyances that come with having
to
generate many protocols, simply naming them is cognitively expensive.  I
don't
need to bind an identifier when declaring a Union or specializing a generic
but
if I want to say I have a type with some attribute it MUST be named.


On Fri, Jul 10, 2020 at 10:00 AM Paul Moore  wrote:

> On Fri, 10 Jul 2020 at 17:45, Steven D'Aprano  wrote:
>
> > I must admit I like the look of this, but I don't know what I would use
> > it for.
>
> It feels very much like the sort of "here's some syntax that might
> match someone's mental model of something" that is common in languages
> that focus on allowing users to build their own DSLs¹ (Lua and Groovy
> are two examples of the type of language I'm thinking of, although I
> don't know if either has this particular syntax).
>
> Python typically doesn't encourage DSL-style programming, so this type
> of "syntax looking for a use case" isn't very popular.
>
> Paul
>
> ¹ DSL = Domain Specific Language, in case anyone isn't familiar with the
> term.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HUELPGHPCAIFLUKCQ5G7O2LKQFRRZ6CU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q3LB4F76MPWC33D2KZFNO6PL4XSOSS6V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Paul Moore]

On Fri, 10 Jul 2020 at 17:45, Steven D'Aprano  wrote:

> I must admit I like the look of this, but I don't know what I would use
> it for.

It feels very much like the sort of "here's some syntax that might
match someone's mental model of something" that is common in languages
that focus on allowing users to build their own DSLs¹ (Lua and Groovy
are two examples of the type of language I'm thinking of, although I
don't know if either has this particular syntax).

Python typically doesn't encourage DSL-style programming, so this type
of "syntax looking for a use case" isn't very popular.

Paul

¹ DSL = Domain Specific Language, in case anyone isn't familiar with the term.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HUELPGHPCAIFLUKCQ5G7O2LKQFRRZ6CU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Fri, Jul 10, 2020 at 02:48:31PM +0200, Alex Hall wrote:

> I believe he was saying it would be weird if `d[1, 2]` called
> `d.__getitem__((1, 2))` but `d[1, 2, x=3]` called `d.__getitem__(1, 2,
> x=3)`. More generally, if `__getitem__` always receives a single positional
> argument now, it should probably stay that way for consistency.

Right -- I already said as much too :-)

I think Jonathan's K class is a red herring (except to the degree that 
it could be used for experimentation). For backwards compatibility, it 
would be difficult or impossible to support multiple positional 
arguments. But we could add keyword only args to subscripting (if there 
is a compelling need for them) easily.

Existing signatures would stay the same:

 def __getitem__(self, item)
 def __setitem__(self, item, value)

but those who wanted keyword args could add them, with or without 
defaults:

 def __getitem__(self, item, *, spam=None)
 def __setitem__(self, item, value, *, spam, eggs=None)

I trust the expected behaviour is obvious, but in case it's not:

myobj[1, 2, spam=3]  # calls __getitem__((1, 2), spam=3)

myobj[1, 2, spam=3]  = 999
# calls __setitem__((1, 2), 999, spam=3, eggs=None)

(And similarly for delitem of course.)


I must admit I like the look of this, but I don't know what I would use 
it for.


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NYXVHO233EUIAND7WC5SRFXYWC74KSLL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Fri, Jul 10, 2020 at 01:59:06PM +0100, Jonathan Fine wrote:

> I wrote: Let's proceed. We continue to use d = Dummy(). Given that
>>>> key = d[1, 2, 3, a=4, b=5]
> is allowed, what should we be able to say about the key. Clearly it should
> be an instance of a class and there should be a way of creating such an
> instance without going via d. Let's call the class K (for key).
> 
> I'll now expand on this, as it's not clear to everyone that the key should
> be an instance of a class.

Of course it is clear that if `key` *exists at all*, then it will be an 
instance of a class, and furthermore that there will be ways to 
instantiate that class independently of subscript syntax.

This is fundamental to Python's design.

*If* it exists though. And I'm not convinced that there's any need for 
this K class, which seems to be analogous to slice(). Or at least, it 
needs to be justified, and not just assumed.

The alternative model is to treat subscripting analogously to function 
calls, and bind the arguments to parameters in the method. In that case, 
your "key" object won't even exist.

That's how slicing used to work, and there's no reason why we couldn't 
do the same thing in the future for any enhancement.

Before getting bogged down in the question of how to implement this, 
we ought to firstly establish what this is, and whether we need it.

Having read your posts repeatedly, I think it is safe to say that you 
are thinking along the lines of a something analogous to the slice() 
builtin. It might have made your explanation easier to understand if you 
had made that analogy explicit:

# Current Python
myobj[a:b:c] --> myobj.__getitem__(slice(a, b, c))

# Proposed enhancement:

myobj[a, b, c, spam=d, eggs=e] 
--> myobj.__getitem__(lump(a, b, c, spam=d, eggs=e))
# and similar for assignment and deletion

where "lump" (for lack of a better name) is a helper object analogous to 
"slice", but accepting arbitrary positional and keyword arguments.

(Note: to be precise, dunder methods aren't looked up on the instance 
themselves, but on the class. But as shorthand, I'll continue to pretend 
otherwise.)

As Paul says, this is pretty obvious stuff so far. Also obvious is that 
we can define our own "lump" class and experiment with it:

# Much like your Dummy class.
class lump:
def __init__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs

In your `myobj.__getitem__` method, we can handle the "lump":

# myobj's class
def __getitem__(self, item):
print(item.args)
print(item.kwargs)

which is enough for experimentation. You can even alias `K=lump` if you 
prefer :-)

Some additional issues:

For getters, why not just make the object callable and write 
`myobj(spam=1)`? We need to justify the use of square brackets.

For setters, if we're going to change the syntax to allow keywords in 
subscripts, perhaps it is better/easier to just allow function calls on 
the left hand side of assignments? We need to consider why we are doing 
this, what we hope to do, and whether square or round brackets are 
preferable.

For both: there are serious backwards compatibility concerns regarding 
the use of multiple parameters, since they are currently legal and bound 
up in a tuple argument:

# Current Python
myobj[1, 2]  # calls __getitem__ with a single tuple argument (1, 2)

# How to distinguish the above from this?
myobj[1, 2]  # call __getitem__ with two int arguments?

However, there is no such concern regarding keyword arguments.


> So far I know, every Python object that is accessible to the Python user is
> an instance of a class (or type).

Correct. Even type itself is an instance of itself.


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ST7OUUJMKG4VKSFQL3X3WVYC64TETGF6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Hi Ricky

Thank you for your very helpful comment, which Paul has usefully amplified.
You wrote:

This is confusing to me because `my_mapping[1,2,3]` is already valid syntax
> equivalent to `my_mapping.__setitem__(my_mapping, (1,2,3))`. Wouldn't a
> change to translate the tuple into this K object lead to big problems for a
> lot of existing code...?


Yes it would, if we don't take care to maintain backwards compatibility. At
present we have:
>>> class Dummy:
... def __getitem__(self, *argv, **kwargs):
... return argv, kwargs
>>> d = Dummy()
>>> d[1, 2, 3]
(((1, 2, 3),), {})

This behaviour must be preserved. Only when the new syntax is used
should d.__getitem__ get an instance of the new class K. Otherwise, it
should as before get a tuple, exactly as before. (Therefore, it might be
good that K(1, 2, 3) returns the tuple (1, 2, 3), rather than an instance
of K. As I recall, this can be done by using a suitable K.__new__ method.)

And this would include the use of colon to obtain slices.:
>>> d[0:1,2:3,4:5]
(((slice(0, 1, None), slice(2, 3, None), slice(4, 5, None)),), {})

This solution to backwards compatibility is another reason for using a
K.adjust_get decorator, when defining the __getitem__ for a map that relies
on the new syntax. That way, if the programmer wants the new feature, the
appropriate marshalling of positional (and keyword) arguments is done by a
dedicated piece of code.

I hope this helps, and thank you again for your query (and Paul for his
constructive critical interest).
-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7F3NDSZ5DMV5XBPEOPPNST2YR5RFSD47/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Paul Moore]

On Fri, 10 Jul 2020 at 15:23, Jonathan Fine  wrote:
> With hindsight, I can see your difficulty. The original idea is to extend the 
> syntax so that
> >>> m = MyMap()
> >>> m[1, 2, 3, a=4, b=5] = 'foobar'
> is allowed in some future version of Python.
>
> Python's syntax already allows:
> >>> m = MyMap()
> >>> m[K(1, 2, 3, a=4, b=5)] = 'foobar'
>
> My idea is to implement this, so that we can better explore the original 
> idea. (In particular, this would allow the idea to be tested in practice, and 
> perhaps acquire a community of users.)
[...]
> The code I wrote, which wasn't clear enough, sketched an adapter between two 
> points of view (although for get rather than set).

OK, thanks. I'm probably not the target for your comment in that case,
as (now that you've clarified) the approach you suggest seems
straightforward enough to me, but (as I said) looks more at
implementation than design. I get what you're saying, that this
approach would allow people to try out the idea in current Python,
with minimally-intrusive extra syntax needed. So maybe it'll encourage
people to try the idea out for their use cases.

On Fri, 10 Jul 2020 at 15:32, Ricky Teachey  wrote:
>  This is confusing to me because `my_mapping[1,2,3]` is already valid syntax 
> equivalent to `my_mapping.__setitem__(my_mapping, (1,2,3))`. Wouldn't a 
> change to translate the tuple into this K object lead to big problems for a 
> lot of existing code...?

This is much more the sort of question I think we should be exploring
at this stage. You can't change how currently-valid syntax is
interpreted, so you get a big change of behaviour between d[1,2] and
d[1, 2, k=3]. This is where a real-world use case is essential, to try
to explore whether that behaviour change is minor, or a showstopper.
There's no way to tell with made up examples. Thanks, Ricky, for
pointing this out explicitly.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/D5AW3KUPMZ2ZLAQODTCN3YOSXDZ3F3NG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

> Let's assume MyMap does have a __setitem__ method. Extending what you
> pointed out, the assignment
> >>> m[K(1, 2, 3, a=4, b=5)] = 'foobar'
> will then be equivalent to
> >>> MyMap.__setitem__(m, k, 'foobar')
> where
> >>> k = K(1, 2, 3, a=4, b=5)
>
>
 This is confusing to me because `my_mapping[1,2,3]` is already valid
syntax equivalent to `my_mapping.__setitem__(my_mapping, (1,2,3))`.
Wouldn't a change to translate the tuple into this K object lead to big
problems for a lot of existing code...?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YR5Z6YEXDPEJDZHZN5LO7TLSXG26TFMZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Hi Paul

I'm writing in response to one of your comments, all of which are useful.
Please forgive me, Paul, for not responding to the others in this message.

I wrote:

> Next, the implementation of such K-mappings. Here, K.set and K.get
> decorators will help. Something like (not tested):
>  class MyMap:
> @K.adjust_get
> def __getitem__(self, x1, x2, x3, a, b):
> pass
> where K.adjust_get interfaces between the K-object and the definition of
> __getitem__.


About this you wrote:

> I have no idea how this works, or what it even means.


With hindsight, I can see your difficulty. The original idea is to extend
the syntax so that
>>> m = MyMap()
>>> m[1, 2, 3, a=4, b=5] = 'foobar'
is allowed in some future version of Python.

Python's syntax already allows:
>>> m = MyMap()
>>> m[K(1, 2, 3, a=4, b=5)] = 'foobar'

My idea is to implement this, so that we can better explore the original
idea. (In particular, this would allow the idea to be tested in practice,
and perhaps acquire a community of users.)

For simplicity, assume MyMap = type(m). For
>>> m[K(1, 2, 3, a=4, b=5)] = 'foobar'
to execute successfully, MyMap must have a __setitem__ method.

Aside: Here's an example of failure.
>>> True[1] = 2
TypeError: 'bool' object does not support item assignment

Let's assume MyMap does have a __setitem__ method. Extending what you
pointed out, the assignment
>>> m[K(1, 2, 3, a=4, b=5)] = 'foobar'
will then be equivalent to
>>> MyMap.__setitem__(m, k, 'foobar')
where
>>> k = K(1, 2, 3, a=4, b=5)

The code I wrote, which wasn't clear enough, sketched an adapter between
two points of view (although for get rather than set). The basic idea is
that
>>> MyMap.__setitem__(k, 'foobar')
is equivalent to something like
>>> func(m, 'foobar', 1, 2, 3, a=4, b=5)
where func is defined in the class body of MyMap. Hence the adapter
decorator, which is to hide from the author of func many of the details of
the implementation of K.

I hope this helps.
-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YC4QXMOX3ZX2APGOEM7FUVEI3HRGTPBX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Paul Moore]

On Fri, 10 Jul 2020 at 11:54, Jonathan Fine  wrote:
> Let's proceed. We continue to use d = Dummy(). Given that
>>>> key = d[1, 2, 3, a=4, b=5]
> is allowed, what should we be able to say about the key. Clearly it should be 
> an instance of a class, and there should be a way of creating such an 
> instance without going via d. Let's call the class K (for key).
>
> It is natural to say that
>>>> key_1 = d[1, 2, 3, a=4, b=5]
>>>> key_2 = K(1, 2, 3, a=4, b=5)
> should define two K objects, that are identical.

Accepted.

> Once we have designed and implemented the class K, we can achieve many of the 
> benefits of this proposal within existing Python.

There's a flaw in your extrapolation here - see below.

> Here goes. First syntax.
> >>> value = d[K(1, 2, 3, a=4, b=5)]
> >>> d[K(1, 2, 3, a=4, b=5)] = value

Correct, but *only* if the type of d accepts objects of type K in
__getitem__. Your arguments are starting to get tricky to follow
because it's not clear if you intend "d" to still be an instance of
Dummy. If it is, of course, then d[K(1,2,3, a=4, b=5)] is a very
different value than d[1, 2, 3, a=4, b=5]. But maybe you'll patch this
up. You've not made an explicitly false statement yet.

> Next, the implementation of such K-mappings. Here, K.set and K.get decorators 
> will help. Something like (not tested):
> class MyMap:
> @K.adjust_get
> def __getitem__(self, x1, x2, x3, a, b):
> pass
> where K.adjust_get interfaces between the K-object and the definition of 
> __getitem__.

I have no idea how this works, or what it even means.

> Aside. Here, not tested, is an implementation of K.adjust_get.
> class K:
> @staticmethod
> def adjust_get(fn):
> def __getitem__(self, k):
> return fn(self, *k.argv, **k.kwargs)
> return __getitem__

I'm still confused. This code is (of course) valid, but I have no idea
how you think it would be used. So it doesn't really explain anything,
I'm still not clear what you are trying to say.

> This introduction of K allows collections to be created, that can be used 
> today with the syntax
> >>> value = d[K(1, 2, 3, a=4, b=5)]
> >>> d[K(1, 2, 3, a=4, b=5)] = value

Yes. Obviously. This is the same mechanism slice() uses.

> Further, if the K()-less syntax
> >>> value = d[1, 2, 3, a=4, b=5]
> >>> d[1, 2, 3, a=4, b=5] = value
> is added to Python, the existing K-collections could continue to be used, 
> with only a change to the decorator K.adjust_get (and also of course 
> K.adjust_set).

So you're suggesting that like slice, the d[1, 2, 3, a=4, b=5] syntax
could be translated into a K-object. Sure. But that doesn't do any
good unless classes implement a __getitem__ that recognises and
handles objects of type K. And you've not demonstrated how that would
happen, unless you were trying to describe that in your adjust_get
discussion.

So as far as I can see, all you've said is that a syntax like d[1, 2,
3, a=4, b=5] could be translated into indexing with a custom object.
Well yes, of course it could. And it probably would be. But that's
just describing an implementation approach, it doesn't really give any
argument for why we'd *want* to do this, or how (or if) we'd extend
existing (built in or user defined) types to understand this syntax.

Maybe I'm unusual in that I didn't see any difficulty in implementing
this proposal. But as a general principle, I don't tend to worry about
"how will this be implemented" when discussing new features. If it's
impossible to implement it, that will become obvious in due course :-)
What I'm interested in is always "why is this a good idea". And that's
where I've yet to see any really compelling use cases in this thread.

> I think that is enough for now. I hope it helps some of us, sometimes. And 
> does no harm elsewhere.

It's certainly a useful exposition of how you'd arrive at the
implementation mechanism that slice uses, and extend it for this
proposal (excluding that confusing adjust_get bit) which may well be
useful to some of the readers here. And yes, it certainly doesn't do
any harm clarifying it. But I'd recommend focusing on design, not
implementation, for now.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VZ5QQMCP7ZWY6L2YQFAILTKMBP2P4WU3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Hi

I will clarify my previous post, by making a stronger statement. Every
Python object is an instance of a class.

I wrote: Let's proceed. We continue to use d = Dummy(). Given that
   >>> key = d[1, 2, 3, a=4, b=5]
is allowed, what should we be able to say about the key. Clearly it should
be an instance of a class, and there should be a way of creating such an
instance without going via d. Let's call the class K (for key).

I'll now expand on this, as it's not clear to everyone that the key should
be an instance of a class.

So far I know, every Python object that is accessible to the Python user is
an instance of a class (or type). In fact, the type command tells us the
object's type.

>>> [type(obj) for obj in (None, True, (), {}, [], type)]
[, , , ,
, ]

Using d = Dummy(), where d's __getitem__(self, k) simply returns k, we
write using the proposed new syntax
>>> key = d[1, 2, 3, a=4, b=5]
and give this I suggest that
>>> key2 = K(1, 2, 3, a=4, b=5)
should produce a key2 that is equal to key. Here
>>> K = type(key)
should provide K.

I hope this helps clarify what I wrote. I also thank Alex Hall for
providing helpful clarification, from a different point of view.

-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MHQTDJ6LNA3IYFONOWUOLTZFKSO4SUNU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Alex Hall]

On Fri, Jul 10, 2020 at 2:33 PM Steven D'Aprano  wrote:

> On Fri, Jul 10, 2020 at 11:52:19AM +0100, Jonathan Fine wrote:
>
> > FUTURE
> > Let's proceed. We continue to use d = Dummy(). Given that
> >>>> key = d[1, 2, 3, a=4, b=5]
> > is allowed, what should we be able to say about the key. Clearly it
> should
> > be an instance of a class
>
> That's not clear at all.
>
> Subscripting is just syntactic sugar for a method call, `__getitem__`,
> and we already know how to associate positional and keyword arguments to
> method calls. No special magic class is needed.
> ...
> There would be difficulty with positional arguments, since they are
> already parsed as a tuple:
>
> py> {(1,2): 999}[1,2]
> 999
>
> so this may rule out adding multiple positional arguments to
> subscripting syntax. But I don't think there is any backwards
> compatibility issue with adding keyword arguments.
>

Jonathan already pointed out that positional arguments are passed as a
tuple:

Here goes:

>>> d = Dummy()
> >>> d[1]
> ((1,), {})
> >>> d[1, 2]
> (((1, 2),), {})
> >>> d[:]
> ((slice(None, None, None),), {})
> >>> d['a':'b']
> ((slice('a', 'b', None),), {})
> We see that the interpreter passes to __getitem__ a single positional
> argument (which is usually called the key). This is another difference
> between d[something] and f(something).


I believe he was saying it would be weird if `d[1, 2]` called
`d.__getitem__((1, 2))` but `d[1, 2, x=3]` called `d.__getitem__(1, 2,
x=3)`. More generally, if `__getitem__` always receives a single positional
argument now, it should probably stay that way for consistency.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EMJSDLKJ3FPXZNBMZJFZ4I5L34DKAVG3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Fri, Jul 10, 2020 at 08:23:09AM -0400, Ricky Teachey wrote:

> My mind instantly went to the idea of using this syntax as a way write
> single line mathematical function definitions:
> 
> f[x, y] = x + y

This won't work, because the right hand side will be evaluated first. 
The above is legal syntax today, and roughly equivalent to:

value = x + y
index = (x, y)
f.__setitem__(index, value)

except of course no such local variables value and index are created. 
But you can see why that cannot be used to define a function: the right 
hand side is evaluated, and all your `__setitem__` method will see is 
the value of x + y, not the fact that it is "x + y".



-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6W4WVUUKJ2RVJMLOJNXKJQ7EG7V22S3V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Joao S. O. Bueno]

I believe that one of the most popular Python domains that benefit from
"abusing" indexes is data analysis in the numpy/Pandas world.

I am not familiar enough with Pandas to make useful speculation on how
named indexes could enhance the usage of dataframes,   - maybe someone more
familiar can come up with suggestions on how this syntax could be useful?

On Fri, 10 Jul 2020 at 09:24, Ricky Teachey  wrote:

>
>
> On Fri, Jul 10, 2020, 6:54 AM Jonathan Fine  wrote:
>
>> Hi All
>>
>> SUMMARY
>> This is a longish post. It looks at the idea in general terms, and
>> outlines a way to get the desired semantics (by not syntax) with Python as
>> it is today. And this would be forward compatible with the new syntax, if
>> provided later.
>>
>
> This post was filled with inspiring ideas for me. Thank you.
>
>
>> PRESENT
>> I like the idea of allowing
>> >>> d[1, 2, 3, a=4, b=5]
>> and will explore it further.
>>
>> First, we can already write
>> >>> f(1, 2, 3, a=4, b=5)
>> but that only works for the get operation. For set the present behaviour
>> is
>> >>> f(1, 2, 3, a=4, b=5) = None
>> SyntaxError: can't assign to function call
>> and I see no good reason to change that.
>>
>> Going further, I'd say that allowing both
>> >>> d[something] = value
>> >>> value = d[something]
>> is essential to the difference between f(something) and d[something].
>> Both are expressions, but only one of them can be assigned to.
>>
>
>
>> Here goes. First syntax.
>> >>> value = d[K(1, 2, 3, a=4, b=5)]
>> >>> d[K(1, 2, 3, a=4, b=5)] = value
>>
>
>
> My mind instantly went to the idea of using this syntax as a way write
> single line mathematical function definitions:
>
> f[x, y] = x + y
>
> The example function doesn't even require the suggested K() object since
> no kwargs or defaults are used.
>
> Of course one would need to instantiate any these single line functions
> using a little bit of boilerplate up top. But this could be when you
> provide the docstring:
>
> f = MathFunction("Simple math function")
> f[x, y] = x + y
>
> And calling them would use a different bracket type (parentheses):
>
> >>> f(1,2)
> 3
>
> ...but these are surmountable hurdles.
>
>> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/QKDZ4Y6KBIVEFJ34ITLZHUT4IPE3QBBQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VUY3NPQ2FBAJ4L6LAMR7CU757IQRZFCN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Steven D'Aprano]

On Fri, Jul 10, 2020 at 11:52:19AM +0100, Jonathan Fine wrote:

> FUTURE
> Let's proceed. We continue to use d = Dummy(). Given that
>>>> key = d[1, 2, 3, a=4, b=5]
> is allowed, what should we be able to say about the key. Clearly it should
> be an instance of a class

That's not clear at all.

Subscripting is just syntactic sugar for a method call, `__getitem__`, 
and we already know how to associate positional and keyword arguments to 
method calls. No special magic class is needed.

Slicing today is somewhat special, using a built-in class to map the 
parts of the slice to a single parameter of the `__getitem__` method, 
but that's not how it worked originally. In Python 1 and Python 2, 
slicing with a single colon calls a dunder method `__getslice__` with 
two positional arguments:

seq[1:5] --> seq.__getslice__(1, 5)

not a single slice object argument. Only if `__getslice__` doesn't exist 
is a slice object passed to `__getitem__` instead.

So in principle, if we agreed that the syntax was desirable, we could 
map keyword arguments in slice syntax to ordinary parameters:

obj[3, spam=4]
# call obj.__getitem__(3, spam=4)


and likewise for setitem and delitem.

There would be difficulty with positional arguments, since they are 
already parsed as a tuple:

py> {(1,2): 999}[1,2]
999

so this may rule out adding multiple positional arguments to 
subscripting syntax. But I don't think there is any backwards 
compatibility issue with adding keyword arguments.


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/D5FWL4MVG4W46BUQMNVFXLJJJMNQ75L6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Ricky Teachey]

On Fri, Jul 10, 2020, 6:54 AM Jonathan Fine  wrote:

> Hi All
>
> SUMMARY
> This is a longish post. It looks at the idea in general terms, and
> outlines a way to get the desired semantics (by not syntax) with Python as
> it is today. And this would be forward compatible with the new syntax, if
> provided later.
>

This post was filled with inspiring ideas for me. Thank you.


> PRESENT
> I like the idea of allowing
> >>> d[1, 2, 3, a=4, b=5]
> and will explore it further.
>
> First, we can already write
> >>> f(1, 2, 3, a=4, b=5)
> but that only works for the get operation. For set the present behaviour is
> >>> f(1, 2, 3, a=4, b=5) = None
> SyntaxError: can't assign to function call
> and I see no good reason to change that.
>
> Going further, I'd say that allowing both
> >>> d[something] = value
> >>> value = d[something]
> is essential to the difference between f(something) and d[something]. Both
> are expressions, but only one of them can be assigned to.
>


> Here goes. First syntax.
> >>> value = d[K(1, 2, 3, a=4, b=5)]
> >>> d[K(1, 2, 3, a=4, b=5)] = value
>


My mind instantly went to the idea of using this syntax as a way write
single line mathematical function definitions:

f[x, y] = x + y

The example function doesn't even require the suggested K() object since no
kwargs or defaults are used.

Of course one would need to instantiate any these single line functions
using a little bit of boilerplate up top. But this could be when you
provide the docstring:

f = MathFunction("Simple math function")
f[x, y] = x + y

And calling them would use a different bracket type (parentheses):

>>> f(1,2)
3

...but these are surmountable hurdles.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QKDZ4Y6KBIVEFJ34ITLZHUT4IPE3QBBQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Jonathan Fine]

Hi All

SUMMARY
This is a longish post. It looks at the idea in general terms, and outlines
a way to get the desired semantics (by not syntax) with Python as it is
today. And this would be forward compatible with the new syntax, if
provided later.

PRESENT
I like the idea of allowing
>>> d[1, 2, 3, a=4, b=5]
and will explore it further.

First, we can already write
>>> f(1, 2, 3, a=4, b=5)
but that only works for the get operation. For set the present behaviour is
>>> f(1, 2, 3, a=4, b=5) = None
SyntaxError: can't assign to function call
and I see no good reason to change that.

Going further, I'd say that allowing both
>>> d[something] = value
>>> value = d[something]
is essential to the difference between f(something) and d[something]. Both
are expressions, but only one of them can be assigned to.

The proposal therefore is to retain this essential difference, and remove
what the proposer regards as an inessential difference. (And I also regard
this as an inessential difference.)

Now let's use this class
class Dummy:
def __getitem__(self, *argv, **kwargs):
return argv, kwargs
to peek into how
>>> value = d[something]
works.

Here goes:
>>> d = Dummy()
>>> d[1]
((1,), {})
>>> d[1, 2]
(((1, 2),), {})
>>> d[:]
((slice(None, None, None),), {})
>>> d['a':'b']
((slice('a', 'b', None),), {})

We see that the interpreter passes to __getitem__ a single positional
argument (which is usually called the key). This is another difference
between d[something] and f(something).

By the way, the slice syntax items[a:b:c] is so often convenient and
widespread, that it's been long enabled in Python. I see no good reason to
change that.

FUTURE
Let's proceed. We continue to use d = Dummy(). Given that
   >>> key = d[1, 2, 3, a=4, b=5]
is allowed, what should we be able to say about the key. Clearly it should
be an instance of a class, and there should be a way of creating such an
instance without going via d. Let's call the class K (for key).

It is natural to say that
   >>> key_1 = d[1, 2, 3, a=4, b=5]
   >>> key_2 = K(1, 2, 3, a=4, b=5)
should define two K objects, that are identical. Once we have designed and
implemented the class K, we can achieve many of the benefits of this
proposal within existing Python.

Here goes. First syntax.
>>> value = d[K(1, 2, 3, a=4, b=5)]
>>> d[K(1, 2, 3, a=4, b=5)] = value

Next, the implementation of such K-mappings. Here, K.set and K.get
decorators will help. Something like (not tested):
class MyMap:
@K.adjust_get
def __getitem__(self, x1, x2, x3, a, b):
pass
where K.adjust_get interfaces between the K-object and the definition of
__getitem__.

Aside. Here, not tested, is an implementation of K.adjust_get.
class K:
@staticmethod
def adjust_get(fn):
def __getitem__(self, k):
return fn(self, *k.argv, **k.kwargs)
return __getitem__

This introduction of K allows collections to be created, that can be used
today with the syntax
>>> value = d[K(1, 2, 3, a=4, b=5)]
>>> d[K(1, 2, 3, a=4, b=5)] = value

Further, if the K()-less syntax
>>> value = d[1, 2, 3, a=4, b=5]
>>> d[1, 2, 3, a=4, b=5] = value
is added to Python, the existing K-collections could continue to be used,
with only a change to the decorator K.adjust_get (and also of course
K.adjust_set).

I think that is enough for now. I hope it helps some of us, sometimes. And
does no harm elsewhere.
-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3LFNGTEYVCS4JOCSERYHS752YT6VQFDA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Neil Girdhar]

What the future of this?

I looked at type annotations in networkx recently (
https://github.com/networkx/networkx/pull/4014), and I wanted to keep 
things simple, so I proposed and implemented

Graph[NodeType]

However, I knew that they may ultimately want

Graph[NodeType, EdgeTypedDict, NodeTypedDict]

but no one is going to want to replace their calls with

Graph[str, dict[str, Any], dict[str, Any]]

That's what too noisy.

This proposal would allow you to have default parameters.  But what's the 
future looking like now?  Do we expect to have a type constructor?

class Graph:
def T(node_type, edge_type_dict=..., node_type_dict=...) -> a type 
annotation

And then

g: Graph.T(whatever) = Graph()

Does that work?

On Friday, July 10, 2020 at 4:20:58 AM UTC-4, Stefano Borini wrote:
>
> I am one of the authors of the PEP. My problem was to deal with 
> natural notation in quantum chemistry mostly. It had no technical 
> purpose, but I still think it would open some interesting options. 
> The PEP was rejected mostly because of lack of interest. 
>
> On Mon, 4 May 2020 at 00:07, Andras Tantos  > wrote: 
> > 
> > Hello! 
> > 
> > I'm not sure I'm addressing the right audience here, so please direct me 
> to the appropriate channel if that's the case... 
> > 
> > My name is Andras Tantos and I'm working on a Python library desscribing 
> HW designs. I came across this problem of __getitem__ and co. not 
> supporting kwargs. Apparently this extension was proposed and rejected as 
> PEP 472. 
> > 
> > Apart from my use-case, which is arguably a corner-case and not worth 
> modifying the language for, I believe there are two important use-cases 
> that are worth considering with the latest improvements in the language: 
> > 
> > 1. With the recent type-hint support, the feature could be made way more 
> descriptive if this PEP got implemented. 
> > 
> > For example, instead of doing the following: 
> > 
> > def func(in: Dict[str, int]) 
> > 
> > one could write: 
> > 
> > def func(in: Dict[key=str, value=int]) 
> > 
> > 2. It would also make 'generic classes' much cleaner to implement, 
> similar to the way type-hints look. Consider the following code: 
> > 
> > class _Generic(object): 
> > Specializations = [] 
> > @classmethod 
> > def __getitem__(cls, *args): 
> > name = f"Generic_{len(cls.Specializations)}" 
> > Specialized = type(name, (cls,), {"specials": tuple(args)}) 
> > cls.Specializations.append(Specialized) 
> > return Specialized 
> > def __init__(self, value = None): 
> > self.value = value 
> > def __str__(self): 
> > if hasattr(self, "specials"): 
> > return(f"[{type(self)} - " + ",".join(str(special) for special in 
> self.specials) + f"] - {self.value}") 
> > else: 
> > return(f"[{type(self)} - GENERIC" + f"] - {self.value}") 
> > Generic = _Generic() 
> > #g = Generic() - fails because of no specialization is given 
> > s1 = Generic[12]() 
> > s2 = Generic[42]("Hi!") 
> > print(s1) 
> > print(s2) 
> > 
> > Running this simple example results in: 
> > 
> > python3 -i python_test.py 
> > [ - 12] - None 
> > [ - 42] - Hi! 
> > 
> > You can see how the specialized parameters got passed as well as the 
> ones to '__init__'. Obviously, in real code the idea would be to filter 
> generic parameters and set up 'Specialized' with the right set of methods 
> and arguments. 
> > 
> > Now, without kwargs support for __getitem__, it's impossible to pass 
> named arguments to the specialization list, which greatly limits the 
> usability of this notation. 
> > 
> > I don't know how convincing these arguments and use-cases are for you, 
> but could you advise me about how to start the 'ball rolling' to drum-up 
> support for re-activating this PEP? 
> > 
> > Thanks again, 
> > Andras Tantos 
> > 
> > 
> > ___ 
> > Python-ideas mailing list -- python...@python.org  
> > To unsubscribe send an email to python-id...@python.org  
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> > Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/6OGAFDWCXT5QVV23OZWKBY4TXGZBVYZS/
>  
> > Code of Conduct: http://python.org/psf/codeofconduct/ 
>
>
>
> -- 
> Kind regards, 
>
> Stefano Borini 
> ___ 
> Python-ideas mailing list -- python...@python.org  
> To unsubscribe send an email to python-id...@python.org  
> https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/UHVZLOU57HS2HGH6E4JCDW6ETAIORKG7/
>  
> Code of Conduct: http://python.org/psf/codeofconduct/ 
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.o

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Stefano Borini]

I am one of the authors of the PEP. My problem was to deal with
natural notation in quantum chemistry mostly. It had no technical
purpose, but I still think it would open some interesting options.
The PEP was rejected mostly because of lack of interest.

On Mon, 4 May 2020 at 00:07, Andras Tantos  wrote:
>
> Hello!
>
> I'm not sure I'm addressing the right audience here, so please direct me to 
> the appropriate channel if that's the case...
>
> My name is Andras Tantos and I'm working on a Python library desscribing HW 
> designs. I came across this problem of __getitem__ and co. not supporting 
> kwargs. Apparently this extension was proposed and rejected as PEP 472.
>
> Apart from my use-case, which is arguably a corner-case and not worth 
> modifying the language for, I believe there are two important use-cases that 
> are worth considering with the latest improvements in the language:
>
> 1. With the recent type-hint support, the feature could be made way more 
> descriptive if this PEP got implemented.
>
> For example, instead of doing the following:
>
> def func(in: Dict[str, int])
>
> one could write:
>
> def func(in: Dict[key=str, value=int])
>
> 2. It would also make 'generic classes' much cleaner to implement, similar to 
> the way type-hints look. Consider the following code:
>
> class _Generic(object):
> Specializations = []
> @classmethod
> def __getitem__(cls, *args):
> name = f"Generic_{len(cls.Specializations)}"
> Specialized = type(name, (cls,), {"specials": tuple(args)})
> cls.Specializations.append(Specialized)
> return Specialized
> def __init__(self, value = None):
> self.value = value
> def __str__(self):
> if hasattr(self, "specials"):
> return(f"[{type(self)} - " + ",".join(str(special) for special in 
> self.specials) + f"] - {self.value}")
> else:
> return(f"[{type(self)} - GENERIC" + f"] - {self.value}")
> Generic = _Generic()
> #g = Generic() - fails because of no specialization is given
> s1 = Generic[12]()
> s2 = Generic[42]("Hi!")
> print(s1)
> print(s2)
>
> Running this simple example results in:
>
> python3 -i python_test.py
> [ - 12] - None
> [ - 42] - Hi!
>
> You can see how the specialized parameters got passed as well as the ones to 
> '__init__'. Obviously, in real code the idea would be to filter generic 
> parameters and set up 'Specialized' with the right set of methods and 
> arguments.
>
> Now, without kwargs support for __getitem__, it's impossible to pass named 
> arguments to the specialization list, which greatly limits the usability of 
> this notation.
>
> I don't know how convincing these arguments and use-cases are for you, but 
> could you advise me about how to start the 'ball rolling' to drum-up support 
> for re-activating this PEP?
>
> Thanks again,
> Andras Tantos
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/6OGAFDWCXT5QVV23OZWKBY4TXGZBVYZS/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UHVZLOU57HS2HGH6E4JCDW6ETAIORKG7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Guido van Rossum]

On Mon, May 4, 2020 at 10:03 Antoine Pitrou  wrote:

> Of course, that's why I originally suggested that `Dict[...]` should be
> spelled `Dict(...)` instead.


...and we’d be hosed now with no path from Dict to dict.

> --
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AWAA4MI2YDQLDGVCFPMD2CULDEGSKWPD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[Antoine Pitrou]

On Sun, 3 May 2020 14:58:41 -0700
Andras Tantos  wrote:
> 
> 1. With the recent type-hint support, the feature could be made way more 
> descriptive if this PEP got implemented.
> 
> For example, instead of doing the following:
> 
>      def func(in: Dict[str, int])
> 
> one could write:
> 
>      def func(in: Dict[key=str, value=int])

Of course, that's why I originally suggested that `Dict[...]` should be
spelled `Dict(...)` instead.

Regards

Antoine.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WB66IID6GTMG4CBD4US7A56HYDJOBZNN/
Code of Conduct: http://python.org/psf/codeofconduct/