[issue24379] Add operator.subscript as a convenience for creating slices

[STINNER Victor]

STINNER Victor  added the comment:

> Raymond, Tal and Guido -- do any of you work routinely with multi-dimensional 
> arrays?

Hi Stephan Hoyer,

IMHO a *closed* issue is not the most appropriate place to request Guido to 
change his mind. You may get more traction on python-ideas where you may find 
more supporters who need the operator.

The question is not only if the operator makes sense (obviously, it does, for 
you), but if it "belongs" to the standard library. Is it popular enough? Is it 
consistent with other functions of the operator module? etc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Stephan Hoyer]

Stephan Hoyer  added the comment:

Raymond, Tal and Guido -- do any of you work routinely with multi-dimensional 
arrays?

In my experience as someone who uses Python everyday for numerical computing 
(and has been doing so for many years), the need for an operator like this 
comes up with some regularity. With multi-dimensional indexing, this allows 
lets you cut down quite a bit on boilerplate, e.g., compare

  subscript[:, 0, ::-1] vs (slice(None), 0, slice(None, None, -1))

It's absolutely true that subscript is an easy four line recipe, and indeed a 
form of this recipe is already included in both NumPy and pandas. But I think 
this is a case where the lack of a common idiom is actively harmful. I do 
multi-dimensional indexing in using at least four different libraries (pandas, 
xarray, numpy and tensorflow), and it feels wrong to use a utility from 
numpy/pandas for other projects. I could write my own version of 
operator.subscript in each project (and yes, I've done so before), but for any 
individual use case it's less hassle to write things out the long way. 

In practice, the preferred way to write such long expressions seems to be to 
redundantly repeat indexing operations, e.g.,

  x[:, 0, ::-1]
  y[:, 0, ::-1]

rather than

  index = subscript[:, 0, ::-1]
  x[index]
  y[index]

This is definitely non-ideal from a readability perspective. It's no longer 
immediately clear that these arrays are being indexed in the same way, and any 
changes would need to be applied twice.

--
nosy: +Stephan Hoyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> rejected
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Guido van Rossum]

Guido van Rossum  added the comment:

Let's close it. Just because someone spent a lot of effort on a patch we don't 
have to accept it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Tal Einat]

Tal Einat  added the comment:

So 3.8 then, or should this be closed?

FWIW I'm -1.  IMO this should be a code recipe somewhere, no need for it to be 
in the stdlib.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Guido van Rossum]

Guido van Rossum added the comment:

Actually I'm with Raymond -- I'm also not sure or mildly against (-0).
Given how easy it is to do without and how cumbersome using the operator
module is I don't see a great benefit. (IMO the operator module should be
the measure of *last* resort -- it is not to become part of anybody's
toolkit for common operators. But I seem to be a minority opinion here.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Maybe then it makes sense to apply this to 3.7? It looks like most people are 
in favour of this (also on python-ideas), only Raymond is not sure/mildly 
against.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ned Deily]

Ned Deily added the comment:

I don't think this is appropriate for 3.6 now.  We're a little more than 4 
weeks from the final release - way past the feature code cut-off - and, from 
the comments here, there is no longer a total consensus that this feature 
should go back in at all after being pulled and largely forgotten about for a 
year.

--
assignee: ned.deily -> 
priority: release blocker -> normal
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

operator_subscript_norefleak_v2.patch LGTM. Waiting for Ned's decision about 
3.6.

--
assignee:  -> ned.deily
priority: normal -> release blocker
stage: patch review -> commit review
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Serhiy, thank you for review! Here is a corrected patch.

--
Added file: 
http://bugs.python.org/file45473/operator_subscript_norefleak_v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

The patch is small and it was initially applied before 3.6b1, I think we should 
ask Ned (added to nosy) if this is OK to apply the fixed version?

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Guido van Rossum]

Guido van Rossum added the comment:

So we now have the refleak fixed. Can we apply the patch? Or is it too late for 
3.6?

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Here is a new patch that does not cause refleaks, I just replaced empty 
__slots__ with a __setattr__: it looks like __slots__ are not needed here to 
save memory (there is only a singleton instance), they were used just to create 
an immutable object.

--
Added file: http://bugs.python.org/file45471/operator_subscript_norefleak.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Steven D'Aprano]

Steven D'Aprano added the comment:

On Sat, Nov 12, 2016 at 08:23:45AM +, Raymond Hettinger wrote:
> I can't even dream up any scenarios where 
> ``operator.subscript[3:7:2]`` would be better than 
> ``slice(3, 7, 2)``.

For that specific example, I completely agree. But I think that in 
general ``slice[::-1]`` is better than either the current alternative 
``slice(None, None, -1)`` or the proposed ``operator.subscript[::-1]``.

And there's no comparison once you get to multi-dimensional slices:

s = slice[::-1, 1:, 1::2]
spam[s] + eggs[s]

versus:

s = slice(slice(None, None, -1), slice(1, None), slice(1, None, 2))
spam[s] + eggs[s]

Even simple examples look nice in slice syntax:

slice[3:7:2]

I had completely forgotten about this feature request, but 
coincidentally re-suggested it on the Python-Ideas mailing 
list earlier today:

https://mail.python.org/pipermail/python-ideas/2016-November/043630.html

The downside is that beginners who are still learning Python's 
syntax may try writing:

slice(::-1)

by mistake. Nevertheless, it seems to me that the advantage to more 
experienced developers to be able to read and write slice objects using 
slice format outweighs the temporary confusion to beginners. (I would 
expect that outside of the Numpy community, few beginners use the 
slice() object directly, so this would not often affect them.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

I think it would be more useful for multidimensional slicing:
>>> subscript[::-1, ..., ::-1]
(slice(None, None, -1), Ellipsis, slice(None, None, -1))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Raymond Hettinger]

Raymond Hettinger added the comment:

When I first looked at this a year ago, I thought it seemed like a reasonable 
idea and might be useful.  Since that time, I've haven't encountered any 
situations whether this would have improved my or someone else's code.  And 
now, I can't even dream up any scenarios where ``operator.subscript[3:7:2]`` 
would be better than ``slice(3, 7, 2)``.   Accordingly, I think we should 
re-evaluate whether there is any real benefit to be had by adding this to the 
standard library.  Perhaps, we're better off without it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Josh Rosenberg]

Josh Rosenberg added the comment:

#28651 opened for the general problem with empty __slots__ and gc failures.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Yury Selivanov]

Changes by Yury Selivanov :


--
Removed message: http://bugs.python.org/msg280422

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Yury Selivanov]

Yury Selivanov added the comment:

Even more reduced test case:

def test_refleak(self):
T = typing.TypeVar('T')
typing.Generic[T]

--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Raymond Hettinger]

Raymond Hettinger added the comment:

> We should open a separate issue for leaks in objects with empty __slots__

The leak should be fixed first rather than introducing stub fields hack.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

It looks like namedtuple suffers the same issue with empty __slots__:

test_collections leaked [0, 0, 2, 0] references, sum=2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Raymond, adding a stub element to __slots__ fixes a leak. Is this enough to 
push the patch again?

We should open a separate issue for leaks in objects with empty __slots__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Ivan Levkivskyi]

Changes by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Emanuel Barry]

Changes by Emanuel Barry :


--
nosy: +ebarry
stage: resolved -> patch review
title: operator.subscript -> Add operator.subscript as a convenience for 
creating slices

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com