date:20171127

Currently during assignment, when target list is a comma-separated list of
targets (*without "starred" target*) the rule is that the object (rhs) must
be an iterable with the same number of items as there are targets in the
target list. That is, no check is performed on the number of targets
present, and if something goes wrong the  ValueError is raised. To show
this on simple example:

>>> from itertools import count, islice
>>> it = count()
>>> x, y = it
>>> it
count(3)

Here the count was advanced two times but assignment did not happen. I
found that in some cases it is too much restricting that rhs must have the
same number of items as targets. It is proposed that if the rhs is a
generator or an iterator (better some object that yields values on demand),
the assignmenet should be lazy and dependent on the number of targets. I
find this feature to be very convenient for interactive use, while it
remains readable, expected, and expressed in a more compact code.

There are some Pros:
1. No overhead
2. Readable and not so verbose code
3. Optimized case for x,y,*z = iterator
4. Clear way to assign values partially from infinite generators.

Cons:
1. A special case of how assignment works
2. As with any implicit behavior, hard-to-find bugs

There several cases with "undefined" behavior:
1. Because the items are assigned, from left to right to the corresponding
targets, should rhs see side effects during assignment or not?
2. Should this work only for generators or for any iterators?
3. Is it Pythonic to distinguish what is on the rhs during assignment, or
it contradicts with duck typing (goose typing)?

In many cases it is possible to do this right now, but in too verbose way:

>>> x, y = islice(gen(), 2)

But it seems for me that:

>>> x, y = gen()

Looks the same, easy to follow and not so verbose. Another case, which is a
pitfall, but will be possible:

>>> x, y = iter([0,1,2,3,4]) # Now it is Ok
>>> x
1
>>> y
2

But:

>>> x, y = [0,1,2,3,4]   # raises ValueError

Any thoughts?

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 8:17 PM, Kirill Balunov  wrote:
> In many cases it is possible to do this right now, but in too verbose way:
>
 x, y = islice(gen(), 2)
>
> But it seems for me that:
>
 x, y = gen()
>
> Looks the same, easy to follow and not so verbose.

This, AIUI, is the nub of the proposal. I don't like the proposed
solution, but if there's some alternative way to say "and then ignore
the rest", that would be much safer.

You said:

> 3. Optimized case for x,y,*z = iterator

It has to be semantically different from that, though. Consider these
two generators:

def gen1():
for _ in range(5):
print("Yielding!")
yield "spam"
yield "ham"

def gen2():
yield 1
yield 2
yield 3
while True:
yield 4

If you use "x, y, *z = gen1()", you'll trigger all the prints and
completely consume the generator. With gen2(), you'll get an infinite
loop. Both of those are semantically different from the islice
behaviour, which would consume only that part that you're looking for.

What would be far safer is a special syntax that makes it clear that
you are not assigning those extra values anywhere. Something along
these lines has been proposed a number of times, and I think it's
probably about time a PEP was written up, if only to get rejected.
Here are a few syntaxes that I believe have been proposed at various
times:

x, y = islice(iter, 2) # status quo
x, y = iter # your proposal
x, y, = iter # omit last destination
x, y, * = iter # unpack into nothing
x, y, ... = iter # assigning to Ellipsis
x, y, *... = iter # as above but clearly sequencing

And there are a few others too. Every one of them has its downsides,
none is perfect. (Personally, I think one of the Ellipsis options is
likely the best, or perhaps the least-bad.) Want to spearhead the PEP?

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 12:40 GMT+03:00 Chris Angelico :

> On Mon, Nov 27, 2017 at 8:17 PM, Kirill Balunov 
> wrote:
> > In many cases it is possible to do this right now, but in too verbose
> way:
> >
>  x, y = islice(gen(), 2)
> >
> > But it seems for me that:
> >
>  x, y = gen()
> >
> > Looks the same, easy to follow and not so verbose.
>
> This, AIUI, is the nub of the proposal.

Yes, it is. For me, x,y = gen() underlines the lazy nature of generators.

> I don't like the proposed
> solution, but if there's some alternative way to say "and then ignore
> the rest", that would be much safer.
>

Of course, everyone has a subjective assessment of what is good and what is
bad. But here I am in something agree with you that at the present time some
alternative way would be much safer. But if started from scratch, it seems
to me natural to emphasize the nature of generators and iterators.

> You said:
>
> > 3. Optimized case for x,y,*z = iterator
>
> It has to be semantically different from that, though. Consider these
> two generators:
>
> def gen1():
> for _ in range(5):
> print("Yielding!")
> yield "spam"
> yield "ham"
>
> def gen2():
> yield 1
> yield 2
> yield 3
> while True:
> yield 4
>
> If you use "x, y, *z = gen1()", you'll trigger all the prints and
> completely consume the generator. With gen2(), you'll get an infinite
> loop. Both of those are semantically different from the islice
> behaviour, which would consume only that part that you're looking for.

The idea is not to consume generator completely, but to get enough values
to bind to x and y. It should be equivalent to islice(iter, 2), and perceiveв
as "bind x, y, where you don't need values for z at all".

x, y = islice(iter, 2) # status quo
> x, y = iter # your proposal
>

As I wrote above, I would like to see them equivalent

x, y, = iter # omit last destination
>

I don't like this, it is hard to notice this nuance. It is valid to have
trailing comma (and I find it to be a brilliant feature to have (x,y)
equivalent to (x,y,)).
This reminds me of  `  in Python2, although I never used it and start my
journey with Python3.

> x, y, * = iter # unpack into nothing
>

Maybe, I like it.

> x, y, ... = iter # assigning to Ellipsis
> x, y, *... = iter # as above but clearly sequencing
>

Yes, it is nice to see Ellipsis as zero-length deques (or throw away
container), it can be used also in the middle of list of targets. What I
don't like about last three examples, that they imply by their form to be
some kind of iteration which must completely consume the generator, which
is opposite to the proposed idea. Moreover, this will not work with
infinite generators, falling into infinite loop.

> And there are a few others too. Every one of them has its downsides,
> none is perfect. (Personally, I think one of the Ellipsis options is
> likely the best, or perhaps the least-bad.) Want to spearhead the PEP?
>

To be honest, I'm quite new to the entire ecosystem of Python. Therefore,
someone can perceive this as an ordinary attempt by a novice to change
everything that is bad in his opinion. In addition, it seems to me that
these changes, if approved, can not be made earlier than Python3.8, so I
would like to get some feedback first. Nevertheless these words should not
be taken as a renouncement, I would be happy and very interested in writing
PEP and implementing it. But not to get stuck I need some support and
guidelines from interested dev.

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 9:45 PM, Kirill Balunov  wrote:
> 2017-11-27 12:40 GMT+03:00 Chris Angelico :
>> If you use "x, y, *z = gen1()", you'll trigger all the prints and
>> completely consume the generator. With gen2(), you'll get an infinite
>> loop. Both of those are semantically different from the islice
>> behaviour, which would consume only that part that you're looking for.
>
>
> The idea is not to consume generator completely, but to get enough values to
> bind to x and y. It should be equivalent to islice(iter, 2), and perceiveв
> as "bind x, y, where you don't need values for z at all".

In terms of language proposals, you can't just say "don't need values
for"; the semantics have to be EITHER "consume and discard" OR "don't
consume". We already have a perfectly good way of spelling "consume
and discard":

x, y, _ = iter

following the convention that a single underscore means "don't really
care" (eg "for _ in range(3)"). So this proposal is about not
consuming an iterator.

>> x, y, * = iter # unpack into nothing
>
> Maybe, I like it.
>
>> x, y, ... = iter # assigning to Ellipsis
>> x, y, *... = iter # as above but clearly sequencing
>
>
> Yes, it is nice to see Ellipsis as zero-length deques (or throw away
> container), it can be used also in the middle of list of targets. What I
> don't like about last three examples, that they imply by their form to be
> some kind of iteration which must completely consume the generator, which is
> opposite to the proposed idea. Moreover, this will not work with infinite
> generators, falling into infinite loop.

Since this has to be about non-consumption of the generator/iterator,
Ellipsis cannot be a zero-length deque. Thus this syntax would have to
be restricted to the *last* entry, and it then means "don't check for
more elements".

The assignment "x, y = it" is roughly equivalent to:

try:
_iter = iter(it)
x = next(_iter)
y = next(_iter)
except StopIteration:
raise ValueError
else:
try:
next(_iter)
except StopIteration:
pass # good, we got the right number of elements
else:
raise ValueError

The proposed semantics, if I understand you correctly, are:

try:
_iter = iter(it)
x = next(_iter)
y = next(_iter)
except StopIteration:
raise ValueError
# no "else" clause, we're done here

And I think this would be a good thing to be able to spell
conveniently. The only questions are: what spelling is the best, and
is it sufficiently useful to justify the syntax?

>> And there are a few others too. Every one of them has its downsides,
>> none is perfect. (Personally, I think one of the Ellipsis options is
>> likely the best, or perhaps the least-bad.) Want to spearhead the PEP?
>
> To be honest, I'm quite new to the entire ecosystem of Python. Therefore,
> someone can perceive this as an ordinary attempt by a novice to change
> everything that is bad in his opinion. In addition, it seems to me that
> these changes, if approved, can not be made earlier than Python3.8, so I
> would like to get some feedback first. Nevertheless these words should not
> be taken as a renouncement, I would be happy and very interested in writing
> PEP and implementing it. But not to get stuck I need some support and
> guidelines from interested dev.

Yes; now that we're into alphas for Python 3.7, it's not going to land
until 3.8. That's fine.

The PEP process is basically a way of gathering all the arguments
for-and-against into a single, coherent document, rather than having
them scattered all over the python-ideas email archive. The PEP author
has the final say on what is the thrust of the proposal, and then
Guido (or his delegate) will decide to accept or reject the PEP. If
it's accepted, the change will be made; if it's not, the PEP remains
in the repository as a permanent record of the proposal. That way, the
*next* person to come along and suggest something can pick up from
where this discussion left off, rather than starting fresh.

Start by perusing PEP 1, and the template in PEP 12:

https://www.python.org/dev/peps/pep-0001/
https://www.python.org/dev/peps/pep-0012/

The PEP editors (myself included) are here to help you; don't hesitate
to reach out with questions.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

> In terms of language proposals, you can't just say "don't need values
> for"; the semantics have to be EITHER "consume and discard" OR "don't
> consume". We already have a perfectly good way of spelling "consume
> and discard":
>
> x, y, _ = iter
>

You mean ( x, y, *_ = iter ) ?

Since this has to be about non-consumption of the generator/iterator,
> Ellipsis cannot be a zero-length deque. Thus this syntax would have to
> be restricted to the *last* entry, and it then means "don't check for
> more elements".
>

Yes, you are right to the *last* entry. (*last* depends on proposed syntax
(spelling)).


> The proposed semantics, if I understand you correctly, are:
>
> try:
> _iter = iter(it)
> x = next(_iter)
> y = next(_iter)
> except StopIteration:
> raise ValueError
> # no "else" clause, we're done here
>

Yes, "roughly" this semantics is proposed, with some assumptions on _iter =
iter(it).
As I can see at the moment, these cases should behave differently:

>>> x, y = [1,2,3,4] # must raise ValueError
>>> x, y = iter([1,2,3,4])  # should work

But at the same time, it violates current situation. So maybe, as you have
said we need special syntax. I will think about it.


> Start by perusing PEP 1, and the template in PEP 12:
>
> https://www.python.org/dev/peps/pep-0001/
> https://www.python.org/dev/peps/pep-0012/
>
> The PEP editors (myself included) are here to help you; don't hesitate
> to reach out with questions.
>

Thank you!

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On 27 November 2017 at 12:31, Kirill Balunov  wrote:
> As I can see at the moment, these cases should behave differently:
>
 x, y = [1,2,3,4] # must raise ValueError
 x, y = iter([1,2,3,4])  # should work
>
> But at the same time, it violates current situation. So maybe, as you have
> said we need special syntax. I will think about it.

I would find this confusing. Consider where you don't have literals:

def f(vals):
x, y = vals

data = [1,2,3,4]

f(data)
data = iter(data)
f(data)

Having the two calls behave differently would be a recipe for errors
as someone refactors the calling code.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 15:39 GMT+03:00 Paul Moore :

> On 27 November 2017 at 12:31, Kirill Balunov 
> wrote:
> > As I can see at the moment, these cases should behave differently:
> >
>  x, y = [1,2,3,4] # must raise ValueError
>  x, y = iter([1,2,3,4])  # should work
> >
> > But at the same time, it violates current situation. So maybe, as you
> have
> > said we need special syntax. I will think about it.
>
> I would find this confusing. Consider where you don't have literals:
>
> def f(vals):
> x, y = vals
>
> data = [1,2,3,4]
>
> f(data)
> data = iter(data)
> f(data)
>
> Having the two calls behave differently would be a recipe for errors
> as someone refactors the calling code.
>

I can not completely disagree with you, but we all adults here. My first
proposal was about generators only, but they are very similar to iterators
in their behavior. Whatever it was with this syntax, there will be no
difference:

def f(vals):
x, y = vals

data = [1,2,3,4]

f(data)
data = (i for i in data)
f(data)

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 11:31 PM, Kirill Balunov
 wrote:
>
>> In terms of language proposals, you can't just say "don't need values
>> for"; the semantics have to be EITHER "consume and discard" OR "don't
>> consume". We already have a perfectly good way of spelling "consume
>> and discard":
>>
>> x, y, _ = iter
>
>
> You mean ( x, y, *_ = iter ) ?

Uhh, yeah, that's what I meant. Sorry. Anyhow, point is, there IS a
syntax for that, so we don't need another.

>> The proposed semantics, if I understand you correctly, are:
>>
>> try:
>> _iter = iter(it)
>> x = next(_iter)
>> y = next(_iter)
>> except StopIteration:
>> raise ValueError
>> # no "else" clause, we're done here
>
>
> Yes, "roughly" this semantics is proposed, with some assumptions on _iter =
> iter(it).
> As I can see at the moment, these cases should behave differently:
>
 x, y = [1,2,3,4] # must raise ValueError
 x, y = iter([1,2,3,4])  # should work
>
> But at the same time, it violates current situation. So maybe, as you have
> said we need special syntax. I will think about it.

That's the part I disagree with, but if you're the PEP author, you can
make the recommendation be anything you like. However, one very strong
piece of advice: it's easier to get a proposal accepted if the
backward compatibility section simply says "the proposed notation is a
SyntaxError in current versions of Python". Changing the semantics of
currently legal code requires that you demonstrate that the current
semantics are, in some way, faulty or buggy.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread Daniel Moisset

On 27 November 2017 at 06:40, Chris Angelico  wrote:

> Here are a few syntaxes that I believe have been proposed at various
> times:
>
> x, y = islice(iter, 2) # status quo
> x, y = iter # your proposal
> x, y, = iter # omit last destination
>

Just to clear the list, this one (trailing comma) would be
ambiguous/backward incompatible for the 1 variable case:

x, = iter

which is a relatively common idiom and is expected to raise an error if the
iterator has trailing elements.

-- 
Daniel F. Moisset - UK Country Manager - Machinalis Limited
www.machinalis.co.uk 
Skype: @dmoisset T: + 44 7398 827139

1 Fore St, London, EC2Y 9DT

Machinalis Limited is a company registered in England and Wales. Registered
number: 10574987.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 12:17:31PM +0300, Kirill Balunov wrote:
> Currently during assignment, when target list is a comma-separated list of
> targets (*without "starred" target*) the rule is that the object (rhs) must
> be an iterable with the same number of items as there are targets in the
> target list. That is, no check is performed on the number of targets
> present, and if something goes wrong the  ValueError is raised.

That's a misleading description: ValueError is raised when the number of 
targets is different from the number of items. I consider that to be 
performing a check on the number of targets.

> To show this on simple example:
> 
> >>> from itertools import count, islice
> >>> it = count()
> >>> x, y = it
> >>> it
> count(3)

For everyone else who was confused by this, as I was, that's not 
actually a copy and paste from the REPL. There should be a ValueError 
raised after the x, y assignment. As given, it is confusing because it 
looks like the assignment succeeded, when in fact it didn't.

> Here the count was advanced two times but assignment did not happen.

Correct, because there was an exception raised.

> I found that in some cases it is too much restricting that rhs must 
> have the same number of items as targets. It is proposed that if the 
> rhs is a generator or an iterator (better some object that yields 
> values on demand), the assignmenet should be lazy and dependent on the 
> number of targets.

I think that's problematic. How do you know what objects that yields 
values on demand? Not all lazy iterables are iterators: there are also 
lazy sequences like range.

But even if we decide on a simple rule like "iterator unpacking depends 
on the number of targets, all other iterables don't",  I think that will 
be a bug magnet. It will mean that you can't rely on this special 
behaviour unless you surround each call with a type check:

if isinstance(it, collections.abc.Iterator):
# special case for iterators
x, y = it
else:
# sequences keep the old behaviour
x, y = it[:2]

> I find this feature to be very convenient for 
> interactive use,

There are many things which would be convenient for interactive use that 
are a bad idea outside of the interactive environment. Errors which pass 
silently are one of them. Unpacking a sequence of 3 items into 2 
assignment targets should be an error, unless you explicitly limit it to 
only two items.

Sure, sometimes it would be convenient to unpack just two items out of 
some arbitrarily large iterator just be writing `x, y = it`. But 
other times that would be an error, even in the interactive interpreter.

I don't want Python trying to *guess* whether I want to unpack the 
entire iteratable or just two items. Whatever tiny convenience there is 
from when Python guesses correctly will be outweighed by the nuisance 
value of when it guesses wrongly.

> while it remains readable, expected, and expressed in a more compact 
> code.

I don't think it is expected behaviour. It is different from the current 
behaviour, so it will be surprising to everyone used to the current 
behaviour, annoying to those who like the current behaviour, and a 
general inconvenience to those writing code that runs under multiple 
versions of Python.

Personally, I would not expect this suggested behaviour. I would be very 
surprised, and annoyed, if a simple instruction like:

x, y = some_iterable

behaved differently for iterators and sequences.

> There are some Pros:
> 1. No overhead

No overhead compared to what?

> 2. Readable and not so verbose code
> 3. Optimized case for x,y,*z = iterator

The semantics of that are already set: the first two items are assigned 
to x and y, with all subsequent items assigned to z as a list. How will 
this change optimize this case? It still needs to run through the 
iterator to generate the list.

> 4. Clear way to assign values partially from infinite generators.

It isn't clear at all. If I have a non-generator lazy sequence like:

# Toy example
class EvenNumbers:
def __getitem__(self, i):
return 2*i

it = EvenNumbers()  # A lazy, infinite sequence

then `x, y = it` will keep the current behaviour and raise an exception 
(since it isn't an iterator), but `x, y = iter(it)` will use the new 
behaviour.

So in general, when I'm reading code and I see:

x, y = some_iterable

I have very little idea of which behaviour will apply. Will it be the 
special iterator behaviour that stops at two items, or the current 
sequence behaviour that raises if there are more than two items?

> Cons:
> 1. A special case of how assignment works
> 2. As with any implicit behavior, hard-to-find bugs

Right. Hard-to-find bugs beats any amount of convenience in the 
interactive interpreter. To use an analogy:

"Sure, sometimes my car suddenly shifts into reverse while I'm driving 
at 60 kph, sometimes the engine falls out when I go around the corner, 
and occasionally the brakes c

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 12:44 AM, Daniel Moisset
 wrote:
> On 27 November 2017 at 06:40, Chris Angelico  wrote:
>>
>> Here are a few syntaxes that I believe have been proposed at various
>> times:
>>
>> x, y = islice(iter, 2) # status quo
>> x, y = iter # your proposal
>> x, y, = iter # omit last destination
>
>
> Just to clear the list, this one (trailing comma) would be
> ambiguous/backward incompatible for the 1 variable case:
>
> x, = iter
>
> which is a relatively common idiom and is expected to raise an error if the
> iterator has trailing elements.

Correct. For that and other reasons, I am not in favour of either of
these two proposals. And the status quo is noisy and has duplicated
information (you have to match the ", 2" to the number of assignment
targets). I would support any syntax that (a) is currently illegal,
(b) reads reasonably well, and (c) can't be TOO easily confused with
something else. Assigning to Ellipsis (with or without a star) is my
current preferred, but I'd happily support others that do the same
job.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 03:31:38PM +0300, Kirill Balunov wrote:

> As I can see at the moment, these cases should behave differently:
> 
> >>> x, y = [1,2,3,4] # must raise ValueError
> >>> x, y = iter([1,2,3,4])  # should work

I *completely disagree* that they should behave differently. That would 
be a radical change to the current equivalency between iterators and 
other iterables.

Of course iterators support next() (and a few other things), while 
iterables (sequences and others) support slicing, __getitem__, and so 
forth. But when it comes to iteration, they behave exactly the same in 
all ways that I can think of:

  for x in iterable: ...

  list(iterable)

  iter(iterable)

  func(*iterable)

and most importantly for this discussion, iterable unpacking:

  a, b, c = *iterable

They all work the same, regardless of whether `iterable` is an iterator, 
a generator, a list, a tuple, a range object, a custom lazy sequence.

Sure, there are a few differences: iterators generally cannot be 
restarted or rewound, while lazy sequences might be, and eager sequences 
like lists can be. You can't peek into an arbitrary iterator without 
consuming the value. But as far as iteration itself goes, they are all 
the same.

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 12:55 AM, Steven D'Aprano  wrote:
> But even if we decide on a simple rule like "iterator unpacking depends
> on the number of targets, all other iterables don't",  I think that will
> be a bug magnet. It will mean that you can't rely on this special
> behaviour unless you surround each call with a type check:
>
>
> if isinstance(it, collections.abc.Iterator):
> # special case for iterators
> x, y = it
> else:
> # sequences keep the old behaviour
> x, y = it[:2]

Nah, far easier:

x, y = iter(it)

since that'll be a no-op in the first case, and trigger new behaviour
in the second. However, I don't like this behaviour-switch. I'd much
rather have actual syntax.

> I don't want Python trying to *guess* whether I want to unpack the
> entire iteratable or just two items. Whatever tiny convenience there is
> from when Python guesses correctly will be outweighed by the nuisance
> value of when it guesses wrongly.

Exactly.

>> There are some Pros:
>> 1. No overhead
>
> No overhead compared to what?

I think the point here (correct me if I'm wrong?) is that it takes
work to probe the iterator to see if there's a third item, so grabbing
just the first two items is simply *doing less work*. It's not doing
MORE work (constructing an islice object, pumping it, then discarding
it) - it's simply skipping the check that it would otherwise do.

>> 2. Readable and not so verbose code
>> 3. Optimized case for x,y,*z = iterator
>
> The semantics of that are already set: the first two items are assigned
> to x and y, with all subsequent items assigned to z as a list. How will
> this change optimize this case? It still needs to run through the
> iterator to generate the list.

Maybe 'optimized case for "x, y, *_ = iterator" where you then never
use _ and it has no side effects'? But that could be worded better.

>> In many cases it is possible to do this right now, but in too verbose way:
>>
>> >>> x, y = islice(gen(), 2)
>
> I don't think that is excessively verbose.
>
> But maybe we should consider allowing slice notation on arbitrary
> iterators:
>
> x, y = it[:2]

I do think islice is verbose, but the main problem is that you have to
match the second argument to the number of assignment targets. Slice
notation is an improvement, but it still has that same problem.

But perhaps this should be added to the list of options for the PEP.

> Perhaps a better idea might be special syntax to tell the interpreter
> you don't want to run the right-hand side to completion. "Explicit is
> better than implicit" -- maybe something special like:
>
> x, y, * = iterable
>
> will attempt to extract exactly two items from iterable, without
> advancing past the second item. And it could work the same for
> sequences, iterators, lazy sequences like range, and any other iterable.
>
> I don't love having yet another meaning for * but that would be better
> than changing the standard behaviour of iterator unpacking.

That's one of the options that I mentioned, as it's been proposed in
the past. The problem is that it depends on internal whitespace to
distinguish it from augmented assignment; granted, there's no way to
use "*=" with multiple targets (or even in the single-target case, you
can't do "x,*=it" with the comma in it), but that's still a
readability problem.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

>
> > To show this on simple example:
> >
> > >>> from itertools import count, islice
> > >>> it = count()
> > >>> x, y = it
> > >>> it
> > count(3)
>
> For everyone else who was confused by this, as I was, that's not
> actually a copy and paste from the REPL. There should be a ValueError
> raised after the x, y assignment. As given, it is confusing because it
> looks like the assignment succeeded, when in fact it didn't.
>
>
> > Here the count was advanced two times but assignment did not happen.
>
> Correct, because there was an exception raised.
>

Sorry for that, I did not want to embarrass anyone, so I wrote below that
the assignment did not happen. But probably the code should speak for
itself, especially if it looks like a copy from REPL


> if isinstance(it, collections.abc.Iterator):
> # special case for iterators
> x, y = it
> else:
> # sequences keep the old behaviour
> x, y = it[:2]
>

No, it can be simply x, y = iter(it)


> > Cons:
> > 1. A special case of how assignment works
> > 2. As with any implicit behavior, hard-to-find bugs
>
> Right. Hard-to-find bugs beats any amount of convenience in the
> interactive interpreter. To use an analogy:
>
> "Sure, sometimes my car suddenly shifts into reverse while I'm driving
> at 60 kph, sometimes the engine falls out when I go around the corner,
> and occasionally the brakes catch fire, but gosh the cup holder makes it
> really convenient to drink coffee while I'm stopped at traffic lights!"
>

 :-)

Perhaps a better idea might be special syntax to tell the interpreter
> you don't want to run the right-hand side to completion. "Explicit is
> better than implicit" -- maybe something special like:
>
> x, y, * = iterable
>

I wrote somewhere above, that "may be I like this form". But for me *
'"starred" target implies -> collect something from iterable. So now I'm
towards:

x, y, ... = iterable

But I have not summed up yet what pitfalls can be on this path.

Thank you your remarks were very extensive!

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 01:01:01AM +1100, Chris Angelico wrote:

> And the status quo is noisy and has duplicated
> information (you have to match the ", 2" to the number of assignment
> targets).

Er, not really. How about:

a, b = islice(iterable, 3, 10, 4)

A somewhat unusual case, to be sure, but still legal.

> I would support any syntax that (a) is currently illegal,
> (b) reads reasonably well, and (c) can't be TOO easily confused with
> something else.

Honestly, I don't see this as such a big deal that Python needs to 
support it at all: maybe +0.25 on the idea of non-consuming iterable 
unpacking. islice does the job. If we have it at all, it is yet another 
special syntax to learn, and I'm not convinced that the benefit outways 
the cost of learning yet more Perlish magic syntax for a marginal 
use-case.

Especially if the syntax looks like grit on Tim's monitor.

On the other hand, if we did get this, I would prefer magic syntax over 
a backwards incompatible change.

On balance, given that * is already used for at least 11 different 
things already[1], I'd actually prefer the grit on the monitor. Perhaps 
x, y, ... = iterable to indicate non-consuming iterable unpacking.

Or maybe 

x, y, / = iterable 

since in some sense this is the opposite of unpacking -- the whole point 
is to NOT unpack the remaining items. And the slash kind of looks like 
cutting off the process from continuing: unpack two items, then stop.

... maybe +0.5 with the slash syntax *wink*

[1] Multiplication, exponentiation, sequence replication, *varargs, 
**keyword varargs, regexes, globs, import wild cards, iterable 
unpacking in function calls, extended iterable unpacking in assignments, 
dict unpacking -- have I missed any?

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 17:14 GMT+03:00 Chris Angelico :
>
>
> Nah, far easier:
>
> x, y = iter(it)
>

Yes, you are right.

>> 2. Readable and not so verbose code
> >> 3. Optimized case for x,y,*z = iterator
> >
> > The semantics of that are already set: the first two items are assigned
> > to x and y, with all subsequent items assigned to z as a list. How will
> > this change optimize this case? It still needs to run through the
> > iterator to generate the list.
>
> Maybe 'optimized case for "x, y, *_ = iterator" where you then never
> use _ and it has no side effects'? But that could be worded better.
>

Yes, you did not need to consume and then to throw out _, and in other
cases not to hang in an endless loop.


> I do think islice is verbose, but the main problem is that you have to
> match the second argument to the number of assignment targets. Slice
> notation is an improvement, but it still has that same problem.
>
> But perhaps this should be added to the list of options for the PEP.
>

 Inconvenience is that in both cases: islice and iter[:2], you should
specify the exact number of assignment targets.


> That's one of the options that I mentioned, as it's been proposed in
> the past. The problem is that it depends on internal whitespace to
> distinguish it from augmented assignment; granted, there's no way to
> use "*=" with multiple targets (or even in the single-target case, you
> can't do "x,*=it" with the comma in it), but that's still a
> readability problem.
>

Your suggestion using Ellipsis at the moment seems to me the most readable,
like:

x, ... = iterable
x, y, ... = iterable

But I have not summed up yet what pitfalls can be on this path.

I really do not like to use "starred" targets in any way:

x, y, * = iterable
x, y, *...

Because any "starred" target implies consuming or collecting, and
contradicts with the proposed behavior.

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread Koos Zevenhoven

On Mon, Nov 27, 2017 at 4:50 PM, Kirill Balunov 
wrote:

>
> I really do not like to use "starred" targets in any way:
>
> x, y, * = iterable
>

This one won't work, because it looks like in-place multiplication
(__imul__) which clashes with the above for a single name as assignment
target:

x *= 2  # "x = x * 2"



> x, y, *...
>
> Because any "starred" target implies consuming or collecting, and
> contradicts with the proposed behavior.
>
>
Consuming does not contradict with your proposal, but maybe you mean *fully
consuming*. I think you are proposing partial or incremental consumption of
the rhs.

—Koos

-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 01:14:40AM +1100, Chris Angelico wrote:

> Nah, far easier:
> 
> x, y = iter(it)
> 
> since that'll be a no-op in the first case

Not necessarily a no-op. __iter__ might have side-effects.

Of course if you can guarantee that you ONLY have iterators, then 
there's no need to test for an iterator. But it isn't always 
appropriate to convert sequences to an iterator.

And besides, if you're going to call iter(), that's not that much 
shorter than islice() (assuming you've already imported it, of course). 
You only save five characters or so.

But the big problem here is that iterable unpacking would be 
conceptually split into "iterator unpacking" and "all other iterables 
unpacking". I think that's unnecessary complication.

> I do think islice is verbose, but the main problem is that you have to
> match the second argument to the number of assignment targets.

Yes, there is a certain amount of redundancy in having to specify the number 
of items in a slice, using either syntax:

a, b = sequence[:2]
a, b = islice(iterable, 2)

but it is minimal duplication, hardly the sort of thing DRY is concerned 
with:

http://www.artima.com/intv/dry.html

and there is an equally important principle that is satisfied by being 
explicit about the number of items you want. (In the face of ambiguity, 
refuse the temptation to guess.)

If you get the number wrong, you will find out immediately. And the fix 
is trivial.

In this case, there is a small but real benefit to counting the 
assignment targets and being explicit about the number of items to 
slice. Consider an extension to this "non-consuming" unpacking that 
allowed syntax like this to pass silently:

a, b = x, y, z

That ought to be a clear error, right? I would hope you don't think that 
Python should let that through. Okay, now we put x, y, z into a list, 
then unpack the list:

L = [x, y, z]
a, b = L

That ought to still be an error, unless we explicity silence it. One way 
to do so is with an explicit slice:

a, b = L[:2]

This isn't repeating yourself, it isn't really duplicated or redundant 
information. It is telling the interpreter, and more importantly the 
reader, that you want exactly two items out of a list that could contain 
any arbitrary number of items and that you are planning to bind them to 
exactly two targets.

Same goes if we use islice(iterable) instead.

Another way would be special syntax to say you want non-consuming 
unpacking, swapping out the "ambiguity" Zen for the "explicitly 
silencing errors" Zen.

So I really don't see this as anything more than, at best, a fairly 
minor piece of syntactic sugar. It doesn't really add anything to the 
language, or allow us to do anything cool we couldn't do before.

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 1:46 AM, Steven D'Aprano  wrote:
> On Tue, Nov 28, 2017 at 01:01:01AM +1100, Chris Angelico wrote:
>
>> And the status quo is noisy and has duplicated
>> information (you have to match the ", 2" to the number of assignment
>> targets).
>
> Er, not really. How about:
>
> a, b = islice(iterable, 3, 10, 4)
>
> A somewhat unusual case, to be sure, but still legal.

Well, sure. But the situation that's equivalent to the proposal here
is simply islice(iterable, 2). There's no proposal to have something
that assigns the way you're slicing there, and there's certainly no
proposal to abolish islice.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 2:35 AM, Steven D'Aprano  wrote:
> In this case, there is a small but real benefit to counting the
> assignment targets and being explicit about the number of items to
> slice. Consider an extension to this "non-consuming" unpacking that
> allowed syntax like this to pass silently:
>
> a, b = x, y, z
>
> That ought to be a clear error, right? I would hope you don't think that
> Python should let that through. Okay, now we put x, y, z into a list,
> then unpack the list:
>
> L = [x, y, z]
> a, b = L
>
> That ought to still be an error, unless we explicity silence it. One way
> to do so is with an explicit slice:
>
> a, b = L[:2]

I absolutely agree with this for the default case. That's why I am
ONLY in favour of the explicit options. So, for instance:

a, b = x, y, z # error
a, b, ... = x, y, z # valid (evaluates and ignores z)

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On 27 November 2017 at 16:05, Chris Angelico  wrote:
> On Tue, Nov 28, 2017 at 2:35 AM, Steven D'Aprano  wrote:
>> In this case, there is a small but real benefit to counting the
>> assignment targets and being explicit about the number of items to
>> slice. Consider an extension to this "non-consuming" unpacking that
>> allowed syntax like this to pass silently:
>>
>> a, b = x, y, z
>>
>> That ought to be a clear error, right? I would hope you don't think that
>> Python should let that through. Okay, now we put x, y, z into a list,
>> then unpack the list:
>>
>> L = [x, y, z]
>> a, b = L
>>
>> That ought to still be an error, unless we explicity silence it. One way
>> to do so is with an explicit slice:
>>
>> a, b = L[:2]
>
> I absolutely agree with this for the default case. That's why I am
> ONLY in favour of the explicit options. So, for instance:
>
> a, b = x, y, z # error
> a, b, ... = x, y, z # valid (evaluates and ignores z)

Agreed, only explicit options are even worth considering (because of
backward compatibility if for no other reason). However, the unpacking
syntax is already complex, and hard to search for. Making it more
complex needs a strong justification. And good luck in doing a google
search for "..." if you saw that code in a project you had to
maintain. Seriously, has anyone done a proper investigation into how
much benefit this proposal would provide? It should be reasonably easy
to do a code search for something like "=.*islice", to find code
that's a candidate for using the proposed syntax. I suspect there's
very little code like that.

I'm -1 on this proposal without a much better justification of the
benefits it will bring.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] generator vs iterator etc. (was: How assignment should work with generators?)

2017-11-27 Thread Koos Zevenhoven

On Mon, Nov 27, 2017 at 3:55 PM, Steven D'Aprano 
 wrote:

> On Mon, Nov 27, 2017 at 12:17:31PM +0300, Kirill Balunov wrote:
> 
>
> > 2. Should this work only for generators or for any iterators?
>
> I don't understand why you are even considering singling out *only*
> generators. A generator is a particular implementation of an iterator. I
> can write:
>
> def gen():
>yield 1; yield 2; yield 3
>
> it = gen()
>
> or I can write:
>
> it = iter([1, 2, 3])
>
> and the behaviour of `it` should be identical.
>
>
>
I can see where this is coming from. The thing is that "iterator" and
"generator" are mostly synonymous, except two things:

(1) Generators are iterators that are produced by a generator function

(2) Generator functions are sometimes referred to as just "generators"

The concept of "generator" thus overlaps with both "iterator" and
"generator function".

Then there's also "iterator" and "iterable", which are two different things:

(3) If `obj` is an *iterable*, then `it = iter(obj)` is an *iterator* (over
the contents of `obj`)

(
4) Iterators yield values, for example on explicit calls to next(it).

Personally I have leaned towards keeping a clear distinction between
"generator function" and "generator", which leads to the situation that
"generator" and "iterator" are mostly synonymous for me. Sometimes, for
convenience, I use the term "generator" to refer to "iterators" more
generally. This further seems to have a minor benefit that "generators" and
"iterables" are less easily confused with each other than "iterators" and
"iterables".

I thought about this issue some time ago for the `views` package, which has
a separation between sequences (seq) and other iterables (gen):

https://github.com/k7hoven/views

The functionality provided by `views.gen` is not that interesting—it's
essentially a subset of itertools functionality, but with an API that
parallels `views.seq` which works with sequences (iterable, sliceable,
chainable, etc.). I used the name `gen`, because iterator/iterable variants
of the functionality can be implemented with generator functions (although
also with other kinds of iterators/iterables). Calling the thing `iter`
would have conflicted with the builtin `iter`.

HOWEVER, this naming can be confusing for those that lean more towards
using "generator" to also mean "generator function", and for those that are
comfortable with the term "iterator" despite its resemblance to "iterable".

Now I'm actually seriously considering to consider renaming `views.gen` to `
views.iter` when I have time. After all, there's already `views.range`
which "conflicts" with the builtin range.

Anyway, the point is that the naming is suboptimal.

SOLUTION: Maybe (a) all iterators should be called iterators or (b) all
iterators should be called generators, regardless of whether they are
somehow a result of a generator function having been called in the past.

(I'm not going into the distinction between things that can receive values
via `send` or any other possible distinctions between different types of
iterators and iterables.)

—Koos

(discussion originated from python-ideas, but cross-posted to python-dev
in case there's more interest there)

-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread Koos Zevenhoven

On Mon, Nov 27, 2017 at 3:55 PM, Steven D'Aprano 
wrote:

> On Mon, Nov 27, 2017 at 12:17:31PM +0300, Kirill Balunov wrote:
> 
>
> > 2. Should this work only for generators or for any iterators?
>
> I don't understand why you are even considering singling out *only*
> generators. A generator is a particular implementation of an iterator. I
> can write:
>
> def gen():
>yield 1; yield 2; yield 3
>
> it = gen()
>
> or I can write:
>
> it = iter([1, 2, 3])
>
> and the behaviour of `it` should be identical.
>
>
>
I can see where this is coming from, but I wrote about it in a new
thread: "generator
vs iterator etc. (was: How assignment should work with generators?)".


>
> But maybe we should consider allowing slice notation on arbitrary
> iterators:
>
> x, y = it[:2]
>
>
> I have not thought this through in any serious detail, but it seems to
> me that if the only problem here is the inconvenience of using islice(),
> we could add slicing to iterators. I think that would be better than
> having iterators and other iterables behave differently.
>
>
Making iterators behave like sequences (slicing etc.) introduces various
issues including memory considerations and backwards compatibility. That's
why the `views` package [1] keeps a clear separations between sequences and
iterators. IterABLES are a bit fuzzy here, but they at least should be able
to produce an iterator. I should have time to discuss this more at a later
point, if needed.

—Koos

[1] https://github.com/k7hoven/views



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread C Anthony Risinger

This proposal resonates with me. I've definitely wanted to use unpacking to
crank an iterator a couple times and move on without exhausting the
iterator. It's a very natural and intuitive meaning for unpacking as it
relates to iterators.

In my mind, this ask is aligned with, and has similar motivation to, lazy
zip(), map(), and keys() in Python 3. Syntax support for unpacking as it
stands today is not very conducive to iterables and conflicts with a
widespread desire to use more of them.

Couple thoughts:

* Perhaps existence of `__len__` should influence unpacking? There is a
semantic difference (and typically a visual one too) between 1-to-1
matching a fixed-width sequence/container on the RHS to identifiers on the
LHS, even if they look similar (ie. "if RHS has a length make it fit,
otherwise don't").

* (related to above) Perhaps the "rigidity"(?) of both RHS and LHS should
influence unpacking? If they are both fixed-width, expect exact match. If
either is variable-width, then lazily unravel until a different problem
happens (eg. LHS is fixed-width but RHS ran out of values... basically we
always unravel lazily, but keep track of when LHS or RHS become variable,
and avoid checking length if they do).

* Python 4 change as the language moves towards lazy iterators everywhere?
If `__len__` influenced behavior like mentioned above, then a mechanical
fix to code would simply be `LHS = tuple(*RHS)`, similar to keys().

While I like the original proposal, adding basic slice support to iterables
is also a nice idea. Both are independently useful, eg.
`gen.__getitem__(slice())` delegates to islice(). This achieves the goal of
allowing meaningful unpacking of an iterator window, using normal syntax,
without breaking existing code.

The fixed/variable-width idea makes the most sense to me though. This
enables things like:

>>> a, b, c = (1, *range(2, 100), 3)
(1, 2, 3)

Since both sides are not explicitly sized unpacking is not forcibly sized
either.

Expecting LHS/RHS to exactly match 100% of the time is the special case
here today, not the proposed general unpacking rules that will work well
with iterators. This change also has a Lua-like multiple return value feel
to it that appeals to me.

Thanks,

On Nov 27, 2017 10:23 AM, "Paul Moore"  wrote:

> On 27 November 2017 at 16:05, Chris Angelico  wrote:
> > On Tue, Nov 28, 2017 at 2:35 AM, Steven D'Aprano 
> wrote:
> >> In this case, there is a small but real benefit to counting the
> >> assignment targets and being explicit about the number of items to
> >> slice. Consider an extension to this "non-consuming" unpacking that
> >> allowed syntax like this to pass silently:
> >>
> >> a, b = x, y, z
> >>
> >> That ought to be a clear error, right? I would hope you don't think that
> >> Python should let that through. Okay, now we put x, y, z into a list,
> >> then unpack the list:
> >>
> >> L = [x, y, z]
> >> a, b = L
> >>
> >> That ought to still be an error, unless we explicity silence it. One way
> >> to do so is with an explicit slice:
> >>
> >> a, b = L[:2]
> >
> > I absolutely agree with this for the default case. That's why I am
> > ONLY in favour of the explicit options. So, for instance:
> >
> > a, b = x, y, z # error
> > a, b, ... = x, y, z # valid (evaluates and ignores z)
>
> Agreed, only explicit options are even worth considering (because of
> backward compatibility if for no other reason). However, the unpacking
> syntax is already complex, and hard to search for. Making it more
> complex needs a strong justification. And good luck in doing a google
> search for "..." if you saw that code in a project you had to
> maintain. Seriously, has anyone done a proper investigation into how
> much benefit this proposal would provide? It should be reasonably easy
> to do a code search for something like "=.*islice", to find code
> that's a candidate for using the proposed syntax. I suspect there's
> very little code like that.
>
> I'm -1 on this proposal without a much better justification of the
> benefits it will bring.
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Using an appropriate tone in emails (was: Adding a thin wrapper class around the functions in stdlib.heapq)

2017-11-27 Thread Brett Cannon

On Tue, 21 Nov 2017 at 14:57 Nick Timkovich  wrote:

> On Tue, Nov 21, 2017 at 4:16 PM, Sven R. Kunze  wrote:
>
>> Maybe, that suffices: https://pypi.python.org/pypi/xheap
>>
> I still think the heapq.heap* functions are atrocious and they should
> immediately be packaged with *no additional features* into a stdlib object
> for reasons along the line of
> https://mail.python.org/pipermail/python-ideas/2017-October/047514.html
>

I wanted to address the general tone people should aim for on this list so
that people don't inadvertently insult people. Now I'm going to preface
this as I'm not explicitly calling out Nick here on this, I'm just using
his email as an illustrative example as it's what happened to trigger me to
write this email.

So Nick said above that the design of the heapq module is "atrocious", to
the extent that it should be "immediately" fixed with an OO wrapper. So
obviously Nick doesn't like the design of the heapq module. ;) And that's
okay! And he's totally within his rights to express the feeling that the
heapq module as it stands doesn't meet his needs.

But calling it "atrocious" and so bad that it needs to be fixed
"immediately" as if it's a blight upon the stdlib is unnecessarily
insulting to those that have worked on the module. To convey the feeling
that you think an OO wrapper would be helpful as the current design doesn't
work for you, you could just phrase it as I just did to get the same point
across without insulting anyone. Basically if you wouldn't like your own
work called "atrocious" by someone you respect, then it's probably best to
not use that phrasing when talking about a stranger's code either.

If you want more context as to why this all matters in order to keep open
source sustainable, you can watch my 15 minute keynote from JupyterCon this
year for a more in-depth, verbal explanation:
https://www.youtube.com/watch?v=y19s6vPpGXA .

And BTW, the heapq module is 15 years old, has a history, and there are
explicit reasons it's designed the way it is, so from that perspective I
would argue it has a good design.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using an appropriate tone in emails (was: Adding a thin wrapper class around the functions in stdlib.heapq)

+1

Basically, if you're posting here, you probably want something to change.
And that means that somebody has to make that change. And someone has to
approve it. Etc. And you are trying to influence those people to
make/approve/etc. that change, since for them it's less work if nothing
change. And how do you influence people? Not by opening with an insult.

(Also if you think that none of this matters because we're all rational
actors, grow up. :-)

--Guido

On Mon, Nov 27, 2017 at 12:17 PM, Brett Cannon  wrote:

>
>
> On Tue, 21 Nov 2017 at 14:57 Nick Timkovich 
> wrote:
>
>> On Tue, Nov 21, 2017 at 4:16 PM, Sven R. Kunze  wrote:
>>
>>> Maybe, that suffices: https://pypi.python.org/pypi/xheap
>>>
>> I still think the heapq.heap* functions are atrocious and they should
>> immediately be packaged with *no additional features* into a stdlib object
>> for reasons along the line of https://mail.python.org/
>> pipermail/python-ideas/2017-October/047514.html
>>
>
> I wanted to address the general tone people should aim for on this list so
> that people don't inadvertently insult people. Now I'm going to preface
> this as I'm not explicitly calling out Nick here on this, I'm just using
> his email as an illustrative example as it's what happened to trigger me to
> write this email.
>
> So Nick said above that the design of the heapq module is "atrocious", to
> the extent that it should be "immediately" fixed with an OO wrapper. So
> obviously Nick doesn't like the design of the heapq module. ;) And that's
> okay! And he's totally within his rights to express the feeling that the
> heapq module as it stands doesn't meet his needs.
>
> But calling it "atrocious" and so bad that it needs to be fixed
> "immediately" as if it's a blight upon the stdlib is unnecessarily
> insulting to those that have worked on the module. To convey the feeling
> that you think an OO wrapper would be helpful as the current design doesn't
> work for you, you could just phrase it as I just did to get the same point
> across without insulting anyone. Basically if you wouldn't like your own
> work called "atrocious" by someone you respect, then it's probably best to
> not use that phrasing when talking about a stranger's code either.
>
> If you want more context as to why this all matters in order to keep open
> source sustainable, you can watch my 15 minute keynote from JupyterCon this
> year for a more in-depth, verbal explanation:
> https://www.youtube.com/watch?v=y19s6vPpGXA .
>
> And BTW, the heapq module is 15 years old, has a history, and there are
> explicit reasons it's designed the way it is, so from that perspective I
> would argue it has a good design.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?


Chris Angelico wrote:


x, y, * = iter # unpack into nothing


I'm surprised this isn't already allowed. It seems like the
One Obvious Way to me.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?


Chris Angelico wrote:

The problem is that it depends on internal whitespace to
distinguish it from augmented assignment;


Ah, didn't spot that. I guess the ellipsis is the next best
thing then.

An alternative would be to require parens:

   (x, y, *) = z

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 1:13 PM, Greg Ewing 
wrote:

> Chris Angelico wrote:
>
> x, y, * = iter # unpack into nothing
>>
>
> I'm surprised this isn't already allowed. It seems like the
> One Obvious Way to me.

I'm not that surprised. While it appears to rhyme with the use of a lone
'*' in function signatures, it would actually mean the opposite: in
signatures it means "don't allow any more". These opposite meanings would
cause some confusion.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 1:18 PM, Greg Ewing 
wrote:

> Chris Angelico wrote:
>
>> The problem is that it depends on internal whitespace to
>> distinguish it from augmented assignment;
>>
>
> Ah, didn't spot that. I guess the ellipsis is the next best
> thing then.
>
> An alternative would be to require parens:
>
>(x, y, *) = z
>

But that would have the same issue.

Is this problem really important enough that it requires dedicated syntax?
Isn't the itertools-based solution good enough? (Or failing that, couldn't
we add something to itertools to make it more readable rather than going
straight to new syntax?)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 8:24 AM, Guido van Rossum  wrote:
> On Mon, Nov 27, 2017 at 1:18 PM, Greg Ewing 
> wrote:
>>
>> Chris Angelico wrote:
>>>
>>> The problem is that it depends on internal whitespace to
>>> distinguish it from augmented assignment;
>>
>>
>> Ah, didn't spot that. I guess the ellipsis is the next best
>> thing then.
>>
>> An alternative would be to require parens:
>>
>>(x, y, *) = z
>
>
> But that would have the same issue.
>
> Is this problem really important enough that it requires dedicated syntax?
> Isn't the itertools-based solution good enough? (Or failing that, couldn't
> we add something to itertools to make it more readable rather than going
> straight to new syntax?)

I don't think there's much that can be done without syntax; the
biggest problem IMO is that you need to tell islice how many targets
it'll be assigned into. It needs some interpreter support to express
"grab as many as you have targets for, leaving everything else behind"
without stating how many that actually is. So the question is whether
that is sufficiently useful to justify extending the syntax. There are
a number of potential advantages and several competing syntax options,
and this suggestion keeps coming up, so I think a PEP is warranted.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

While I was more optimistic when I proposed this idea, moving on I
gradually become less and less optimistic. I did not underestimate how much
this would change the semantics. At present, if the lhs is comma-separated
list of targets, the rhs is evaluated, and only then if they match the
assignment happens. Here it depends on the number of targets in lhs, and
should be evaluated lazily. So someone too perlish clever can assume that
with the proposed syntax:

>>> def gen():
>>> for i in ['a', 'b', 'c', 'd']:
>>> v = x if 'x' in globals() else 'var '
>>> yield v + i

>>> x, y = gen()
>>> x
var a
>>> y
var ab

This is bad and I do not like it, but I do not see any serious reasons why
it should not be allowed. In case of Ellipsis they also should trigger
special behavior.

 While I like this feature of lazy assignment, may be it becomes more
special than it deserves.

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?


C Anthony Risinger wrote:
* Perhaps existence of `__len__` should influence unpacking? There is a 
semantic difference (and typically a visual one too) between 1-to-1 
matching a fixed-width sequence/container on the RHS to identifiers on 
the LHS, even if they look similar (ie. "if RHS has a length make it 
fit, otherwise don't").


-1. There's a convention that an iterator can implement __len__
to provide a hint about the number of items it will return
(useful for preallocating space, etc.) It would be very
annoying if such an iterator behaved differently from other
iterators when unpacking.

Another thing is that the proposed feature will be useful
on non-iterator iterables as well, since it saves the overhead
of unpacking the rest of the items only to throw them away.

While I like the original proposal, adding basic slice support to 
iterables is also a nice idea.


It's not as nice as it seems. You're asking for __getitem__
to be made part of the iterator protocol, which would be a
huge change affecting all existing iterators.

Otherwise, it would just be a piecemeal affair. Some iterators
would support slicing and some wouldn't, so you couldn't
rely on it.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 1:32 PM, Chris Angelico  wrote:

> On Tue, Nov 28, 2017 at 8:24 AM, Guido van Rossum 
> wrote:
> > On Mon, Nov 27, 2017 at 1:18 PM, Greg Ewing  >
> > wrote:
> >>
> >> Chris Angelico wrote:
> >>>
> >>> The problem is that it depends on internal whitespace to
> >>> distinguish it from augmented assignment;
> >>
> >>
> >> Ah, didn't spot that. I guess the ellipsis is the next best
> >> thing then.
> >>
> >> An alternative would be to require parens:
> >>
> >>(x, y, *) = z
> >
> >
> > But that would have the same issue.
> >
> > Is this problem really important enough that it requires dedicated
> syntax?
> > Isn't the itertools-based solution good enough? (Or failing that,
> couldn't
> > we add something to itertools to make it more readable rather than going
> > straight to new syntax?)
>
> I don't think there's much that can be done without syntax; the
> biggest problem IMO is that you need to tell islice how many targets
> it'll be assigned into. It needs some interpreter support to express
> "grab as many as you have targets for, leaving everything else behind"
> without stating how many that actually is. So the question is whether
> that is sufficiently useful to justify extending the syntax. There are
> a number of potential advantages and several competing syntax options,
> and this suggestion keeps coming up, so I think a PEP is warranted.
>

OK, that's reasonable, and at first blush the ellipsis proposal looks okay.
My PEP queue for Python 3.7 is full though, so I would like to put this off
until 3.8.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 8:49 AM, Guido van Rossum  wrote:
> My PEP queue for Python 3.7 is full though, so I would like to put this off
> until 3.8.
>

Yeah, I don't think this could reasonably be raced into 3.7 even if it
were critically important, and it's not. 3.8 will be fine.

Kirill, do you want to spearhead the discussion? I'm happy to help out.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 19:23 GMT+03:00 Paul Moore :


> It should be reasonably easy
> to do a code search for something like "=.*islice", to find code
> that's a candidate for using the proposed syntax. I suspect there's
> very little code like that.


While isclice is something  equivalent, it can be used in places like:

x, y = seq[:2]
x, y, z = seq[:3]

With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using an appropriate tone in emails (was: Adding a thin wrapper class around the functions in stdlib.heapq)

2017-11-27 Thread Nick Timkovich

On Mon, Nov 27, 2017 at 8:17 PM, Brett Cannon  wrote:

> But calling it "atrocious" and so bad that it needs to be fixed
> "immediately" as if it's a blight upon the stdlib is unnecessarily
> insulting to those that have worked on the module. To convey the feeling
> that you think an OO wrapper would be helpful as the current design doesn't
> work for you, you could just phrase it as I just did to get the same point
> across without insulting anyone. Basically if you wouldn't like your own
> work called "atrocious" by someone you respect, then it's probably best to
> not use that phrasing when talking about a stranger's code either.
>

Sorry for the curt tone, I did lose some sight on the code being designed
by people rather than a faceless organization. My intention wasn't to
disparage the original authors but sprung more out of my frustration and
perception from that thread and those before that the status quo would not
change and that if a contribution was proffered, would simply be dismissed
or ignored. To motivate any change, there must be some argument levied
against the status quo, but hopefully I can articulate it better.

That little corner is something I'm interested in, and not having
contributed to CPython before, I'm unsure how it "really works". The steps
at https://devguide.python.org/stdlibchanges/ suggest trying to elicit
community feedback from the lists as a step, so negative feedback tends to
kill the enthusiasm to actually make the PR. In the absence of code,
concrete arguments are almost impossible as we're discussing the shape of
clouds.

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?


Guido van Rossum wrote:
While it appears to rhyme with the use of a lone 
'*' in function signatures, it would actually mean the opposite: in 
signatures it means "don't allow any more".


That's the usage that's out of step with the rest -- all
the others can be seen as some form of wildcard.

So it's already confusing, and I don't think adding another
wildcard meaning would make things any worse.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On 27 November 2017 at 21:54, Kirill Balunov  wrote:
> 2017-11-27 19:23 GMT+03:00 Paul Moore :
>
>>
>> It should be reasonably easy
>> to do a code search for something like "=.*islice", to find code
>> that's a candidate for using the proposed syntax. I suspect there's
>> very little code like that.
>
>
> While isclice is something  equivalent, it can be used in places like:
>
> x, y = seq[:2]
> x, y, z = seq[:3]

But in those places,

x, y, *_ = seq

works fine at the moment. So if the programmer didn't feel inclined to
use x, y, *_ = seq, there's no reason to assume that they would get
any benefit from x, y, ... = seq either.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread Serhiy Storchaka


27.11.17 23:24, Guido van Rossum пише:
Is this problem really important enough that it requires dedicated 
syntax? Isn't the itertools-based solution good enough? (Or failing 
that, couldn't we add something to itertools to make it more readable 
rather than going straight to new syntax?)


I want to remind PEP 204 and PEP 212. The special purposed syntaxes were 
proposed to help solving much more common problems. They were rejected 
in favor of builtins range() and enumerate(). And we are happy with 
these builtins.


The function for solving this problem already exists. It's 
itertools.islice(). It isn't builtin, but this problem is much less 
common than use cases for range() and enumerate().


If we don't have special non-function syntax for range(), enumerate(), 
zip(), itertools.chain(), itertools.count(), itertools.repeat(), etc, I 
don't think we should have a special syntax for one particular case of 
using itertools.islice().


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using an appropriate tone in emails (was: Adding a thin wrapper class around the functions in stdlib.heapq)

On 27 November 2017 at 21:59, Nick Timkovich  wrote:
> On Mon, Nov 27, 2017 at 8:17 PM, Brett Cannon  wrote:
>>
>> But calling it "atrocious" and so bad that it needs to be fixed
>> "immediately" as if it's a blight upon the stdlib is unnecessarily insulting
>> to those that have worked on the module. To convey the feeling that you
>> think an OO wrapper would be helpful as the current design doesn't work for
>> you, you could just phrase it as I just did to get the same point across
>> without insulting anyone. Basically if you wouldn't like your own work
>> called "atrocious" by someone you respect, then it's probably best to not
>> use that phrasing when talking about a stranger's code either.
>
>
> Sorry for the curt tone, I did lose some sight on the code being designed by
> people rather than a faceless organization. My intention wasn't to disparage
> the original authors but sprung more out of my frustration and perception
> from that thread and those before that the status quo would not change and
> that if a contribution was proffered, would simply be dismissed or ignored.
> To motivate any change, there must be some argument levied against the
> status quo, but hopefully I can articulate it better.
>
> That little corner is something I'm interested in, and not having
> contributed to CPython before, I'm unsure how it "really works". The steps
> at https://devguide.python.org/stdlibchanges/ suggest trying to elicit
> community feedback from the lists as a step, so negative feedback tends to
> kill the enthusiasm to actually make the PR. In the absence of code,
> concrete arguments are almost impossible as we're discussing the shape of
> clouds.

In my experience (and this reiterates Brett's point) the proposals
that get the best responses are those that are presented positively -
instead of focusing on the (perceived) problems with the current
situation, describe the benefits that will come from the proposed
change. If you can't do that, then it's unlikely there is enough
justification for a change. Certainly, negative feedback can be
demotivating, and when you have a great idea and all you hear is "but
what if...?" it's hard to remain positive. But you're not going to get
better feedback if you criticise - at best, people will stop
listening, and you'll have avoided some of the arguments, but at the
cost of no-one being willing to support your proposal and so it dies.

Your perception isn't wrong, by the way. It *is* hard to persuade
people that the status quo needs to change. But that's not because
there's no interest in change. Rather, it's because there's a strong
sense among both the core developers and the frequent contributors on
this list, of the significant impact any change will have - it's hard
to conceive of just how many people will be affected by even the
smallest change we make, and that's a big responsibility.

So while it's often hard, focusing on the positives (and being willing
to accept that the status quo is sufficient for many people) really is
the only way to gain support.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?


Guido van Rossum wrote:
Is this problem really important enough that it requires dedicated 
syntax? Isn't the itertools-based solution good enough?


Well, it works, but it feels very clumsy. It's annoying to
have to specify the number of items in two places.

Also, it seems perverse to have to tell Python to do *more*
stuff to mitigate the effects of stuff it does that you
didn't want it to do in the first place.

Like I said, I'm actually surprised that this doesn't already
work. To me it feels more like filling in a piece of
functionality that was overlooked, rather than adding a
new feature. Filling in a pothole in the road rather than
bulding a new piece of road.

(Pushing the road analogy maybe a bit too far, the current
itertools solution is like digging *more* potholes to make
the road bumpy enough that you don't notice the first
pothole.)

(Or failing 
that, couldn't we add something to itertools to make it more readable 
rather than going straight to new syntax?)


I'm not sure how we would do that. Even if we could, it
would still feel clumsy having to use anything from itertools
at all.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] generator vs iterator etc. (was: How assignment should work with generators?)

On Mon, Nov 27, 2017 at 06:35:38PM +0200, Koos Zevenhoven wrote:

> SOLUTION: Maybe (a) all iterators should be called iterators 

All iterators *are* called iterators. Just as all mammals are called 
"mammals".

The subset of iterators which are created as generators are *also* 
called generators, just as the mammals which are dogs are called "dogs" 
when it is necessary to distinguish a dog from some other mammal.

> or (b) all
> iterators should be called generators, regardless of whether they are
> somehow a result of a generator function having been called in the past.

Absolutely not. That would be confusing -- it would be analogous to 
calling all sequences (lists, tuples, deques etc) "strings". What 
benefit is there to calling all iterators "generators", losing the 
distinction between those which are defined using def and yield and 
those created using iter()? Sometimes that distinction is important.

You are right that sometimes the term "generator" is used as shorthand 
for "generator function". Most of the time the distinction doesn't 
actually matter, since you cannot (easily?) create a generator without 
first creating a generator function. 

Or if it does matter, it is clear in context which is meant.

For those few times where it *does* matter, there is no substitute for 
precision in language, and that depends on the author, not the 
terminology.

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 10:49:45AM +1300, Greg Ewing wrote:
> C Anthony Risinger wrote:
> >* Perhaps existence of `__len__` should influence unpacking? There is a 
> >semantic difference (and typically a visual one too) between 1-to-1 
> >matching a fixed-width sequence/container on the RHS to identifiers on 
> >the LHS, even if they look similar (ie. "if RHS has a length make it 
> >fit, otherwise don't").
> 
> -1. There's a convention that an iterator can implement __len__
> to provide a hint about the number of items it will return
> (useful for preallocating space, etc.)

I think you mean __length_hint__ for giving a hint. If an iterator 
supported __len__ itself, I'd expect it to be exact, not a hint. I'm not 
sure that its a strong convention, but we certainly could create custom 
iterators that supported len().

https://www.python.org/dev/peps/pep-0424/

> >While I like the original proposal, adding basic slice support to 
> >iterables is also a nice idea.
> 
> It's not as nice as it seems. You're asking for __getitem__
> to be made part of the iterator protocol, which would be a
> huge change affecting all existing iterators.

There are ways around that. As I said earlier, I'm not going to champion 
this idea, but for the sake of brainstorming, the interpreter could 
implement obj[slice] as:

if obj has __getitem__:
call obj.__getitem__(slice)
elif iter(obj) is obj:
call itertools.islice(obj, slice)
else:
raise TypeError

or similar. Its not literally necessary to require iterators themselves 
to support a __getitem__ method in order to add slicing to the iterator 
protocol.

(Similar to the way bool() falls back on testing for non-zero length in 
the event that __bool__ doesn't exist, or != falls back on calling and 
NOT'ing __eq__ if __ne__ doesn't exist.)

So I think this is solvable if it needs to be solved.

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 06:53:23PM +0200, Koos Zevenhoven wrote:

> Making iterators behave like sequences (slicing etc.) introduces various
> issues including memory considerations and backwards compatibility.

Perhaps.

I'm not going to actively champion the idea of supporting slicing for 
iterators, but the idea I had was for no more than having 
iterator[start:stop:step] to do *exactly* what 
itertools.islice(iterator, start, stop, step) does now.

> That's
> why the `views` package [1] keeps a clear separations between sequences and
> iterators. IterABLES are a bit fuzzy here, but they at least should be able
> to produce an iterator.

I don't think the concept of iterable is fuzzy: they are a superset of 
iterators.

To be precise, an iterable is something which supports iteration, which 
means it must either:

- support the iterator protocol with __iter__ and __next__ raising 
StopIteration;

- or support the classic sequence protocol with __getitem__ raising 
IndexError at the end of the sequence.

I think that's the only two possibilities. That covers (all?) 
collections, sequences, lists, lazy computed sequences like range, 
iterators, generators, etc.

https://docs.python.org/3/glossary.html#term-iterable

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 11:15:56AM +1300, Greg Ewing wrote:
> Guido van Rossum wrote:
> >While it appears to rhyme with the use of a lone 
> >'*' in function signatures, it would actually mean the opposite: in 
> >signatures it means "don't allow any more".
> 
> That's the usage that's out of step with the rest -- all
> the others can be seen as some form of wildcard.
> 
> So it's already confusing, and I don't think adding another
> wildcard meaning would make things any worse.

How does "stop iterating here" equate to a wildcard?

We already have a "wildcard" for iterable unpacking, using the extended 
iterable unpacking syntax:

x, y, *z = iterable

which unpacks the first two items into x and y and the rest into z. This 
is the opposite: *stop* unpacking, so that after x and y are unpacked 
the process stops. I don't see how this is conceptually a wild card.

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Repurpose `assert' into a general-purpose check

The `assert' statment was created the same as in previous languages like 
C/C++: a check to only do in debug mode, when you can't yet trust your 
code to manage and pass around internal data correctly. Examples are 
array bounds and object state integrity constraints.


Unlike C, Python does the aforementioned checks all the time, i.e. it's 
effectively always in "debug mode". Furthermore, validation checks are 
an extremily common use case.


I have been using assert in my in-house code for input validation for a 
long time, the only thing preventing me from doing the same in public 
code is the fact that it only raises AssertionError's while library code 
is expected to raise TypeError or ValueError on invalid input.


The last drop that incited me to do this proposition was 
https://stackoverflow.com/questions/2142202/what-are-acceptable-use-cases-for-pythons-assert-statement 
where none of the _seven_ answer authors (beside me) managed to make a 
_single realistic use case_ for `assert' as a debug-only check.


---

So, I'm hereby proposing to:

* make `assert' stay in optimized mode
* change its syntax to raise other types of exceptions

E.g.: "assert condition, type, value" or "assert condition, type, 
exception_constructor_argument(s)" to maintain backward compatibility.


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

On Tue, Nov 28, 2017 at 1:12 PM, Ivan Pozdeev via Python-ideas
 wrote:
> The `assert' statment was created the same as in previous languages like
> C/C++: a check to only do in debug mode, when you can't yet trust your code
> to manage and pass around internal data correctly. Examples are array bounds
> and object state integrity constraints.
>
> Unlike C, Python does the aforementioned checks all the time, i.e. it's
> effectively always in "debug mode". Furthermore, validation checks are an
> extremily common use case.
>
> I have been using assert in my in-house code for input validation for a long
> time, the only thing preventing me from doing the same in public code is the
> fact that it only raises AssertionError's while library code is expected to
> raise TypeError or ValueError on invalid input.

Actually, Python does have a way of disabling assertions (the -O
flag), so they should be treated the same way they are in C.
Assertions should not be used as shorthands for "if cond: raise Exc"
in the general case.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check


On 28.11.2017 5:19, Chris Angelico wrote:


Actually, Python does have a way of disabling assertions (the -O
flag), so they should be treated the same way they are in C.
Assertions should not be used as shorthands for "if cond: raise Exc"
in the general case.
I'm claiming, and provided evidence, that there are no use cases for 
this in Python, so no-one (of any significance) will suffer when the 
disabling is cut out.
In any case, we will probably do it with a __future__ statement for a 
transitional period.


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread Mike Miller


Hmm, I didn't like the options below because they say to me, "consume 
everything:"

x, y, * = iterable
x, y, ... = iterable


Believe the question behind the idea was, how to grab a couple items and then 
*stop?*  If the syntax route is chosen, I'd expect something that tells me it is 
going to stop, like a "full stop" as the period/dot is called in jolly ol' 
England, e.g.:


x, y, . = iterable

Not sure about the second comma though.

-Mike



On 2017-11-27 13:18, Greg Ewing wrote:

Chris Angelico wrote:

The problem is that it depends on internal whitespace to
distinguish it from augmented assignment;


Ah, didn't spot that. I guess the ellipsis is the next best
thing then.

An alternative would be to require parens:

    (x, y, *) = z

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

2017-11-27 Thread Jason Maldonis

>
> Assertions should not be used as shorthands for "if cond: raise Exc"
> in the general case.
>


I'm just a lurker and usually I agree with why the suggested features
shouldn't be implemented, but I actually might chime in to pitch this one a
bit more -- and I think it can be done nicely without breaking backward
compatibility.

As a scientist, I like assert statements for two reasons: 1) you can
disable them to speed up the code, and 2) it's how we think as scientists.

Consider these two pieces of code, and which one you'd prefer to read:

if not condition:
raise ValueError

assert condition:
raise ValueError

As a scientist, I prefer the second one because I naturally read it as:
"condition is true, therefore ..." and I can predict the next step of the
algorithm naturally by filling in the "..." in my mind.  It makes the
assumptions of the code a bit more explicit than the code `if not
condition:`, which I must think about and to translate to "the
condition must be true" before I can continue reading.

That, in addition to being able to disable the assert statements, makes me
like and use them a reasonable amount. However, every time I do use them, I
always think "crap, when this breaks I'm going to have to come back here
and read my code/comments because the error message isn't very helpful",
and that makes me not want to write assert statements.  So I like writing
them because while I'm writing/reading the code, they make sense, but I
don't like reading their error message output because it isn't useful to me
as a user/developer.

I realize this is very minor, but I actually really like it, and I think
the below syntax would be pretty nice and backwards compatible:

assert condition:
raise ValueError

Jason
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

2017-11-27 Thread Nathan Schneider

On Mon, Nov 27, 2017 at 9:28 PM, Ivan Pozdeev via Python-ideas <
python-ideas@python.org> wrote:

> On 28.11.2017 5:19, Chris Angelico wrote:
>
> Actually, Python does have a way of disabling assertions (the -O
>> flag), so they should be treated the same way they are in C.
>> Assertions should not be used as shorthands for "if cond: raise Exc"
>> in the general case.
>>
> I'm claiming, and provided evidence, that there are no use cases for this
> in Python, so no-one (of any significance) will suffer when the disabling
> is cut out.
> In any case, we will probably do it with a __future__ statement for a
> transitional period.
>
>
I think it would be interesting to investigate how assert statements are
used in the wild. I can think of three kinds of uses:

1) Nonredundant checking: The assertion aims to be a concise way to raise
exceptions under certain conditions that might arise even presuming that
the code is internally correct. (For example, to check that the interface
with another piece of code is as expected, or to validate user input.)

This seems to be the use case the OP is targeting. It is probably not the
use envisioned by the designers of the assert statement. But perhaps assert
is frequently used in this way by people who don't know about the
optimization flag or assume their program won't be run with optimization.

2) Redundant checking: The assertion acts as a test of code correctness. If
the code is correct, it should have no effect; and presuming the code is
well-tested, it can thus be ignored with optimization turned on. But it is
useful as a sanity check of a tricky algorithm, and can help a reader to
understand the implementation. Is it often the case that optimizing these
assertions away brings significant performance savings?

3) Temporary debugging: I often write something like assert False,(x,y,z)
as a quick way to force the program to terminate and print some values.
Unlike print(x,y,z), this has the advantage of being blatantly obvious as
debugging code that does not belong in the final version.

It seems unlikely that one would purposefully turn on optimization in
conjunction with temporary assertions, so this use case may be irrelevant
to the proposal.

Best,
Nathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using an appropriate tone in emails (was: Adding a thin wrapper class around the functions in stdlib.heapq)

2017-11-27 Thread bunslow

On Mon, Nov 27, 2017 at 4:36 PM, Paul Moore  wrote:

> On 27 November 2017 at 21:59, Nick Timkovich 
> wrote:
> > On Mon, Nov 27, 2017 at 8:17 PM, Brett Cannon  wrote:
> >>
> >> But calling it "atrocious" and so bad that it needs to be fixed
> >> "immediately" as if it's a blight upon the stdlib is unnecessarily
> insulting
> >> to those that have worked on the module. To convey the feeling that you
> >> think an OO wrapper would be helpful as the current design doesn't work
> for
> >> you, you could just phrase it as I just did to get the same point across
> >> without insulting anyone. Basically if you wouldn't like your own work
> >> called "atrocious" by someone you respect, then it's probably best to
> not
> >> use that phrasing when talking about a stranger's code either.
> >
> >
> > Sorry for the curt tone, I did lose some sight on the code being
> designed by
> > people rather than a faceless organization. My intention wasn't to
> disparage
> > the original authors but sprung more out of my frustration and perception
> > from that thread and those before that the status quo would not change
> and
> > that if a contribution was proffered, would simply be dismissed or
> ignored.
> > To motivate any change, there must be some argument levied against the
> > status quo, but hopefully I can articulate it better.
> >
> > That little corner is something I'm interested in, and not having
> > contributed to CPython before, I'm unsure how it "really works". The
> steps
> > at https://devguide.python.org/stdlibchanges/ suggest trying to elicit
> > community feedback from the lists as a step, so negative feedback tends
> to
> > kill the enthusiasm to actually make the PR. In the absence of code,
> > concrete arguments are almost impossible as we're discussing the shape of
> > clouds.
>
> In my experience (and this reiterates Brett's point) the proposals
> that get the best responses are those that are presented positively -
> instead of focusing on the (perceived) problems with the current
> situation, describe the benefits that will come from the proposed
> change. If you can't do that, then it's unlikely there is enough
> justification for a change. Certainly, negative feedback can be
> demotivating, and when you have a great idea and all you hear is "but
> what if...?" it's hard to remain positive. But you're not going to get
> better feedback if you criticise - at best, people will stop
> listening, and you'll have avoided some of the arguments, but at the
> cost of no-one being willing to support your proposal and so it dies.
>

My first submission to this list was predicated on what I'd read in PEPs --
and many of those, since they recommend major-enough changes to require a
PEP, have sections (often lengthy) dedicated to "what's wrong with the
status quo". My attempt to imitate that obviously crossed some boundaries
in retrospect, and of course now that it's brought up here I see that
spinning it as "what can be done to make it better" is psychologically much
more effective than "why the current way sucks" (because semantically these
are either approximately or exactly the same). But that's where it came
from, at least with some of my earlier threads, and I suspect the author of
the topic message of the OP will have a similar sentiment.

(One major example I can point to is PEP 465 -- because it proposed such a
major change to the language, literally half its text amounts to "what's
wrong with the status quo", quantifiably and repeatedly. It was also a
highly persuasive PEP due in no small part to its "why current things suck"
section.)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

2017-11-27 Thread Ned Batchelder

On 11/27/17 9:12 PM, Ivan Pozdeev via Python-ideas wrote:
The `assert' statment was created the same as in previous languages
like C/C++: a check to only do in debug mode, when you can't yet trust
your code to manage and pass around internal data correctly. Examples
are array bounds and object state integrity constraints.

Unlike C, Python does the aforementioned checks all the time, i.e.
it's effectively always in "debug mode". Furthermore, validation
checks are an extremily common use case.

I have been using assert in my in-house code for input validation for
a long time, the only thing preventing me from doing the same in
public code is the fact that it only raises AssertionError's while
library code is expected to raise TypeError or ValueError on invalid
input.

The last drop that incited me to do this proposition was
https://stackoverflow.com/questions/2142202/what-are-acceptable-use-cases-for-pythons-assert-statement
where none of the _seven_ answer authors (beside me) managed to make a
_single realistic use case_ for `assert' as a debug-only check.

---

So, I'm hereby proposing to:

* make `assert' stay in optimized mode
* change its syntax to raise other types of exceptions

E.g.: "assert condition, type, value" or "assert condition, type,
exception_constructor_argument(s)" to maintain backward compatibility.

You are proposing:

assert condition, type, value

Why not just use Python as it is now, with this:?

if not condition: raise type(value)

I don't see a reason to change the assert statement. You can do what you
need without the change.

--Ned.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check


On 28.11.2017 6:34, Ned Batchelder wrote:



You are proposing:

    assert condition, type, value

Not specifically this, that's just an example.

Actually, the way I'm using them,

    assert condition, "error message", type

would probably be the most expressive way.


Why not just use Python as it is now, with this:?

It's the most expressive way the language provides to write that logic.
With Python's design focus on promoting expressive, readable and 
intuitive syntax, that's enough of a reason.


    if not condition: raise type(value)

I don't see a reason to change the assert statement. You can do what 
you need without the change.
I can do anything in any Turing-complete language without any changes to 
the language. That's no reason to never change anything, is it.


The rationale basically is:
* As it was intended, the statement has no practical use -- basically a 
rudiment, due to disappear eventually

* It can instead be reused as syntax sugar to cover a very common use case

--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?


Kirill Balunov wrote:
So someone too perlish 
clever can assume that with the proposed syntax:


 >>> def gen():
 >>> for i in ['a', 'b', 'c', 'd']:
 >>> v = x if 'x' in globals() else 'var '
 >>> yield v + i

 >>> x, y = gen()
 >>> x
var a
 >>> y
var ab


There's no need for that to happen. It can still unpack all the values
it needs before performing any assignments.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?


Steven D'Aprano wrote:

How does "stop iterating here" equate to a wildcard?


The * means "I don't care what else the sequence has in it".

Because I don't care, there's no need to iterate any further.

--
Greg

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

On Tue, Nov 28, 2017 at 05:12:36AM +0300, Ivan Pozdeev via Python-ideas wrote:
> The `assert' statment was created the same as in previous languages like 
> C/C++: a check to only do in debug mode, when you can't yet trust your 
> code to manage and pass around internal data correctly.

That's not the sole use for assertions. Assertions are useful as checked 
comments, as "this can't happen" checks, as checks on your algorithm 
logic, as a poor man's form of design by contract, and more.

> Unlike C, Python does the aforementioned checks all the time, i.e. it's 
> effectively always in "debug mode".

Apart from -O which disables assertions. But in any case, the best use 
of assertions is not checking things which the interpreter is going to 
do anyway, but checking things which the interpreter can not and does 
not check automatically: your program logic. There is no way that the 
Python interpreter is going to do this check automatically, unless I 
write the assertion:

assert 0 <= r < abs(y)

That is copied straight out of one of my functions.

The point is, if you are only using assertions to check for things which 
the interpreter already does, you're not making good use of assertions.

> Furthermore, validation checks are an extremily common use case.
> I have been using assert in my in-house code for input validation for a 
> long time, 

I maintain that using assert for input validation for public functions 
(even if "public" in this case remains in-house) is certainly an abuse 
of assert, and STRONGLY oppose any change that supports that bad habit. 
Its okay to use asserts to validate input to *private* functions and 
methods (although even that is a slight code smell) but not public 
functions.

> So, I'm hereby proposing to:
> 
> * make `assert' stay in optimized mode
> * change its syntax to raise other types of exceptions

Very strong -1 on this. We already have syntax for raising other sorts 
of exceptions: raise.

if condition: raise Exception(message)

is the right way to spell 

assert not condition, Exception, message

The two require virtually the same amount of typing, so you don't even 
save any effort by using assert with an alternate exception type.

Leave assert for assertions. It is a *good* thing that assertions are 
visually distinct from input error checking. As things stand now, assert 
is a clear signal to the reader that this is an internal check which may 
be disabled safely.

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

On Tue, Nov 28, 2017 at 07:35:45AM +0300, Ivan Pozdeev via Python-ideas wrote:

> Actually, the way I'm using them,
> 
>     assert condition, "error message", type
> 
> would probably be the most expressive way.

I disagree that is expressive -- I call it *misleading*. I see something 
which looks like an assertion (that is, a checked comment, a contract, a 
check on an internal piece of logic etc) but it is actually being used 
as a test.

> I can do anything in any Turing-complete language without any changes to 
> the language. That's no reason to never change anything, is it.

"We can change this" is not a reason to change this. There needs to be a 
*good* reason to change, and you have given no good reasons for this 
change.

> The rationale basically is:
> * As it was intended, the statement has no practical use -- basically a 
> rudiment, due to disappear eventually

Nonsense. I make extensive use of assert as a way of checking 
assertions, and I will fight tooth and nail against any proposal to 
either remove it or to misuse it for public input tests instead of 
assertions.

> * It can instead be reused as syntax sugar to cover a very common use case

There is no need for such syntactic sugar. It would be harmful 
to use assert for something which is not an assertion.

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] generator vs iterator etc. (was: How assignment should work with generators?)

2017-11-27 Thread Stephen J. Turnbull

Steven D'Aprano writes:

 > The subset of iterators which are created as generators are *also* 
 > called generators,

As long as we're being precise, I don't think that is precisely correct:

>>> (x for x in range(1))
 at 0x10dee5e08>
>>> iter(range(1))

>>> iter((1,))

The two iterators have the same duck-type, the generator is different.
A generator (object) is, of course, an interable.

 > You are right that sometimes the term "generator" is used as
 > shorthand for "generator function".

I've always thought "generator factory" would be a better term, but
"generator function" will do.  I generally use "generator object" to
make the distinction, though.

 > Most of the time the distinction doesn't actually matter, since you
 > cannot (easily?) create a generator without first creating a
 > generator function.

At least you can create a generator (object) with the generator
function created and called implicitly by using a generator
expression.

Reverting from pedantic mode.  Hear, hear! this:

 > Or if it does matter, it is clear in context which is meant.
 > 
 > For those few times where it *does* matter, there is no substitute for 
 > precision in language, and that depends on the author, not the 
 > terminology.

Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Mon, Nov 27, 2017 at 06:31:28PM -0800, Mike Miller wrote:

> Believe the question behind the idea was, how to grab a couple items and 
> then *stop?*  If the syntax route is chosen, I'd expect something that 
> tells me it is going to stop, like a "full stop" as the period/dot is 
> called in jolly ol' England, e.g.:
> 
> x, y, . = iterable

Sadly, that fails the "syntax should not look like grit on Tim's 
monitor" test. Ellipsis at least has three pieces of grit in sequence, 
which makes it more noticable.


> Not sure about the second comma though.

Without the comma, it will be visually too hard to distinguish from 

x, y , = iterable


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread Steve Barnes



On 27/11/2017 21:49, Guido van Rossum wrote:
> On Mon, Nov 27, 2017 at 1:32 PM, Chris Angelico  > wrote:
> 
> On Tue, Nov 28, 2017 at 8:24 AM, Guido van Rossum  > wrote:
> > On Mon, Nov 27, 2017 at 1:18 PM, Greg Ewing 
> mailto:greg.ew...@canterbury.ac.nz>>
> > wrote:
> >>
> >> Chris Angelico wrote:
> >>>
> >>> The problem is that it depends on internal whitespace to
> >>> distinguish it from augmented assignment;
> >>
> >>
> >> Ah, didn't spot that. I guess the ellipsis is the next best
> >> thing then.
> >>
> >> An alternative would be to require parens:
> >>
> >>    (x, y, *) = z
> >
> >
> > But that would have the same issue.
> >
> > Is this problem really important enough that it requires dedicated 
> syntax?
> > Isn't the itertools-based solution good enough? (Or failing that, 
> couldn't
> > we add something to itertools to make it more readable rather than going
> > straight to new syntax?)
> 
> I don't think there's much that can be done without syntax; the
> biggest problem IMO is that you need to tell islice how many targets
> it'll be assigned into. It needs some interpreter support to express
> "grab as many as you have targets for, leaving everything else behind"
> without stating how many that actually is. So the question is whether
> that is sufficiently useful to justify extending the syntax. There are
> a number of potential advantages and several competing syntax options,
> and this suggestion keeps coming up, so I think a PEP is warranted.
> 
> 
> OK, that's reasonable, and at first blush the ellipsis proposal looks 
> okay. My PEP queue for Python 3.7 is full though, so I would like to put 
> this off until 3.8.
> 
> -- 
> --Guido van Rossum (python.org/~guido 

Can we please take a note to ensure any future PEP clearly states which 
ellipsis (personally I prefer the first) of:

  - as 3 consecutive full stop characters (U+002E) i.e. ...
  - the Chicago system of 3 space separated full stops . . .
  - Unicode Horizontal ellipsis (U+2026) (at least there is a keyboard 
short cut for this) …
  - Unicode Midline horizontal ellipsis (U+22EF) ⋯
  - any of the other ellipsis characters 
(https://en.wikipedia.org/wiki/Ellipsis#Computer_representations)

As clarifying this early could save a lot of later discussion such as 
the recent minus, hyphen, underscore debate.

-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect 
those of my employer.

---
This email has been checked for viruses by AVG.
http://www.avg.com

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread electronnn

How about x, y ?= iterable with ?= being the lazy assignment operator?

On Tue, Nov 28, 2017 at 9:45 AM, Steven D'Aprano 
wrote:

> On Mon, Nov 27, 2017 at 06:31:28PM -0800, Mike Miller wrote:
>
> > Believe the question behind the idea was, how to grab a couple items and
> > then *stop?*  If the syntax route is chosen, I'd expect something that
> > tells me it is going to stop, like a "full stop" as the period/dot is
> > called in jolly ol' England, e.g.:
> >
> > x, y, . = iterable
>
> Sadly, that fails the "syntax should not look like grit on Tim's
> monitor" test. Ellipsis at least has three pieces of grit in sequence,
> which makes it more noticable.
>
>
> > Not sure about the second comma though.
>
> Without the comma, it will be visually too hard to distinguish from
>
> x, y , = iterable
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] generator vs iterator etc. (was: How assignment should work with generators?)

On Tue, Nov 28, 2017 at 03:11:25PM +0900, Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
> 
>  > The subset of iterators which are created as generators are *also* 
>  > called generators,
> 
> As long as we're being precise, I don't think that is precisely correct:
> 
> >>> (x for x in range(1))
>  at 0x10dee5e08>
> >>> iter(range(1))
> 
> >>> iter((1,))
> 
> 
> The two iterators have the same duck-type, the generator is different.

How is the generator different? It quacks like a range_iterator and 
tuple_iterator, it swims like them, it flies like them. Is there some 
iterator method or protocol that generators don't support?


> A generator (object) is, of course, an interable.

And also an iterator:

py> collections.abc
py> isinstance((x+1 for x in range(5)), collections.abc.Iterator)
True


>  > Most of the time the distinction doesn't actually matter, since you
>  > cannot (easily?) create a generator without first creating a
>  > generator function.
> 
> At least you can create a generator (object) with the generator
> function created and called implicitly by using a generator
> expression.

Ah yes, I forget about generator expressions, thanks.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

On Tue, Nov 28, 2017 at 06:15:47PM +1300, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >How does "stop iterating here" equate to a wildcard?
> 
> The * means "I don't care what else the sequence has in it".
>
> Because I don't care, there's no need to iterate any further.

I'll grant you that. But I don't see how that relates to being a 
wildcard. I'm not seeing the connection. I mean, you wouldn't interpret

   from module import *

to mean "I don't care what's in the module, so don't bother importing 
anything" would you? So the concept of wildcard here seems to be the 
opposite to its use here:

- in imports, it means "import everything the module offers";
- in extended iterable unpacking, it means "collect everything";

both of which (to me) seem related to the concept of a wildcard; but in 
this proposed syntax, we have

   x, y, * = iterable

which means the opposite to "collect everything", instead meaning 
"collect nothing and stop".

Anyway, given that * = is visually ambiguous with *= I don't think this 
syntax is feasible (even though * = is currently a syntax error, or at 
least it is in 3.5).

-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using an appropriate tone in emails (was: Adding a thin wrapper class around the functions in stdlib.heapq)

2017-11-27 Thread Nathaniel Smith

On Mon, Nov 27, 2017 at 7:22 PM, bunslow  wrote:
> My first submission to this list was predicated on what I'd read in PEPs --
> and many of those, since they recommend major-enough changes to require a
> PEP, have sections (often lengthy) dedicated to "what's wrong with the
> status quo". My attempt to imitate that obviously crossed some boundaries in
> retrospect, and of course now that it's brought up here I see that spinning
> it as "what can be done to make it better" is psychologically much more
> effective than "why the current way sucks" (because semantically these are
> either approximately or exactly the same). But that's where it came from, at
> least with some of my earlier threads, and I suspect the author of the topic
> message of the OP will have a similar sentiment.

To quote Brett's original email:
> So obviously Nick doesn't like the design of the heapq module. ;) And that's 
> okay! And he's totally within his rights to express the feeling that the 
> heapq module as it stands doesn't meet his needs.
> But calling it "atrocious" and so bad that it needs to be fixed "immediately" 
> as if it's a blight upon the stdlib is unnecessarily insulting to those that 
> have worked on the module.

You can and should talk about problems with the status quo! But it's
totally possible to do this without insulting anyone. Brett's talking
about tone, not content.

> (One major example I can point to is PEP 465 -- because it proposed such a
> major change to the language, literally half its text amounts to "what's
> wrong with the status quo", quantifiably and repeatedly. It was also a
> highly persuasive PEP due in no small part to its "why current things suck"
> section.)

Maybe, but you won't find the word "suck" anywhere in that section
:-). And of course, the nice thing about PEP 465 is that it's
complaining about a missing feature, which sort of by definition means
that it's not complaining about anyone in particular's work.

Nonetheless, an earlier draft of PEP 465 did inadvertently talk about
an old PEP in an overly-flippant manner, and I ended up apologizing to
the author and fixing it. (Which of course also made the PEP
stronger.) It's cool, no-one's perfect. If you think you've made a
mistake, then apologize and try to do better, that's all.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

2017-11-27 Thread C Anthony Risinger

On Nov 28, 2017 12:32 AM, "Steven D'Aprano"  wrote:

On Tue, Nov 28, 2017 at 06:15:47PM +1300, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >How does "stop iterating here" equate to a wildcard?
>
> The * means "I don't care what else the sequence has in it".
>
> Because I don't care, there's no need to iterate any further.

I'll grant you that. But I don't see how that relates to being a
wildcard. I'm not seeing the connection. I mean, you wouldn't interpret

   from module import *

to mean "I don't care what's in the module, so don't bother importing
anything" would you? So the concept of wildcard here seems to be the
opposite to its use here:

- in imports, it means "import everything the module offers";
- in extended iterable unpacking, it means "collect everything";

both of which (to me) seem related to the concept of a wildcard; but in
this proposed syntax, we have

   x, y, * = iterable

which means the opposite to "collect everything", instead meaning
"collect nothing and stop".

Anyway, given that * = is visually ambiguous with *= I don't think this
syntax is feasible (even though * = is currently a syntax error, or at
least it is in 3.5).

If not already considered, what if the RHS had to be explicitly unpacked?

Something like:

a, b, c = *iterator

Which would essentially be:

a, b, c = (*iterator,)

This enables lazy assignment by default but `*` can force complete
expansion (and exact matching) of the RHS.

It's a breaking change, but it does have a straightforward fix (simply wrap
and unpack any relevant RHS).

Thanks,

-- 

C Anthony
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check


On 28.11.2017 8:59, Steven D'Aprano wrote:

On Tue, Nov 28, 2017 at 07:35:45AM +0300, Ivan Pozdeev via Python-ideas wrote:


Actually, the way I'm using them,

     assert condition, "error message", type

would probably be the most expressive way.

I disagree that is expressive -- I call it *misleading*. I see something
which looks like an assertion (that is, a checked comment, a contract, a
check on an internal piece of logic etc) but it is actually being used
as a test.



I can do anything in any Turing-complete language without any changes to
the language. That's no reason to never change anything, is it.

"We can change this" is not a reason to change this. There needs to be a
*good* reason to change, and you have given no good reasons for this
change.



The rationale basically is:
* As it was intended, the statement has no practical use -- basically a
rudiment, due to disappear eventually

Nonsense. I make extensive use of assert as a way of checking
assertions, and I will fight tooth and nail against any proposal to
either remove it or to misuse it for public input tests instead of
assertions.
I invite you to show me a single use case for those "assertions" because 
after ~20 years of experience in coding (that included fairly large 
projects), I've yet to see one.


Any, every check that you make at debug time either
* belongs in production as well (all the more because it's harder to 
diagnose there), or
* belongs in a test -- something coded independently from the program 
(if your code as a whole cannot be trusted, how any specific part of it 
can?), or
* isn't needed at all because a fault will inevitably surface somewhere 
down the line (as some exception or an incorrect result that a test will 
catch).


Finally, I've got much experience using existing code outside its 
original use cases, where the original author's assumptions may no 
longer hold but the specific logic can be gauded to produce the desired 
result. Coding these assumptions in would undermine that goal.


So, I see "debug assertions" as either intentionally compromizing 
correctness for performance (a direct opposite of Python's design 
principles), or as an inferiour, faulty, half-measure rudiment from 
times when CI wasn't a thing (thus not something that should be taught 
and promoted as a best practice any longer).



* It can instead be reused as syntax sugar to cover a very common use case

There is no need for such syntactic sugar. It would be harmful
to use assert for something which is not an assertion.





--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

2017-11-27 Thread Elazar

Just a note : in typechecked code (such as mypy's source code) assert is
used to guide the checker:

assert isinstance(x, CallableType)
return x.args  # checker knows it's valid

So the assert becomes a kind of type annotation. The runtime check helps
during tests, but is not that important - failure will be caught relatively
soon. And I believe that the ability to remove the check at runtime is
important, since isinstance calls have non-negligible impact on performance
in mypy.
(but other contributors here can correct me on this).

Elazar

בתאריך יום ג׳, 28 בנוב׳ 2017, 09:12, מאת Ivan Pozdeev via Python-ideas ‏<
python-ideas@python.org>:

> On 28.11.2017 8:59, Steven D'Aprano wrote:
> > On Tue, Nov 28, 2017 at 07:35:45AM +0300, Ivan Pozdeev via Python-ideas
> wrote:
> >
> >> Actually, the way I'm using them,
> >>
> >>  assert condition, "error message", type
> >>
> >> would probably be the most expressive way.
> > I disagree that is expressive -- I call it *misleading*. I see something
> > which looks like an assertion (that is, a checked comment, a contract, a
> > check on an internal piece of logic etc) but it is actually being used
> > as a test.
> >
> >
> >> I can do anything in any Turing-complete language without any changes to
> >> the language. That's no reason to never change anything, is it.
> > "We can change this" is not a reason to change this. There needs to be a
> > *good* reason to change, and you have given no good reasons for this
> > change.
> >
> >
> >> The rationale basically is:
> >> * As it was intended, the statement has no practical use -- basically a
> >> rudiment, due to disappear eventually
> > Nonsense. I make extensive use of assert as a way of checking
> > assertions, and I will fight tooth and nail against any proposal to
> > either remove it or to misuse it for public input tests instead of
> > assertions.
> I invite you to show me a single use case for those "assertions" because
> after ~20 years of experience in coding (that included fairly large
> projects), I've yet to see one.
>
> Any, every check that you make at debug time either
> * belongs in production as well (all the more because it's harder to
> diagnose there), or
> * belongs in a test -- something coded independently from the program
> (if your code as a whole cannot be trusted, how any specific part of it
> can?), or
> * isn't needed at all because a fault will inevitably surface somewhere
> down the line (as some exception or an incorrect result that a test will
> catch).
>
> Finally, I've got much experience using existing code outside its
> original use cases, where the original author's assumptions may no
> longer hold but the specific logic can be gauded to produce the desired
> result. Coding these assumptions in would undermine that goal.
>
> So, I see "debug assertions" as either intentionally compromizing
> correctness for performance (a direct opposite of Python's design
> principles), or as an inferiour, faulty, half-measure rudiment from
> times when CI wasn't a thing (thus not something that should be taught
> and promoted as a best practice any longer).
> >
> >> * It can instead be reused as syntax sugar to cover a very common use
> case
> > There is no need for such syntactic sugar. It would be harmful
> > to use assert for something which is not an assertion.
> >
> >
> >
>
> --
> Regards,
> Ivan
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?