[Python-ideas] Re: A proposal (and implementation) to add assignment and LOAD overloading

[David Mertz]

On Thu, Jun 27, 2019 at 9:13 AM Steven D'Aprano  wrote:

> The only risk here is if your refactoring does something silly, such as
> reusing a variable which overrides assignment:
>

How would you propose to write this code without reusing a variable?

def frobnicate(data):
stuff = []
for datum in data:
f = process_the(data)
g = munge_result(f)
stuff.append(g)
return stuff

Under the proposal, we have no guarantee that each iteration through the
loop will give us a new `f` to work with.  Maybe yes, maybe no.  Without
looking at the source code in `process_the()` we have no way of knowing
whether `f` is being bound to a new object each time or some completely
different arbitrary action.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5IQOHCTV5F4KYHTWSYHYCF6PERZKOJ3S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Comparison operator support (>= and <=) for type

[David Mertz]

On Wed, Jun 19, 2019, 7:35 PM Jeffrey Kintscher 
wrote:

> It is not at all intuitive that
>
> issubclass(A, (B, C))
>
> means "Is A a subclass of B or a subclass of C?" when it could also mean
> "Is A a subclass of both B and C?".
>

I've used issubclass() maybe 20 times in more than 20 years programming
Python. I know the correct meaning.

But sure, I can imagine not knowing which of the two things you mention is
the meaning. Either could be useful. How many times would you need to look
it up?!

For folks who actually use issubclass() fairly often, that came be a
burden. For those of us who rarely use it, consulting the docs once a year
also isn't a burden.

This just doesn't warrant new syntax.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HMNPRFVIFQ7EY3XGQ2PGZFKTPYSZKX5T/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Operator as first class citizens -- like in scala -- or yet another new operator?

[David Mertz]

On Sun, May 26, 2019, 1:12 PM Barry Scott  wrote:

> You said this: "Well, depends on how we define narrow ... you are writing
> probablythis email on a HDL designed machine ... and the entire world is
> powered by HDL designed silicons. that is not small for me at all."
>
> Which I take to mean that because there are billions of chips in the world
> there are billions of python users. And the change you want is justified
> by the billions of python users.
>

I think the analogy is more like saying there are billions of Verilog
users. Chips designed using this software are probably more numerous than
are human beings, and nearly all humans use one or more such chips. This
seems like a poor line of reasoning though. There are really only maybe
tens of thousands of people in the world who use Verilog, and that is a
nice target.

Pursuing the same indirect reasoning, we could also claim there are
billions of users of PSS/E, something I had not heard of until a web search
a few minutes ago. Apparently this is software used in design of electrical
distribution systems, required to make all of those ICs operate at all. Or
software used for building the construction equipment needed to construct
the power lines and factories where chips are made. Or...

In any case, as I said, modifying the whole Python language to allow use of
exactly the same symbols as Verilog users, seems foolish to me.

That said, '<=' is already perfectly well available for non-blocking
assignment. '=' cannot be overriden, but it occurs to me that a custom
class could perfectly well use '==' for blocking assignment just by
customizing a class. Yes, that would require 'foo.eq(bar)' for the old
meaning of equality (or something similar), but it seems like assignment is
more common than equality checks in Verilog.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Operator as first class citizens -- like in scala -- or yet another new operator?

[David Mertz]

I don't really understand HDL/Verilog, but I've worked with people who do.
In fact, I even wrote a pre-processor that transformed the same DSL to
Python, C++, and Verilog.

In my mind, the HDL use case is FAR too narrow and specialized to warrant a
new arrow operator, let an entirely new parser and semantics around
arbitrary operators. There are several existing dunders that could
plausibly be repurposed already (<<, <<=, <=, etc). Those might look
sightly different than the verilog operators, but that's a very small
price. In fact, just using attributes and assignment is an incredibly low
bar too, and allows whatever overriding you wish.

I just don't buy the idea that such a DSL can only be useful if it spells
'abc <== message' and useless if it spelled the same thing as 'abc <<=
message'.


On Fri, May 24, 2019, 9:06 AM Yanghao Hua  wrote:

> On Fri, May 24, 2019 at 12:29 PM Greg Ewing 
> wrote:
> >
> > Yanghao Hua wrote:
> > > I have explained the problem of use
> > > descriptors in previous replies, where you cannot have a local signal,
> > > e.g. obj.signal = thing # works, but local_signal = thing # doesn't
> > > work.
> >
> > Maybe you could do something like:
> >
> > local = Signals()
> > local.signal1 = ...
> > local.signal2 = ...
>
> In structure design ... and especially when you design a hardware that
> is meant to be automatically converted into verilog or even logic
> gates, I personally would really want to have a one-to-one
> relationship of the python-objects vs the actual hardware structures.
> The granularity is at signal/bit level. This is why I really think
> giving a special assignment in python which users could override is
> really helpful, rather than having to have this kind of special case:
> if you do "self.abc = thing" descriptor mechanism is invoked, but the
> very next line if you do "abc = thing" ... nothing will happen. This
> special case can be completely removed and having a much better
> conceptual consistency if the "<==" assignment operator always behaves
> the same, doesn't matter if it is "self.abc <== thing" or "abc <==
> thing".
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: Allowing any variable to be used in a 'with... as...' expression

[David Mertz]

Oh, I neglected to include the definition of my boring Foo class:

>>> class Foo(object):
... def __init__(self, val):
... self.val = val
...

The only important thing about it is that it DOES NOT have __enter__() or
__exit__() methods.

On Sat, May 18, 2019 at 9:40 PM David Mertz  wrote:

> On Sat, May 18, 2019 at 8:17 PM Yonatan Zunger  wrote:
>
>> Instead, we should permit any expression to be used. If a value does not
>> expose an __enter__ method, it should behave as though its __enter__
>> method is return self; if it does not have an __exit__ method, it should
>> behave as though that method is return False.
>>
>
> I'm not sure why you would want this.  But it is really easy to implement
> with a named function already.  I think it would be better to experiment
> with a wrapper function first, maybe put it on PyPI, before changing the
> language for something whose purpose is unclear (to me).
>
> I think you probably mean something other than what you actually write.
> It doesn't really make sense for "any expression" as far as I can tell.
> What would it possibly mean to write:
>
> with (2+2) as foo:
> print(foo)
>
> But anyway, my toy simple implementation which satisfies the stated
> requirement:
>
> >>> from contextlib import contextmanager
> >>> @contextmanager
> ... def zungerfy(obj):
> ... if hasattr(obj, '__enter__'):
> ... yield obj.__enter__()
> ... else:
> ... yield obj
> ... if hasattr(obj, '__exit__'):
> ... obj.__exit__()
> ...
> >>> with zungerfy(Foo(42)) as foo:
> ... print("Hello", foo.val)
> ...
> ...
> Hello 42
> >>> with zungerfy(open('README.md')) as foo:
> ... print("Hello", foo.readline())
>
> ...
> Hello ## About the course
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: Allowing any variable to be used in a 'with... as...' expression

[David Mertz]

>
> I think you probably mean something other than what you actually write.
> It doesn't really make sense for "any expression" as far as I can tell.
> What would it possibly mean to write:
>
> with (2+2) as foo:
> print(foo)
>

I have occasionally thought it would be nice to do something like this (and
I could, but I haven't, so I guess I don't think it that strongly):

>>> @contextmanager
... def bind(val):
... yield val
...
>>> with bind(2+2) as four:
... print(four)
...
4

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: Allowing any variable to be used in a 'with... as...' expression

[David Mertz]

On Sat, May 18, 2019 at 8:17 PM Yonatan Zunger  wrote:

> Instead, we should permit any expression to be used. If a value does not
> expose an __enter__ method, it should behave as though its __enter__
> method is return self; if it does not have an __exit__ method, it should
> behave as though that method is return False.
>

I'm not sure why you would want this.  But it is really easy to implement
with a named function already.  I think it would be better to experiment
with a wrapper function first, maybe put it on PyPI, before changing the
language for something whose purpose is unclear (to me).

I think you probably mean something other than what you actually write.  It
doesn't really make sense for "any expression" as far as I can tell.  What
would it possibly mean to write:

with (2+2) as foo:
print(foo)

But anyway, my toy simple implementation which satisfies the stated
requirement:

>>> from contextlib import contextmanager
>>> @contextmanager
... def zungerfy(obj):
... if hasattr(obj, '__enter__'):
... yield obj.__enter__()
... else:
... yield obj
... if hasattr(obj, '__exit__'):
... obj.__exit__()
...
>>> with zungerfy(Foo(42)) as foo:
... print("Hello", foo.val)
...
...
Hello 42
>>> with zungerfy(open('README.md')) as foo:
... print("Hello", foo.readline())

...
Hello ## About the course


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

On Sun, May 12, 2019, 5:36 PM Paul Moore  wrote:

> On Sun, 12 May 2019 at 21:06, David Mertz  wrote:
> > I thought of 'as' initially, and it reads well as English. But it felt
> to me like the meaning was too different from the other meanings of 'as' in
> Python. I might be persuaded otherwise.
>
> If you think in terms of "named loops" rather than "labels", "as"
> makes a lot more sense.


Every name created with 'as' currently is a regular Python object, with a
memory address, and some methods, that can participate in comparison (at
least equality), that has a type, can be passed as a function argument,
etc. Actually, all that is true of EVERY name in Python other than keywords.

It seems like none of this would be true of a loop name/label. Hence my
first thought that a different way of introducing the word is more clear...
Maybe we can just prefix lines with labels instead, and add a 'goto'
command :-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

On Sun, May 12, 2019, 3:33 PM Gustavo Carneiro  wrote:

> # Hypothetical future labelled break:
>> def find_needle_in_haystacks():
>> for haystack in glob.glob('path/to/stuff/*') label HAYSTACKS:
>> fh = open(fname)
>> header = fh.readline()
>> if get_format(header) == 'foo':
>> for line in fh:
>> if process_foo(line) == -1:
>> break HAYSTACKS
>> elif get_format(header) == 'bar':
>> for line in fh:
>> if process_bar(line) == -1:
>> break HAYSTACKS
>>
>
> Nice.  Suggestion:
> 1. replace "label HAYSTACKS" with "as HAYSTACKS", if possible, so no new
> keyword is introduced;
> 2. replace "break HAYSTACKS" with "break from HAYSTACKS"
>

I thought of 'as' initially, and it reads well as English. But it felt to
me like the meaning was too different from the other meanings of 'as' in
Python. I might be persuaded otherwise.

Likewise, 'from' isn't really very similar to other existing uses.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

I thought of "we could return immediately" shortly after I posted it. Yes,
that's true in the example I wrote. But it's easy enough to vary it so that
something happens after the loop as well. I also noticed that for the
simple version, it might be slightly shorter to put the conditional inside
the loop rather than to decide between loops.

There are lots of ways to work around the absence of a labelled break. Many
times putting the inner loop in a comprehension can simplify things too,
for example (that's roughly what my actual code was refactored to, mostly
not just because of the break issue).

On Sun, May 12, 2019, 3:38 PM Chris Angelico  wrote:

> On Mon, May 13, 2019 at 3:26 AM David Mertz  wrote:
> >
> > To be clear in this thread, I don't think I'm really ADVOCATING for a
> multi-level break.  My comments are simply noting that I personally fairly
> often encounter the situation where they would be useful.  At the same
> time, I worry about Python gaining sometimes-useful features that
> complicate the overall language and bring it closer to Perl's "everyone can
> write in their own style."
> >
> > So to answer Chris' question... well, i did find something recent, but
> it has a ton of other extraneous stuff.  But I've simplified to something
> that isn't that much different from my example from my tablet yesterday,
> but is fairly realistic still.
> >
> > I have an existing function that looks roughly like this:
> >
> > def find_needle_in_haystacks():
> > found_needle = False
> > for haystack in glob.glob('path/to/stuff/*'):
> > fh = open(fname)
> > header = fh.readline()
> > if get_format(header) == 'foo':
> > for line in fh:
> > status = process_foo(line)
> > if status = -1:
> > found_needle = True
> > break
> > elif get_format(header) == 'bar':
> > for line in fh:
> > status = process_bar(line)
> > if status = -1:
> > found_needle = True
> > break
> >
> > if found_needle:
> > break
> >
>
> Cool. How much code is there AFTER this loop? The most obvious way to
> handle this is to immediately 'return' instead of multi-level
> breaking.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

To be clear in this thread, I don't think I'm really ADVOCATING for a
multi-level break.  My comments are simply noting that I personally fairly
often encounter the situation where they would be useful.  At the same
time, I worry about Python gaining sometimes-useful features that
complicate the overall language and bring it closer to Perl's "everyone can
write in their own style."

So to answer Chris' question... well, i did find something recent, but it
has a ton of other extraneous stuff.  But I've simplified to something that
isn't that much different from my example from my tablet yesterday, but is
fairly realistic still.

I have an existing function that looks roughly like this:

def find_needle_in_haystacks():
found_needle = False
for haystack in glob.glob('path/to/stuff/*'):
fh = open(fname)
header = fh.readline()
if get_format(header) == 'foo':
for line in fh:
status = process_foo(line)
if status = -1:
found_needle = True
break
elif get_format(header) == 'bar':
for line in fh:
status = process_bar(line)
if status = -1:
found_needle = True
break

if found_needle:
break

If I had a "labelled break" feature, I might rewrite it something like the
following (picking convenient syntax that I'm not per-se advocating as
best, even if the concept is adopted):

# Hypothetical future labelled break:
def find_needle_in_haystacks():
for haystack in glob.glob('path/to/stuff/*') label HAYSTACKS:
fh = open(fname)
header = fh.readline()
if get_format(header) == 'foo':
for line in fh:
if process_foo(line) == -1:
break HAYSTACKS
elif get_format(header) == 'bar':
for line in fh:
if process_bar(line) == -1:
break HAYSTACKS

In any case, it's often possible to either (a) redefine the inner loop
>> as some sort of container operation, or (b) redefine the outer
>> operation as a function, and use "return". So it would be helpful to
>> have an example that CAN'T be rewritten in one of those ways, to use
>> as a compelling example.
>>
>
-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

I'll try to find a concrete one tomorrow. I did this recently, but I might
have factored it away somehow. The real life code has lots of extraneous
parts that need to be removed or simplified though. Both because they
aren't FLOSS, and because the details would muddy things. "Trimmed from
real" won't look much different than "invented from scratch".

And yes, OF COURSE everything CAN be rewritten. But that's true in regards
to *every* proposed addition. The sentinel break_outer variable is often
the was to do it without really big restructuring (but with a handful of
ugly bookkeeping lines instead).

On Sat, May 11, 2019, 11:22 PM Chris Angelico  wrote:

> On Sun, May 12, 2019 at 1:16 PM David Mertz  wrote:
> >
> > Ok, sure. But what if different seen_enough() conditions are in the two
> inner loops? By "break to outer" ... I mean, don't get any more 'main' from
> 'stuff'. I meant to dig through some real code, but forgot because I was
> writing code for work. This example on my tablet is the general pattern I
> often need though. And obviously I've solved it one way or another hundreds
> of times.
> >
> > I think a "labelled break" would be nice though. It might have even been
> a past PEP.
> >
>
> This is why concrete examples are much easier to discuss, heh.
>
> In any case, it's often possible to either (a) redefine the inner loop
> as some sort of container operation, or (b) redefine the outer
> operation as a function, and use "return". So it would be helpful to
> have an example that CAN'T be rewritten in one of those ways, to use
> as a compelling example.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

Ok, sure. But what if different seen_enough() conditions are in the two
inner loops? By "break to outer" ... I mean, don't get any more 'main' from
'stuff'. I meant to dig through some real code, but forgot because I was
writing code for work. This example on my tablet is the general pattern I
often need though. And obviously I've solved it one way or another hundreds
of times.

I think a "labelled break" would be nice though. It might have even been a
past PEP.

On Sat, May 11, 2019, 10:51 PM Chris Angelico  wrote:

> On Sun, May 12, 2019 at 12:43 PM David Mertz  wrote:
> >
> > Terry reminds me of a common case I encounter that cannot be transformed
> into a loop over itertools.product(). E.g.
> >
> > for main in stuff:
> > if thing_about(main):
> > for detail in more_stuff(stuff, main):
> > if seen_enough(detail):
> > # break to outer somehow
> > else:
> > for detail in different_stuff():
> > if seen_enough(detail):
> > # break to outer somehow
> >
>
> For that kind of loop, there's a different problem, which is the
> duplication. So I would start by rewriting that as:
>
> for main in stuff:
> if thing_about(main):
> details = more_stuff(stuff, main)
> else:
> details = different_stuff()
> for detail in details:
> if seen_enough(detail):
> ...
>
> I'm not sure what you mean by "break to outer somehow", though, so I
> don't know what you're actually needing here. Concrete examples would
> help.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

Terry reminds me of a common case I encounter that cannot be transformed
into a loop over itertools.product(). E.g.

for main in stuff:
if thing_about(main):
for detail in more_stuff(stuff, main):
if seen_enough(detail):
# break to outer somehow
else:
for detail in different_stuff():
if seen_enough(detail):
# break to outer somehow


On Sat, May 11, 2019, 6:58 PM Terry Reedy  wrote:

> On 5/11/2019 1:20 PM, haael wrote:
> >
> > Python allows for breaking out from a loop through 'break' and
> > 'continue' keywords. It would be nice if it was possible to break many
> > loop levels using one command.
> >
> > I propose two constructions for that:
> >
> > break
> > break break
> > break break break
> > ...
> >
> > continue
> > break continue
> > break break continue
> > ...
> >
> > And so on.
> >
> > Example:
> >
> > for i in range(10):
> >  for j in range(10):
> >  if i == 2 and j == 3:
> >  break break
> >
> > for i in range(10):
> >  for j in range(10):
> >  if i == 2 and j == 3:
> >  break continue
> >  if i == 7:
> >  break break
>
> > Breaking out from many loops at once is a common practice, currently
> > implemented through boolean testing and exceptions.
>
> Python exceptions generalize both 'return' and 'break' as methods for
> escaping nested contexts and are intended for generalized flow control.
>
> > This proposal would make it cleaner.
>
> I actually think that the general method is 'cleaner', be separating
> 'leave here' and 'arrive here'.  This proposal would mean typing fewer
> words in the special case where there are nested loops within one function.
>
> The cost is a) more syntax to learn and b) tightly tying together the
> two loops more tightly than might be necessary, and thereby inhibiting
> re-factoring.  In the examples above, where the loops are inherently
> tied together, the two loops can, as noted by others, be reduced to one.
>   If they are not inherently tied together, one might want to reduce the
> inner loop to a comprehension or wrap it in a new function so it can
> also be used elsewhere.  If the inner loop raises, wrapping or
> unwrapping it, as needed for reusability or efficiency, does not affect
> the logic.
>
> --
> Terry Jan Reedy
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Break multiple loop levels

[David Mertz]

I don't love the 'break break' syntax idea. But I find myself wanting to
break out of nested loops quite often. Very rarely could this be
transformed to an 'itertools.permutation' de-nesting, albeit perhaps more
often than I actually do that.

My usual kludge is a sentinel 'break_outer' variable that gets set back and
forth between True and False working the loops. Usually work a name that
descriptive of the actual domain of the code.

On Sat, May 11, 2019, 1:40 PM Paul Moore  wrote:

> On Sat, 11 May 2019 at 18:22, haael  wrote:
> > Breaking out from many loops at once is a common practice
>
> Do you have evidence for this? In my experience, it's very rare
> (although I concede it would be *occasionally* convenient).
>
> In most cases where I'd consider breaking out of multiple levels of
> nested loop, I've tended to find that it's probably more readable to
> refactor the code to use a generator, or something like that.
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: "?" Documentation Operator and easy reference to argument types/defaults/docstrings

[David Mertz]

On Thu, Apr 25, 2019 at 6:42 PM Peter O'Connor 
wrote:

> Despite the general beauty of Python, I find myself constantly violating
> the "don't repeat yourself" maxim when trying to write clear, fully
> documented code.  Take the following example:
>

You do know that OPTIONAL type annotations are optional, right.  There's no
requirement to repeat yourself if you don't want to.

But in general, comments or docstrings can do something very different from
just annotate one variable at a time.  If I want to describe the
interaction of `a` and `b`, that simply cannot fit in an annotation/comment
per parameter.  Of if one argument switches the relevance or meaning of
another, etc.


> - Argument/return documentation can be made inline with a new "?"
> operator.  Documentation becomes a first class citizen.
>

We already have a first-class citizen in annotations, this seems like extra
burden for little reason.


> def func_1(
> a: int = 1 ? 'Something about param a',
> b: float = 2.5 ? 'Something else about param b',
> ) -> float ? 'Something about return value of func_1':
> """ Something about func_1 """
> return a*b
>

Why not just this in existing Python:

def func_1(
a: int = 1 # 'Something about param a',
b: float = 2.5 # 'Something else about param b',
) -> float:
"""Something about func_1

a and b interact in this interesting way.
a should be in range 0 < a < 125
floor(b) should be a prime number

Something about return value of func_1
returns a multiplication
"""
return a*b


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Sorted lists

[David Mertz]

On Mon, Apr 8, 2019, 6:26 AM Steven D'Aprano  wrote:

> Given all the points already raised, I think that an explicit SortedList
> might be more appropriate.
>

This one looks cool. I've read about it, but haven't used it:
http://www.grantjenks.com/docs/sortedcontainers/

I think a "sort hint" should be a read-only attribute of some collections,
and statistics should look for the presence of that. You might need to get
third parties to follow that convention, but you're no worse than now if
they don't. Burdening list with a vague suggestion about an invariant it
doesn't maintain is a bad idea.

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Sorted lists

[David Mertz]

On Mon, Apr 8, 2019, 5:46 AM Paul Moore  wrote:

> Still not convinced this is safe enough to be worth it ;-)
>

I'm convinced it's NOT safe enough to be worth it.

On the other hand, a sortedlist subclass that maintained its invariant
(probably remembering a key) sounds cool. I think there are one or more
good ones on PyPI.

Statistics should simply learn to recognize external always-sorted data
structures.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

On Mon, Apr 1, 2019 at 8:54 PM Steven D'Aprano  wrote:

> I can think of at least one English suffix pair that clash: -ify, -fy.
> How about other languages? How comfortable are you to say that nobody
> doing text processing in German or Hindi will need to deal with clashing
> affixes?
>

 Here are the 30 most common suffixes in a large list of Dutch words.  For
similar answers for other languages, see
https://gist.github.com/DavidMertz/1a4aac0e889097d7bf80d8d41a3a644d.  Note
that there is absolutely nothing morphological here, simply dumb string
literals:

% head -30 suffix-frequency-nl.txt
('en', 55338)
('er', 14387)
('de', 12541)
('den', 11427)
('ten', 9402)
('te', 8263)
('ng', 7502)
('es', 7398)
('st', 7102)
('ing', 6949)
('gen', 6836)
('rs', 6592)
('ers', 5581)
('ren', 4842)
('el', 4602)
('ngen', 4451)
('rde', 4255)
('ken', 4203)
('re', 3870)
('je', 3868)
('len', 3784)
('ste', 3680)
('ie', 3658)
('nd', 3635)
('erde', 3620)
('rden', 3593)
('jes', 3307)
('eren', 3193)
('id', 3123)
('rd', 3083)



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

On Mon, Apr 1, 2019 at 10:11 PM Dan Sommers <
2qdxy4rzwzuui...@potatochowder.com> wrote:

> So I've seen someone (likely David Mertz?) ask for something
> like filename.strip_suffix(('.png', '.jpg')).  What is the
> context?  Is it strictly a filename processing program?  Do
> you subsequently have to determine the suffix(es) at hand?
>

Yes, I've sometimes wanted something like "the basename of all the graphic
files in that directory."  But here's another example that is from my
actually current job:

I do machine-learning/data science for a living. As part of that, I
generate a bunch of models that try to make predictions from the same
dataset.  So I name those models like:

dataset1.KNN_distance_n10.gz
dataset1.KNN_distance_n10_poly2_scaled.xz
dataset2.KNN_manhattan_n6.zip
dataset2.KNN_distance_n10_poly2_scaled.xz
dataset1.KNN_minkowski_n5.gz
dataset1.LinSVC_Poly3_Scaled.gz
dataset2.LogReg.bz2
dataset2.LogReg_Poly.gz
dataset1.NuSVC_poly2_scaled.gz


I would like to answer the question "What types of models have I tried
against the datasets?"  Obviously, I *can* answer this question.  But it
would be pleasant to answer it like this:

styles = {model.lstrip(('dataset1', 'dataset2'))
   .rstrip(('gz', 'xz', 'zip', 'bz2))

  for model in models}


That's something very close to code I actually have in production now.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

On Mon, Apr 1, 2019, 8:54 PM Steven D'Aprano  wrote:

> The point I am making is not that we must not ever support multiple
> affixes, but that we shouldn't rush that decision. Let's pick the
> low-hanging fruit, and get some real-world experience with the function
> before deciding how to handle the multiple affix case.
>

There are exactly two methods of strings that deal specifically with
affixes currently. Startswith and endswith. Both of those allow specifying
multiple affixes. That's pretty strong real-world experience, and breaking
the symmetry for no reason is merely confusing. Especially since the
consistency would be obviously as commonly useful.

Now look, the sky won't fall if a single-affix-only method is added. For
that matter, it won't if nothing is added. In fact, the single affix
version makes it a little bit easier to write a custom function handling
multiple affixes.

And the sky won't fall if the remove-just-one semantics are used rather
than remove-from-class.

But adding methods with sneakily helpful capabilities often helps users
greatly. A lot of folks in this thread didn't even know about passing a
tuple to str.startswith() a few days ago. I'm pretty sure that capability
was added by Raymond, who has an amazingly good sense of what little tricks
can prove really powerful. Apologies to a different developer if it wasn't
him, but congrats and thanks to you if so.

Somebody (I won't name names, but they know who they are) wrote to me
> off-list some time ago and accused me of being arrogant and thinking I know
> more than everyone else. Well perhaps I am, but I'm not so arrogant as to
> think that I can choose the right behaviour for clashing affixes for other
> people when my own use-cases don't have clashing affixes.
>

That could be me... Unless it's someone else :-). I think my intent was a
bit different than you characterize, but I'm very guilty of presuming too
much also. So mea culpa.

> Sure, but I've often wanted to do something like "strip off a prefix
> > of http:// or https://";, or something else that doesn't have a
> > semantic that's known to the stdlib.
>
> I presume there's a reason you aren't using urllib.parse and you just need
> a string without the leading scheme. If you're doing further parsing, the
> stdlib has the right batteries for that.
>

I know there are lots of specialized string manipulations in the STDLIB.
Yeah, I could use os.path.splitext, and os.path.split, and
urllib.parse.something, and lots of other things I rarely use. A lot of us
like to manipulate strings in generically stringy ways.

But not until we had a couple of releases of experience with them:
>
> https://docs.python.org/2.7/library/stdtypes.html#l.endswith
> 


Ok. Fair point. I used Python 2.4 without the multiple affix option.

Here's a partial list of English prefixes that somebody doing text
> processing might want to remove to get at the root word:
>
> a an ante anti auto circum co com con contra contro de dis
> en ex extra hyper il im in ir inter intra intro macro micro
> mono non omni post pre pro sub sym syn tele un uni up
>
> I count fourteen clashes:
>
> a: an ante anti
> an: ante anti
> co: com con contra contro
> ex: extra
> in: inter intra intro
> un: uni
>

This seems like a good argument for remove-all-from-class. :-)

stem = word.lstrip(prefix_tup)

But the we really need 'word.porter_stemmer()' as a built-in method.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Built-in parsing library

[David Mertz]

On Mon, Apr 1, 2019 at 2:16 AM Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> In principle, no, one just needs to explain why this battery fits most
> of the toys encountered in practice.  That's good enough, and if
> during discussion somebody shows another one is better on a lot of
> fronts, sure, do that instead.


OK, I'll acknowledge my comment might have overstated the bar to overcome.
A parser added to the standard library doesn't need to be perfect for
everyone.  But adding to stdlib *does* provide a kind of endorsement of the
right default way to go about things.

Among the popular third party libraries, we have several very different
attitudes towards designing grammars.  On the one hand, there are formal
differences in the power of different grammars—some are LL, some LR, some
LALR, some Early.  Also, some libraries separate parser from lexer while
others are scannerless.

But most of these can all parse the simple cases fine, so that's good for
the 90% coverage.  However, cross-cutting that formal power issue, there
are two main programming styles used by different libraries.  Some
libraries use BNF definitions of a grammar as another mini-language inside
Python.  Exactly where those BNF definitions live varies, but using them is
largely similar (i.e. are they in a separate file, in docstrings, contents
of variables, etc).  And sure, EBNF vs. BNF proper.  But other libraries
instead use Python functions or classes to define the productions, where
each class/function is effectively one term of the grammar.  Typically this
latter style allows triggering events as soon as some production is
encountered—the event could be "accumulate a counter", or "write an output
string", or "perform a computation", or other things.

There are lots of good arguments for why to use different libraries along
the axes I mention, on all sides.  What is not possible is to reconcile the
very different decisions into a common denominator.  Something in the
standard library would have to be partisan in selecting one particular
approach as the "official" one.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

On Sun, Mar 31, 2019, 9:35 PM Steven D'Aprano  wrote:

> > That's simply not true, and I think it's clearly illustrated by the
> example I gave a few times. Not just conceivably, but FREQUENTLY I write
> code to accomplish the effect of the suggested:
> >
> >   basename = fname.rstrip(('.jpg', '.gif', '.png'))
> >
> > I probably do this MORE OFTEN than removing a single suffix.
>
> Okay.
>
> Yesterday, you stated that you didn't care what the behaviour was for the
> multiple affix case. You made it clear that "any" semantics would be okay
> with you so long as it was documented. You seemed to feel so strongly about
> your indifference that you mentioned it in two seperate emails.
>

Yes. Because the multiple affix is an edge case that will rarely affect any
of my code. I.e. I don't care much when a single string had multiple
candidate affixes, because that's just not a common situation. That doesn't
mean I'm indifferent to the core purpose that I need frequently. Any of the
several possible behaviors in the edge case will not affect my desired
usage whatsoever.

That doesn't sound like someone who has a clear use-case in mind. If you're
> doing this frequently, then surely one of the following two alternatives
> apply:
>

I don't think I've ever written code that cares about the edge case you
focus on. Ok, I guess technically the code I've written is all buggy in the
sense that it would behave in a manner I haven't thought through when
presented with weird input. Perhaps I should always have been more careful
about those edges.

There simply is no "majority of the time" for a situation I've never
specifically coded for.

The rest gets more and more sophistical.  I'm sure most people here have
written code similar to this (maybe structured differently, but same
purpose):

for fname in filenames:
basename, ext = fname.rsplit('.', 1)
if ext in {'jpg', 'gif', 'png'}:
do_stuff(basename)

In all the times I've written things close to that, I've never thought
about files named 'silly.jpg.gif.png.gif.jpg'. The sophistry is insistently
asking "but what about...?" of this edge case.

For 29 years, we've done without this string primitive, and as a
> consequence the forums are full of examples of people misusing strip and
> getting it wrong.


It's interesting that you keep raising this error. I've made a whole lot of
silly mistakes in Python (and other languages). I have never for a moment
been tempted to think .rstrip() would remove a suffix rather than a
character class.

I did write the book Text Processing in Python a very long time ago, so
I've thought a bit about text processing in Python. Maybe it's just that
I'm comfortable enough with regexen that thinking of a character class
doesn't feel strange to me.

There's a clear case for the single argument version, and fixing that is
> the 90% solution.
>

I think there's very little case for a single argument version. At best,
it's a 10% solution.

Lacking a good set of semantics for removing multiple affixes at once, we
> shouldn't rush to guess what people want. You don't even know what
> behaviour YOU want, let alone what the community as a whole needs.
>

This is both dumb and dishonest. There are basically two choices, both
completely clear. I think the more obvious one is to treat several prefixes
or suffixes as substring class, much as .[rl]strip() does character class.

But another choice indeed is to remove at most one of the affixes. I think
that's a little bit less good for the edge case. But it would be fine
also... and as I keep writing, the difference would almost always be moot,
it just needs to be documented.

>
 the use-case of stripping a single file extension out of a set of
> such extensions, while leaving all others, there's an obvious solution:
>
> if fname.endswith(('.jpg', '.png', '.gif'):
> basename = os.path.splitext(fname)[0]
>

I should probably use of.path.splitext() more than I do. But that's just an
example. Another is, e.g. 'if url.startswith(('http://', 'sftp://',
's3://')): ...'. And lots of similar things that aren't addressed by
os.path.splitext(). E.g. 'if logline.startswith(('WARNING', 'ERROR')): ...'

I posted links to prior art. Unless I missed something, not one of those
> languages or libraries supports multiple affixes in the one call.
>

Also, none of those languages support the amazingly useful signature of
str.startswith(tuple). Well, they do in the sense they support regexen. But
not as a standard method or function on strings. I don't even know if PHP
with it's 5000 string functions had this great convenience.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

On Sun, Mar 31, 2019, 8:11 PM Steven D'Aprano  wrote:

> Regarding later proposals to add support for multiple affixes, to
> recursively delete the affix repeatedly, and to take an additional
> argument to limit how many affixes will be removed: YAGNI.
>

That's simply not true, and I think it's clearly illustrated by the example
I gave a few times. Not just conceivably, but FREQUENTLY I write code to
accomplish the effect of the suggested:

  basename = fname.rstrip(('.jpg', '.gif', '.png'))

I probably do this MORE OFTEN than removing a single suffix.

Obviously I *can* achieve this result now. I probably take a slightly
different approach as the mood strikes me, with three or four different
styles I've used. Actually, I've probably never done it in a way that
wouldn't be subtly wrong for cases like 'base.jpg.gif.png.jpg.gif'.

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Built-in parsing library

[David Mertz]

There are about a half dozen widely used parsing libraries for Python. Each
one of them takes a dramatically different approach to the defining a
grammar. Each one has been debugged for over a decade.

While I can imagine proposing one for inclusion in the standard library,
you'd have to choose one (or write a new one) and explain why that one is
better for everyone (or at least a better starting point) than all the
others are. You're also have to explain why it needs to be in the standard
library rather than installed by 'pip install someparser'.

On Sat, Mar 30, 2019, 1:58 PM Nam Nguyen  wrote:

> Hello list,
>
> What do you think of a universal parsing library in the stdlib mainly for
> use by other libraries in the stdlib?
>
> Through out the years we have had many issues with protocol parsing. Some
> have even introduced security bugs. The main cause of these issues is the
> use of simple regular expressions.
>
> Having a universal parsing library in the stdlib would help cut down these
> issues. Such a library should be minimal yet encompassing, and whole parse
> trees should be entirely expressible in code. I am thinking of combinatoric
> parsing as the main candidate that fits this bill.
>
> What do you say?
>
> Thanks!
> Nam
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Built-in parsing library

[David Mertz]

I just found this nice summary. It's not complete, but it looks well
written. https://tomassetti.me/parsing-in-python/

On Sun, Mar 31, 2019, 3:09 PM David Mertz  wrote:

> There are about a half dozen widely used parsing libraries for Python.
> Each one of them takes a dramatically different approach to the defining a
> grammar. Each one has been debugged for over a decade.
>
> While I can imagine proposing one for inclusion in the standard library,
> you'd have to choose one (or write a new one) and explain why that one is
> better for everyone (or at least a better starting point) than all the
> others are. You're also have to explain why it needs to be in the standard
> library rather than installed by 'pip install someparser'.
>
> On Sat, Mar 30, 2019, 1:58 PM Nam Nguyen  wrote:
>
>> Hello list,
>>
>> What do you think of a universal parsing library in the stdlib mainly for
>> use by other libraries in the stdlib?
>>
>> Through out the years we have had many issues with protocol parsing. Some
>> have even introduced security bugs. The main cause of these issues is the
>> use of simple regular expressions.
>>
>> Having a universal parsing library in the stdlib would help cut down
>> these issues. Such a library should be minimal yet encompassing, and whole
>> parse trees should be entirely expressible in code. I am thinking of
>> combinatoric parsing as the main candidate that fits this bill.
>>
>> What do you say?
>>
>> Thanks!
>> Nam
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

On Sun, Mar 31, 2019 at 12:09 PM MRAB  wrote:

> > That said, I really like Brandt's ideas of expanding the signature of
> > .lstrip/.rstrip instead.
> >
> > mystring.rstrip("abcd") # remove any of these single character suffixes
>
> It removes _all_ of the single character suffixes.
>
> > mystring.rstrip(('foo', 'bar', 'baz')) # remove any of these suffixes
>
> In keeping with the current behaviour, it would strip _all_ of these
> suffixes.
>

Yes, the exact behavior would need to be documented.  The existing case
indeed removes *ALL* of the single letter suffixes.  Clearly that behavior
cannot be changed (nor would I want to, that behavior is useful).

It's a decision about whether passing a tuple of substrings would remove
all of them (perhaps repeatedly) or only one of them.  And if only one, is
it "longest wins" or "first wins."  As I say, any choice of the semantics
would be fine with me if it were documented... since this edge case will be
uncommon in most uses (certainly in almost all of my uses).

E.g.

basename = fname.rstrip(('.jpg', '.png', 'gif'))

This is rarely ambiguous, and does something concretely useful that I've
coded many times.  But what if:

fname = 'silly.jpg.png.gif.png.jpg.gif.jpg'

I'm honestly not sure what behavior would be useful most often for this
oddball case.  For the suffixes, I think "remove them all" is probably the
best; that is consistent with thinking of the string passed in the existing
signature of .rstrip() as an iterable of characters.

But even if the decision was made to "only remove the single thing at end",
I'd still find the enhancement useful.  Sure, once in a while someone might
trip over the choice of semantics in this edge case, but if it were
documented, no big deal.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

The only reason I would support the idea would be to allow multiple
suffixes (or prefixes). Otherwise, it just does too little for a new
method. But adding that capability of startswith/endswith makes the cut off
something easy to get wrong and non-trivial to implement.

That said, I really like Brandt's ideas of expanding the signature of
.lstrip/.rstrip instead.

mystring.rstrip("abcd") # remove any of these single character suffixes

mystring.rstrip(('foo', 'bar', 'baz')) # remove any of these suffixes

Yes, the semantics or removals where one is a substring of another would
need to be decided. As long as it's documented, any behavior would be fine.
Most of the time the issue would be moot.

On Sun, Mar 31, 2019, 4:36 AM Steven D'Aprano  wrote:

> On Sun, Mar 31, 2019 at 04:48:36PM +1100, Chris Angelico wrote:
>
> > Regardless of the method name, IMO the functions should accept a tuple
> > of test strings, as startswith/endwith do. That's a feature that can't
> > easily be spelled in a one-liner. (Though stacked suffixes shouldn't
> > all be removed - "asdf.jpg.png".cutsuffix((".jpg", ".png")) should
> > return "asdf.jpg", not "asdf".)
>
> There's a slight problem with that: what happens if more than one suffix
> matches? E.g. given:
>
> "musical".lcut(('al', 'ical'))
>
> should the suffix "al" be removed, leaving "music"? (First match wins.)
>
> Or should the suffix "ical" be removed, leaving "mus"? (Longest match
> wins.)
>
> I don't think we can decide which is better, and I'm not keen on a
> keyword argument to choose one or the other, so I suggest we stick to
> the 90% solution of only supporting a single suffix.
>
> We can always revisit that in the future.
>
>
> --
> Steven
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

I like this idea quite a lot. I do not think of anything it works best at
first consideration.

On Sat, Mar 30, 2019, 8:28 PM Brandt Bucher  wrote:

>
> > One thing I love about .startswith() and .endswith() is matching
> multiple options. It's a little funny the multiple options must be a tuple
> exactly (not a list, not a set, not an iterator), but whatever. It would be
> about to lack that symmetry in the .cut_suffix() method.
> >
> > E.g now:
> >
> >   if fname.endswith(('.jpg', '.png', '.gif)): ...
> >
> > I'd expect to be able to do:
> >
> >   basename = fname.cut_suffix(('.jpg', '.png', '.gif))
>
> An idea worth considering: one can think of the “strip” family of methods
> as currently taking an iterable of strings as an argument (since a string
> is itself an sequence of strings):
>
> >>> "abcd".rstrip("dc")
> 'ab'
>
> It would not be a huge logical leap to allow them to take any iterable.
> Backward compatible, no new methods:
>
> >>> fname.rstrip(('.jpg', '.png', '.gif'))
>
> It even, in my opinion, can clarify "classic" strip/rstrip/lstrip usage:
>
> >>> "abcd".rstrip(("d", "c"))
> 'ab'
>
> Maybe I’m missing a breaking case though, or this isn’t as clear for
> others. Thoughts?
>
> Brandt
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

On Sat, Mar 30, 2019, 8:42 AM Steven D'Aprano  wrote:

> Most of us have had to cut a prefix or a suffix from a string, often a
> file extension. Its not as common as, say, stripping whitespace, but it
> happens often enough.


I do this all the time! I never really thought about wanting a method
though. I just spell it like this without much thought:

  basename = fname.split(".ext")[0]

But I suppose a method would be helpful. If we have one, PLEASE no
variation of 'trim' in the name. I still forget whether it's .lstrip() or
.ltrim() or .stripl() or etc. after 20 years using Python. Lots of
languages use trim for Python's strip, so having both with subtly different
meanings is a bug magnet.

One thing I love about .startswith() and .endswith() is matching multiple
options. It's a little funny the multiple options must be a tuple exactly
(not a list, not a set, not an iterator), but whatever. It would be about
to lack that symmetry in the .cut_suffix() method.

E.g now:

  if fname.endswith(('.jpg', '.png', '.gif)): ...

I'd expect to be able to do:

  basename = fname.cut_suffix(('.jpg', '.png', '.gif))
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New Project to Capture summaries from this

[David Mertz]

Dropping the mailing list is another topic that often comes up, and is
always a terrible idea. Every suggester had a different platform in mind,
only consistent in all being vastly worse than email for this purpose

That said, if someone writes a FAQ about this mailing list, the first
answer can be "We are not moving discussion to GitHub / Slack / Discuss /
Reddit / StackOverflow / MediaWiki / graffiti on popular buildings / etc"

On Thu, Mar 28, 2019, 11:29 AM Richard Whitehead 
wrote:

> Chris,
>
> As a new member to this list, I can tell you that searching for relevant
> old
> content was effectively impossible, so I'm all for some way of doing that.
>
> Please can I make a more radical suggestion, though: Drop the mailing list.
> How about a GitHub repo - a specific one (with no code), specifically for
> early ideas? Then, if an idea was accepted and turned into an issue to be
> implemented, it could link back to that original discussion. GitHub is
> easily searchable. It can email you if someone comments on an issue you
> have
> raised, etc.
>
> An alternative might be a StackOverflow section, but that wouldn't provide
> such tight integration in the case of an issue being raised.
>
> The new work you're doing would be a good way to populate the repo with its
> initial content.
>
> Richard
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New explicit methods to trim strings

[David Mertz]

All of this would be well served by a 3rd party library on PyPI.  Strings
already have plenty of methods (probably too many).  Having `stringtools`
would be nice to import a bunch of simple functions from.

On Mon, Mar 25, 2019 at 10:45 AM Alex Grigoryev  wrote:

> strip_prefix and strip_suffix I think are the best names from all and work
> perfectly with auto completion. Common use case:
>
> "  mailto:ma...@gmail.com".strip().strip_prefix("mailto:";)
>
>
> On Mar 25 2019, at 4:40 pm, Anders Hovmöller  wrote:
>
>
> Earlier, Anders wrote:
> I propose naming them strip_prefix() and strip_suffix() and just skip the
> one that does both sides since it makes no sense to me.
>
> This is good, except I prefer subtract_prefix(a, b), truncate_suffix etc.
> And for the two step process prefix_subtractor(a)(b) etc.
>
>
> I don't understand the logic for "subtract". That's not a thing for
> non-numbers.
>
> If you don't think "strip" is good, then I suggest "remove". Or one could
> also consider "without" since we're talking about something that removes
> /if present/ (making subtract even worse! Subtract doesn't stop at zero).
> So "without_prefix()".
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> [image: Sent from Mailspring]
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enabling / disabling optional type hinting

[David Mertz]

I agree with Guido. Yes, there are sequences of symbols that used to be
syntax errors but that now have some meaning in Python. The type annotation
colon is one of them. There are moderately many other constructs this can
be said of.

I can vaguely imagine accidentally using a colon rather than an equal sign
because the colon is similar to an assignment in a dict display. I don't
think anyone would do that as a slip of the finger, but more as a brain
glitch.

However, this feels like a very uncommon hazard, and just one among dozens
of similar ones. I don't think anything should be "fixed" at a language
level.

On Sat, Mar 23, 2019, 7:21 PM Guido van Rossum  wrote:

> On Sat, Mar 23, 2019 at 2:43 PM Andre Roberge 
> wrote:
>
>> My original message was referring to someone writing ":" instead of "="
>> by mistake -- nothing to do with the walrus assignment, but rather using
>> the same notation to assign a value to a key as they would when defining a
>> dict.
>>
>
> OK, I read your Original Post for this thread, about accidentally writing
> `d["answer"]: 42` instead of `d["answer"] = 42`.
>
> My reaction is that this was a user mistake of the same kind as
> accidentally writing `x + 1` instead of `x += 1`. That's just going to
> happen, very occasionally. (Though why? The ':' and '=' keys are not that
> close together.) Read your code carefully, or in an extreme case step
> through it in a debugger, and you'll notice the mistake.
>
> It's not a reason to pick on that particular syntax, and not much of a
> reason to try and introduce a mechanism to disable type hints. Sorry.
>
> PS. This particular syntax was introduced by PEP 526, and introduced in
> Python 3.6.
>
> --
> --Guido van Rossum (python.org/~guido)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[David Mertz]

On Thu, Mar 21, 2019, 10:15 PM Steven D'Aprano  wrote:

> What would be the most useful behaviour for dict "addition" in your
> opinion?
>

Probably what I would use most often was a "lossless" merging in which
duplicate keys resulted in the corresponding value becoming a set
containing all the merged values. E.g.

>>> d1 = {1: 55, 2: 77, 3: 88}
>>> d2 = {3: 99, 4: 22}
>>> add(d1, d2)
{1: 55, 2: 77, 3: {88, 99}, 4:22}

I'm sure most users would hate this too. It changes the type of values
between a thing and a set of things, and that has to be sorted out
downstream. But it is lossless in a similar way to Counter or sequence
addition.

I can write what I want perfectly well. Perhaps useing defaultdict as a
shortcut to get there. And I know there are some behaviors I have not
specified here, but my function can do whatever I want in the edge cases.

If we're to see 'd1 + d2' for the first time without having followed this
discussion, my guess would be behavior similar to what I show.


> Of course I could learn it and teach it, but it will always feel
> > like a wart in the language.
>
> Would that wartness be lessoned if it were spelled | or << instead?
>

Yes, definitely. Both those spellings feel pretty natural to me. They don't
have the misleading associations '+' carries. I'm kinda fond of '<<'
because it visitation resembles an arrow that I can think of as "put the
stuff here into there".

> In contrast, once you tell me about the special object "vectorised
> arrays",
> > `arr1 + arr2` does exactly what is expect in NumPy.
>
> I don't know Numpy well enough to know whether that is elementwise
> addition or concatenation or something else, so that example doesn't
> resonate with me. I can't guess what you expect, and I have no confidence
> that my guess (matrix addition of equal-sized arrays, an exception if
> unequal) will be what Numpy does
>

Fair enough. I've worked with NumPy long enough that perhaps I forget what
my first intuition was. I accept that it's non-obvious to many users.

FWIW, I really love NumPy behavior, but it's a shift in thinking vs lists.
E.g.

>>> a = array([1, 2, 3])
>>> b = array([[10, 11, 12], [100, 200, 300]])
>>> a + b
[[  11   13   15 ]
 [ 101 202 303]]

This is "broadcasting" of compatible shapes.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[David Mertz]

On Thu, Mar 21, 2019, 7:48 PM Steven D'Aprano  wrote:

> A number of people including Antoine and Serhiy seem to have taken the
> position that merely adding dict.__add__ will make existing code using +
> harder to understand, as you will need to consider not just numeric
> addition and concatenation, but also merging, when reading code.
>
> Would you agree that this example of + is perfectly clear today?
>
> for digit in digits:
> num = num*10 + digit
>
> By both the naming (digit, num) and presence of multiplication by the
> literal 10, it should be pretty obvious that this is probably doing
> integer addition.
>

Yep. This is clear and will not become less clear if some more objects grow
an .__add__() methods. Already, it is POSSIBLE that `num` and `digit` mean
something other than numbers. Bad naming of variables if so, but not
prohibited.

For example, NumPy uses '+' and '*' for elementwise operations, often with
broadcasting to different areas shapes. Maybe that's code dealing with
vectorised arrays... But probably not.

Holoviews users '+' and '*' to combine elements of graphs. E.g

labelled = low_freq * high_freq * linpoints
overlay + labelled + labelled.Sinusoid.Low_Frequency

ggplot in R has similar behavior. Maybe your loop is composing a complex
graph... But probably not.

Nonetheless, if I see `dict1 + dict2` the meaning you intend in the PEP
does not jump out as the obvious behavior. Nor even as the most useful
behavior. Of course I could learn it and teach it, but it will always feel
like a wart in the language.

In contrast, once you tell me about the special object "vectorised arrays",
`arr1 + arr2` does exactly what is expect in NumPy.

And your subjective feeling is well-noted :-)
>

This is more than "merely subjective." I teach Python. I write books about
Python. I've had tens of millions of readers of articles I've written about
Python. I'm not the only person in this discussion with knowledge of
learners and programmers and scientists... But the opinions I'm expressing
ARE on their behalf too (as I perceive likely surprise and likely bugs).

I like most of the design of Python. Almost all, even. But there are a few
warts in it. This would be a wart.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[David Mertz]

I dislike the symbol '+' to mean "dictionary merging with value updates." I
have no objection to, and mildly support, adding '|' with this meaning.

It's not really possible to give "that one example" where + for meeting
makes code less clear... In my eyes it would be EVERY such use. Every
example presented in this thread or in the PEP feels wrong to me. I know
about operator overloading and dunder methods and custom classes. My
intuition about '+' from math, other programming languages, and Python,
simply does not lead me to expect the proposed meaning.

On Thu, Mar 21, 2019, 12:43 PM Steven D'Aprano  wrote:

> I'd like to make a plea to people:
>
> I get it, there is now significant opposition to using the + symbol for
> this proposed operator. At the time I wrote the first draft of the PEP,
> there was virtually no opposition to it, and the | operator had very
> little support. This has clearly changed.
>
> At this point I don't think it is productive to keep making subjective
> claims that + will be more confusing or surprising. You've made your
> point that you don't like it, and the next draft^1 of the PEP will make
> that clear.
>
> But if you have *concrete examples* of code that currently is easy to
> understand, but will be harder to understand if we add dict.__add__,
> then please do show me!
>
> For those who oppose the + operator, it will help me if you made it
> clear whether it is *just* the + symbol you dislike, and would accept
> the | operator instead, or whether you hate the whole operator concept
> regardless of how it is spelled.
>
> And to those who support this PEP, code examples where a dict merge
> operator will help are most welcome!
>
>
>
>
> ^1 Coming Real Soon Now™.
>
>
> --
> Steven
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] True and False are singletons

[David Mertz]

There are few cases where I would approve of 'if x is True'. However, the
names used in the example suggest it could be one of those rare cases.
Settings of True/False/None (i.e. not set) seem like a reasonable pattern.
In fact, in code like that, merely "truthy" values are probably a bug that
should not pass silently. Obviously this depends on the surrounding code to
decide.

On Mon, Mar 18, 2019, 5:44 PM Greg Ewing 
wrote:

> Richard Damon wrote:
> > On 3/18/19 7:27 AM, Greg Ewing wrote:
> >
> >>if settings[MY_KEY]:
> >>...
>  >
> > That means something VERY different.
>
> Yes, but there needs to be justification for why the difference
> matters and why this particular way is the best way to deal
> with it.
>
> Whenever you write 'x is True' or 'x == True', you are putting
> a burden on all code that assigns to x to ensure that the
> value is actually an instance of bool rather than just a
> truthy or falsy value. That's an unusual requiremebt that
> can lead to obscure bugs.
>
> In the tri-state example, the way I would do it is to guard
> uses of it with 'if x is not None' and then treat the other
> values as truthy or falsy.
>
> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] True and False are singletons

[David Mertz]

It was a VERY long time ago when True and False were not singletons. I
don't think we should still try to write code based on rules that stopped
applying more than a decade ago.

On Mon, Mar 18, 2019, 5:42 PM Greg Ewing 
wrote:

> Oleg Broytman wrote:
> >Three-way (tri state) checkbox. You have to distinguish False and
> > None if the possible valuse are None, False and True.
>
> In that case the conventional way to write it would be
>
>  if settings[MY_KEY] == True:
>  ...
>
> It's not a major issue, but I get nervous when I see code
> that assumes True and False are unique, because things
> weren't always that way.
>
> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New Data Structure - Non Well-Founded Dict

[David Mertz]

This is an interesting challenge you have. However, this list is for
proposing ideas for changes in the Python language itself, in particular
the CPython reference implementation.

Python-list or some discussion site dealing with machine learning or
natural language processing would be appropriate for the task you are
trying to figure out. I suspect that third party libraries contain the data
structures you need, but I cannot recommend anything specific from my
experience.

On Sun, Mar 17, 2019, 12:39 PM Savant Of Illusions 
wrote:

> I am in desperate need of a dict similar structure that allows sets and/or
> dicts as keys *and* values. My application is NLP conceptual plagiarism
> detection. Dealing with infinite grammars communicating illogical
> concepts. Would be even better if keys could nest the same data structure,
> e.g. set(s) or dict(s) in set(s) or dict(s) of the set(s) or dict(s) as
> key(s).
>
> In order to detect conceptual plagiarism, I need to populate a data
> structure with if/then equivalents as a decision tree. But my equivalents
> have potentially infinite ways of arranging them syntactically* and*
> semantically.
>
> A dict having keys with identical set values treats each key as a distinct
> element. I am dealing with semantics or elemental equivalents and many
> different statements treated as equivalent statements involving if/then
> (key/value) or a implies b, where a and/or b can be an element or an
> if/then as an element. Modeling the syntactic equivalences of such claims
> is paramount, and in order to do that, I need the data structure.
>
> Hello, I am Stephanie. I have never contributed to any open source. I am
> about intermediate at python and I am a self-directed learner/hobbyist. I
> am trying to prove with my code that a particular very famous high profile
> pop debate intellectual is plagiarizing Anders Breivik. I can show it via
> observation, but his dishonesty is dispersed among many different
> talks/lectures. I am dealing with a large number of speaking hours as
> transcripts containing breadcrumbs that are very difficult for a human to
> piece together as having come from the manifesto which is 1515 pages and
> about half copied from other sources. The concepts stolen are
> rearrangements and reorganizations of the same identical claims and themes.
> He occasionally uses literal string plagiarism but not very much at once.
> He is very good at elaboration which makes it even more difficult.
>
> Thank you, for your time,
> Stephanie
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[David Mertz]

ch is also quite convenient. Not to mention
> readability !).
>
> To cover your concern: decopatch depends on makefun, so both come at the
> same time when you install decopatch, and decopatch by default relies on
> makefun when you use it in “double-flat” mode to create wrappers as
> explained here https://smarie.github.io/python-decopatch/#even-simpler
>
> --
>
>
>
> Thanks again for this discussion! It is challenging but it is necessary,
> to make sure I did not answer a non-existent need ;)
>
> Kind regards
>
>
>
> --
>
> Sylvain
>
>
>
> *De :* David Mertz 
> *Envoyé :* mardi 12 mars 2019 15:30
> *À :* Sylvain MARIE 
> *Cc :* Steven D'Aprano ; python-ideas <
> python-ideas@python.org>
> *Objet :* Re: [Python-ideas] Problems (and solutions?) in writing
> decorators
>
>
>
> [External email: Use caution with links and attachments]
> --
>
>
>
> The documentation for wrapt mentions:
>
>
> Decorators With Optional Arguments
>
> Although opinion can be mixed about whether the pattern is a good one, if
> the decorator arguments all have default values, it is also possible to
> implement decorators which have optional arguments.
>
> As Graham hints in his docs, I think repurposing decorator factories as
> decorators is an antipattern. Explicit is better than implicit.
>
>
>
> While I *do* understands that what decotools and makefun do are
> technically independent, I'm not sure I ever want them independently in
> practice. I did write the book _Functional Programming in Python_, so I'm
> not entirely unfamiliar with function wrappers.
>
> On Tue, Mar 12, 2019, 10:18 AM David Mertz  wrote:
>
> The wrapt module I linked to (not funtools.wraps) provides all the
> capabilities you mention since 2013. It allows mixed use of decorators as
> decorator factories. It has a flat style.
>
>
>
> There are some minor API difference between your libraries and wrapt, but
> the concept is very similar. Since yours is something new, I imagine you
> perceive some win over what wrapt does.
>
> On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE  wrote:
>
> David, Steven,
>
> Thanks for your interest !
>
> As you probably know, decorators and function wrappers are *completely
> different concepts*. A decorator can directly return the decorated function
> (or class), it does not have to return a wrapper. Even more, it can
> entirely replace the decorated item with something else (not even a
> function or class!). Try it: it is possible to write a decorator to replace
> a function with an integer, even though it is probably not quite useful :)
>
> `decopatch` helps you write decorators, whatever they are. It "just"
> solves the annoying issue of having to handle the no-parenthesis and
> with-parenthesis calls. In addition as a 'goodie', it proposes two
> development styles: *nested* (you have to return a function) and *flat*
> (you directly write what will happen when the decorator is applied to
> something).
> --
> Now about creating signature-preserving function wrappers (in a decorator,
> or outside a decorator - again, that's not related). That use case is
> supposed to be covered by functools.wrapt. Unfortunately as explained here
> https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F308999%2Fwhat-does-functools-wraps-do%2F55102697%2355102697&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186393926&sdata=Dmkd0Ld1HsuCJtJDCS6GDUgwH3GO66zwYMpZ9Ow%2B18k%3D&reserved=0>
> this is not the case because with functools.wrapt:
>  - the wrapper code will execute even when the provided arguments are
> invalid.
>  - the wrapper code cannot easily access an argument using its name, from
> the received *args, **kwargs. Indeed one would have to handle all cases
> (positional, keyword, default) and therefore to use something like
> Signature.bind().
>
> For this reason I proposed a replacement in `makefun`:
> https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-makefun%2F%23signature-preserving-function-wrappers&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186403936&sdata=SO319cmnKXiUK6Zek%2FSioHIhFfnQCZRpAF3kim84mv8%3D&reserved=0>
> --
> Now bridging the gap. Of course a very interesting use cases for
> deco

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[David Mertz]

The documentation for wrapt mentions:

Decorators With Optional Arguments
<https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optional-arguments>

Although opinion can be mixed about whether the pattern is a good one, if
the decorator arguments all have default values, it is also possible to
implement decorators which have optional arguments.
As Graham hints in his docs, I think repurposing decorator factories as
decorators is an antipattern. Explicit is better than implicit.

While I *do* understands that what decotools and makefun do are technically
independent, I'm not sure I ever want them independently in practice. I did
write the book _Functional Programming in Python_, so I'm not entirely
unfamiliar with function wrappers.

On Tue, Mar 12, 2019, 10:18 AM David Mertz  wrote:

> The wrapt module I linked to (not funtools.wraps) provides all the
> capabilities you mention since 2013. It allows mixed use of decorators as
> decorator factories. It has a flat style.
>
> There are some minor API difference between your libraries and wrapt, but
> the concept is very similar. Since yours is something new, I imagine you
> perceive some win over what wrapt does.
>
> On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE  wrote:
>
>> David, Steven,
>>
>> Thanks for your interest !
>>
>> As you probably know, decorators and function wrappers are *completely
>> different concepts*. A decorator can directly return the decorated function
>> (or class), it does not have to return a wrapper. Even more, it can
>> entirely replace the decorated item with something else (not even a
>> function or class!). Try it: it is possible to write a decorator to replace
>> a function with an integer, even though it is probably not quite useful :)
>>
>> `decopatch` helps you write decorators, whatever they are. It "just"
>> solves the annoying issue of having to handle the no-parenthesis and
>> with-parenthesis calls. In addition as a 'goodie', it proposes two
>> development styles: *nested* (you have to return a function) and *flat*
>> (you directly write what will happen when the decorator is applied to
>> something).
>> --
>> Now about creating signature-preserving function wrappers (in a
>> decorator, or outside a decorator - again, that's not related). That use
>> case is supposed to be covered by functools.wrapt. Unfortunately as
>> explained here
>> https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697
>> this is not the case because with functools.wrapt:
>>  - the wrapper code will execute even when the provided arguments are
>> invalid.
>>  - the wrapper code cannot easily access an argument using its name, from
>> the received *args, **kwargs. Indeed one would have to handle all cases
>> (positional, keyword, default) and therefore to use something like
>> Signature.bind().
>>
>> For this reason I proposed a replacement in `makefun`:
>> https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers
>> --
>> Now bridging the gap. Of course a very interesting use cases for
>> decorators is to create decorators that create a signature-preserving
>> wrapper. It is possible to combine decopatch and makefun for this:
>> https://smarie.github.io/python-decopatch/#3-creating-function-wrappers .
>> Decopatch even proposes a "double-flat" development style where you
>> directly write the wrapper body, as explained in the doc.
>>
>> Did I answer your questions ?
>> Thanks again for the quick feedback !
>> Best,
>>
>> Sylvain
>>
>> -Message d'origine-
>> De : Python-ideas 
>> De la part de Steven D'Aprano
>> Envoyé : mardi 12 mars 2019 12:30
>> À : python-ideas@python.org
>> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators
>>
>> [External email: Use caution with links and attachments]
>>
>> 
>>
>>
>>
>> On Tue, Mar 12, 2019 at 09:36:41AM +, Sylvain MARIE via Python-ideas
>> wrote:
>>
>> > I therefore proposed
>> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma
>> > rie.github.io%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s
>> > e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae
>> > 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2
>> > Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it
>> > provides an equivalent of `@functools.wraps` that is truly
>> > signature-preserving
>&g

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[David Mertz]

The wrapt module I linked to (not funtools.wraps) provides all the
capabilities you mention since 2013. It allows mixed use of decorators as
decorator factories. It has a flat style.

There are some minor API difference between your libraries and wrapt, but
the concept is very similar. Since yours is something new, I imagine you
perceive some win over what wrapt does.

On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE  wrote:

> David, Steven,
>
> Thanks for your interest !
>
> As you probably know, decorators and function wrappers are *completely
> different concepts*. A decorator can directly return the decorated function
> (or class), it does not have to return a wrapper. Even more, it can
> entirely replace the decorated item with something else (not even a
> function or class!). Try it: it is possible to write a decorator to replace
> a function with an integer, even though it is probably not quite useful :)
>
> `decopatch` helps you write decorators, whatever they are. It "just"
> solves the annoying issue of having to handle the no-parenthesis and
> with-parenthesis calls. In addition as a 'goodie', it proposes two
> development styles: *nested* (you have to return a function) and *flat*
> (you directly write what will happen when the decorator is applied to
> something).
> --
> Now about creating signature-preserving function wrappers (in a decorator,
> or outside a decorator - again, that's not related). That use case is
> supposed to be covered by functools.wrapt. Unfortunately as explained here
> https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697
> this is not the case because with functools.wrapt:
>  - the wrapper code will execute even when the provided arguments are
> invalid.
>  - the wrapper code cannot easily access an argument using its name, from
> the received *args, **kwargs. Indeed one would have to handle all cases
> (positional, keyword, default) and therefore to use something like
> Signature.bind().
>
> For this reason I proposed a replacement in `makefun`:
> https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers
> --
> Now bridging the gap. Of course a very interesting use cases for
> decorators is to create decorators that create a signature-preserving
> wrapper. It is possible to combine decopatch and makefun for this:
> https://smarie.github.io/python-decopatch/#3-creating-function-wrappers .
> Decopatch even proposes a "double-flat" development style where you
> directly write the wrapper body, as explained in the doc.
>
> Did I answer your questions ?
> Thanks again for the quick feedback !
> Best,
>
> Sylvain
>
> -Message d'origine-
> De : Python-ideas 
> De la part de Steven D'Aprano
> Envoyé : mardi 12 mars 2019 12:30
> À : python-ideas@python.org
> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators
>
> [External email: Use caution with links and attachments]
>
> 
>
>
>
> On Tue, Mar 12, 2019 at 09:36:41AM +, Sylvain MARIE via Python-ideas
> wrote:
>
> > I therefore proposed
> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma
> > rie.github.io%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s
> > e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae
> > 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2
> > Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it
> > provides an equivalent of `@functools.wraps` that is truly
> > signature-preserving
>
> Tell us more about that please. I'm very interested in getting decorators
> preserve the original signature.
>
>
> --
> Steven
> ___
> Python-ideas mailing list
> Python-ideas@python.org
>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&sdata=XcYfEginmDF7kIpGGA0XxDZKpUn9e4p2zPFk7UAruYg%3D&reserved=0
> Code of Conduct:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&sdata=20ZrtVQZbpQ54c96veSXIOfEK7rKy0ggj0omTZg3ri8%3D&reserved=0
>
> __
> This email has been scanned by the Symantec Email Security.cloud service.
> __
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[David Mertz]

What advantage do you perceive decopatch to have over wrapt? (
https://github.com/GrahamDumpleton/wrapt)

On Tue, Mar 12, 2019, 5:37 AM Sylvain MARIE via Python-ideas <
python-ideas@python.org> wrote:

> Dear python enthusiasts,
>
> Writing python decorators is indeed quite a tideous process, in particular
> when you wish to add arguments, and in particular in two cases : all
> optional arguments, and one mandatory argument. Indeed in these two cases
> there is a need to disambiguate between no-parenthesis and with-parenthesis
> usage.
>
> After having struggled with this pattern for two years in various open
> source and industrial projects, I ended up writing a library to hopefully
> solve this once and for all: https://smarie.github.io/python-decopatch/ .
> It is extensively tested (203 tests) against many combinations of
> signature/calls.
> I would gladly appreciate any feedback !
>
> Please note that there is a "PEP proposal draft" in the project page
> because I belive that the best a library can do will always be a poor
> workaround, where the interpreter or stdlib could really fix it properly.
> Sorry for not providing too much implementation details in that page, my
> knowledge of the python interpreter is unfortunately quite limited.
>
> --
>
> Finally there is an additional topic around decorators : people tend to
> believe that decorators and function wrappers are the same, which is
> absolutely not the case. I used the famous `decorator` lib in many projects
> but I was not satisfied because it was solving both issues at the same
> time, maintaining the confusion. I therefore proposed
> https://smarie.github.io/python-makefun/ . In particular it provides an
> equivalent of `@functools.wraps` that is truly signature-preserving (based
> on the same recipe than `decorator`).
> Once again, any feedback would be gladly appreciated !
>
> Kind regards
>
> Sylvain
>
> -Message d'origine-
> De : Python-ideas 
> De la part de Greg Ewing
> Envoyé : vendredi 26 octobre 2018 00:04
> À : python-ideas 
> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators
>
> [External email: Use caution with links and attachments]
>
> 
>
>
>
> Jonathan Fine wrote:
> > I also find writing decorators a bit
> > hard. It seems to be something I have to learn anew each time I do it.
> > Particularly for the pattern
> >
> > @deco(arg1, arg2) def fn(arg3, arg4):
>  > # function body
> >
> > Perhaps doing something with partial might help here. Anyone here
> > interested in exploring this?
> >
>
> I can't think of a way that partial would help. But would you find it
> easier if you could do something like this?
>
>  class deco(Decorator):
>
>  def __init__(self, arg1, arg2):
>  self.arg1 = arg1
>  self.arg2 = arg2
>
>  def invoke(self, func, arg3, arg4):
>  # function body
>
> Implementation:
>
>  class Decorator:
>
>  def __call__(self, func):
>  self._wrapped_function = func
>  return self._wrapper
>
>  def _wrapper(self, *args, **kwds):
>  return self.invoke(self._wrapped_function, *args, **kwds)
>
> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7C295cecd58fc3461f42c108d63ac5e03e%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C63676101605889&sdata=Z8OET1CZZWnmN5czi0rZ1X57%2FDd4a9IDbbSujsDNWzk%3D&reserved=0
> Code of Conduct:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7C295cecd58fc3461f42c108d63ac5e03e%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C63676101615894&sdata=u0hvM5cR%2BR1Vni%2BV48WoNF%2FpriCaOG5%2BFAXayaTGsYY%3D&reserved=0
>
> __
> This email has been scanned by the Symantec Email Security.cloud service.
> __
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] The @update operator for dictionaries

[David Mertz]

Maybe it's just the C++ IO piping that makes me like it, but these actually
seem intuitive to me, whereas `+` or even `|` leaves me queasy.

On Sat, Mar 9, 2019 at 7:40 PM Ian Foote  wrote:

> > It might also be worth considering YAML's own dict merge operator, the
> > "<<" operator, as in https://yaml.org/type/merge.html as this is the
> > existing Python's shift operator added to dict and will require no
> > change to the synatx::
> >
> >   a = a << b
>
>
> I really like this suggestion. It captures the asymmetry, since we could
> have a = a >> b to merge with the other dictionary's keys taking precedence.
>
> My instinct is that a = a << b would take b's values when keys collide and
> a = a >> b would take a's values when keys collide. I'd be very interested
> to know if this matches most peoples' intuitions.
>
> On Sat, 9 Mar 2019 at 18:44, Meitham Jamaa  wrote:
>
>> It might also be worth considering YAML's own dict merge operator, the
>> "<<" operator, as in https://yaml.org/type/merge.html as this is the
>> existing Python's shift operator added to dict and will require no
>> change to the synatx::
>>
>>   a = a << b
>>
>> Meitham
>>
>>
>> On 03/10, Chris Angelico wrote:
>> > On Sun, Mar 10, 2019 at 3:16 AM Jonathan Fine 
>> wrote:
>> > >
>> > >  Anders Hovmöller wrote:
>> > >
>> > > > I don't understand what you mean. Can you provide examples that
>> show the state of the dicts before and after and what the syntax would be
>> the equivalent of in current python?
>> > >
>> > > If a.__radd__ exists, then
>> > > a += b
>> > > is equivalent to
>> > > a = a.__radd__(b)
>> > >
>> > > Similarly, if a.__iat_update__ exists then
>> > > a @update= b
>> > > would be equivalent to
>> > > a = a.__iat_update__(b)
>> > >
>> > > Here's an implementation
>> > > def __iat_update__(self, other):
>> > > self.update(other)
>> > > return self
>> > >
>> > > Thus, 'b' would be unchanged, and 'a' would be the same dictionary as
>> > > before, but updated with 'b'.
>> >
>> > With something this long, how is it better from just writing:
>> >
>> > a = a.update_with(b)
>> >
>> > ? What's the point of an operator, especially if - by your own
>> > statement - it will backward-incompatibly change the language grammar
>> > (in ways that I've yet to understand, since you haven't really been
>> > clear on that)?
>> >
>> > ChrisA
>> >
>>
>> --
>> Meitham Jamaa
>>
>> http://meitham.com
>> GPG Fingerprint: 3934D0B2
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Attribute-Getter Syntax Proposal

[David Mertz]

I'm really old ... I remember thinking how clever attrgetter() was when it
was after to Python 2.4.

On Fri, Mar 8, 2019, 7:51 PM David Mertz  wrote:

> You could use the time machine:
> https://docs.python.org/3/library/operator.html
>
> On Fri, Mar 8, 2019, 11:57 AM Samuel Li  wrote:
>
>> Don't know if this has been suggested before. Instead of writing
>> something like
>>
>> >>> map(lambda x: x.upper(), ['a', 'b', 'c'])
>>
>> I suggest this syntax:
>> >>> map(.upper(), ['a', 'b', 'c'])
>>
>> This would also work for attributes:
>> >>> map(.real, [1j, 2, 3+4j])
>>
>> Internally, this would require translating
>>
>> .attribute -> lambda x: x.attribute
>>
>> and
>>
>> .method(*args, **kwargs) -> lambda x: x.method(*args, **kwargs)
>>
>> This translation should only take place where a "normal" attribute lookup
>> makes no sense (throws a SyntaxError); i.e. foo.bar works as before,
>> foo(.bar) would previously throw a SyntaxError, so the new syntax applies
>> and the .bar is interpreted as an attrgetter.
>>
>> This is of course only a cosmetic improvement over operator.attrgetter
>> and operator.methodcaller, but I think it's nice enough to warrant
>> consideration.
>>
>> If you like this idea or think it's utter garbage, feel free to discuss.
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Attribute-Getter Syntax Proposal

[David Mertz]

You could use the time machine:
https://docs.python.org/3/library/operator.html

On Fri, Mar 8, 2019, 11:57 AM Samuel Li  wrote:

> Don't know if this has been suggested before. Instead of writing something
> like
>
> >>> map(lambda x: x.upper(), ['a', 'b', 'c'])
>
> I suggest this syntax:
> >>> map(.upper(), ['a', 'b', 'c'])
>
> This would also work for attributes:
> >>> map(.real, [1j, 2, 3+4j])
>
> Internally, this would require translating
>
> .attribute -> lambda x: x.attribute
>
> and
>
> .method(*args, **kwargs) -> lambda x: x.method(*args, **kwargs)
>
> This translation should only take place where a "normal" attribute lookup
> makes no sense (throws a SyntaxError); i.e. foo.bar works as before,
> foo(.bar) would previously throw a SyntaxError, so the new syntax applies
> and the .bar is interpreted as an attrgetter.
>
> This is of course only a cosmetic improvement over operator.attrgetter and
> operator.methodcaller, but I think it's nice enough to warrant
> consideration.
>
> If you like this idea or think it's utter garbage, feel free to discuss.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[David Mertz]

I strongly agree with Ka-Ping. '+' is intuitively concatenation not
merging. The behavior is overwhelmingly more similar to the '|' operator in
sets (whether or not a user happens to know the historical implementation
overlap).

I think growing the full collection of set operations world be a pleasant
addition to dicts. I think shoe-horning in plus would always be jarring to
me.

On Wed, Mar 6, 2019, 5:30 AM Ka-Ping Yee  wrote:

> len(dict1 + dict2) does not equal len(dict1) + len(dict2), so using the +
> operator is nonsense.
>
> len(dict1 + dict2) cannot even be computed by any expression involving +.
> Using len() to test the semantics of the operation is not arbitrary; the
> fact that the sizes do not add is a defining quality of a merge.  This is a
> merge, not an addition.  The proper analogy is to sets, not lists.
>
> The operators should be |, &, and -, exactly as for sets, and the
> behaviour defined with just three rules:
>
> 1. The keys of dict1 [op] dict2 are the elements of dict1.keys() [op]
> dict2.keys().
>
> 2. The values of dict2 take priority over the values of dict1.
>
> 3. When either operand is a set, it is treated as a dict whose values are
> None.
>
> This yields many useful operations and, most importantly, is simple to
> explain.  "sets and dicts can |, &, -" takes up less space in your brain
> than "sets can |, &, - but dicts can only + and -, where dict + is like set
> |".
>
> merge and update some items:
>
> {'a': 1, 'b': 2} | {'b': 3, 'c': 4} => {'a': 1, 'b': 3, 'c': 4}
>
> pick some items:
>
> {'a': 1, 'b': 2} & {'b': 3, 'c': 4} => {'b': 3}
>
> remove some items:
>
> {'a': 1, 'b': 2} - {'b': 3, 'c': 4} => {'a': 1}
>
> reset values of some keys:
>
> {'a': 1, 'b': 2} | {'b', 'c'} => {'a': 1, 'b': None, 'c': None}
>
> ensure certain keys are present:
>
> {'b', 'c'} | {'a': 1, 'b': 2} => {'a': 1, 'b': 2, 'c': None}
>
> pick some items:
>
> {'b', 'c'} | {'a': 1, 'b': 2} => {'b': 2}
>
> remove some items:
>
> {'a': 1, 'b': 2} - {'b', 'c'} => {'a': 1}
>
> On Wed, Mar 6, 2019 at 1:51 AM Rémi Lapeyre  wrote:
>
>> Le 6 mars 2019 à 10:26:15, Brice Parent
>> (cont...@brice.xyz(mailto:cont...@brice.xyz)) a écrit:
>>
>> >
>> > Le 05/03/2019 à 23:40, Greg Ewing a écrit :
>> > > Steven D'Aprano wrote:
>> > >> The question is, is [recursive merge] behaviour useful enough and
>> > > > common enough to be built into dict itself?
>> > >
>> > > I think not. It seems like just one possible way of merging
>> > > values out of many. I think it would be better to provide
>> > > a merge function or method that lets you specify a function
>> > > for merging values.
>> > >
>> > That's what this conversation led me to. I'm not against the addition
>> > for the most general usage (and current PEP's describes the behaviour I
>> > would expect before reading the doc), but for all other more specific
>> > usages, where we intend any special or not-so-common behaviour, I'd go
>> > with modifying Dict.update like this:
>> >
>> > foo.update(bar, on_collision=updator) # Although I'm not a fan of the
>> > keyword I used
>>
>> Le 6 mars 2019 à 10:26:15, Brice Parent
>> (cont...@brice.xyz(mailto:cont...@brice.xyz)) a écrit:
>>
>> >
>> > Le 05/03/2019 à 23:40, Greg Ewing a écrit :
>> > > Steven D'Aprano wrote:
>> > >> The question is, is [recursive merge] behaviour useful enough and
>> > > > common enough to be built into dict itself?
>> > >
>> > > I think not. It seems like just one possible way of merging
>> > > values out of many. I think it would be better to provide
>> > > a merge function or method that lets you specify a function
>> > > for merging values.
>> > >
>> > That's what this conversation led me to. I'm not against the addition
>> > for the most general usage (and current PEP's describes the behaviour I
>> > would expect before reading the doc), but for all other more specific
>> > usages, where we intend any special or not-so-common behaviour, I'd go
>> > with modifying Dict.update like this:
>> >
>> > foo.update(bar, on_collision=updator) # Although I'm not a fan of the
>> > keyword I used
>>
>> This won’t be possible update() already takes keyword arguments:
>>
>> >>> foo = {}
>> >>> bar = {'a': 1}
>> >>> foo.update(bar, on_collision=lambda e: e)
>> >>> foo
>> {'a': 1, 'on_collision':  at 0x10b8df598>}
>>
>> > `updator` being a simple function like this one:
>> >
>> > def updator(updated, updator, key) -> Any:
>> > if key == "related":
>> > return updated[key].update(updator[key])
>> >
>> > if key == "tags":
>> > return updated[key] + updator[key]
>> >
>> > if key in ["a", "b", "c"]: # Those
>> > return updated[key]
>> >
>> > return updator[key]
>> >
>> > There's nothing here that couldn't be made today by using a custom
>> > update function, but leaving the burden of checking for values that are
>> > in both and actually inserting the new values to Python's language, and
>> > keeping on our side only the parts that are specific to our use case,
>> > makes in my opinion 

Re: [Python-ideas] Dict joining using + and +=

[David Mertz]

On Mon, Mar 4, 2019, 11:45 AM Steven D'Aprano  wrote:

> > Like other folks in the thread, I also want to merge dicts three times
> per
> > year.
>
> I'm impressed that you have counted it with that level of accuracy. Is it
> on the same three days each year, or do they move about? *wink*
>

To be respectful, I always merge dicts on Eid al-Fitr, Diwali, and Lent. I
was speaking approximate since those do not appears line up with the same
Gregorian year.

> And every one of those times, itertools.ChainMap is the right way to do
> that non-destructively, and without copying.
>
> Can you elaborate on why ChainMap is the right way to merge multiple dicts
> into a single, new dict?
>

Zero-copy.


> ChainMap also seems to implement the opposite behaviour to that usually
> desired: first value seen wins, instead of last:
>

True, the semantics are different, but equivalent, to the proposed dict
addition. I put the key I want to "win" first rather than last.

If you know ahead of time which order you want, you can simply reverse it:
>

This seems nonsensical. If I write, at some future time,
'dict1+dict2+dict3' I need exactly as much to know "ahead of time" which
keys I intend to win.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[David Mertz]

On Mon, Mar 4, 2019, 8:30 AM Serhiy Storchaka  wrote:

> But is merging two dicts a common enough problem that needs introducing
> an operator to solve it? I need to merge dicts maybe not more than one
> or two times by year, and I am fine with using the update() method.
> Perhaps {**d1, **d2} can be more appropriate in some cases, but I did not
> encounter such cases yet.
>

Like other folks in the thread, I also want to merge dicts three times per
year. And every one of those times, itertools.ChainMap is the right way to
do that non-destructively, and without copying.

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[David Mertz]

"foo" + "bar" != "bar" + "foo"

On Wed, Feb 27, 2019, 12:35 PM George Castillo  wrote:

> The key conundrum that needs to be solved is what to do for `d1 + d2` when
>> there are overlapping keys. I propose to make d2 win in this case, which is
>> what happens in `d1.update(d2)` anyways. If you want it the other way,
>> simply write `d2 + d1`.
>
>
> This would mean that addition, at least in this particular instance, is
> not a commutative operation.  Are there other places in Python where this
> is the case?
>
> ~ George
>
>
> On Wed, Feb 27, 2019 at 10:06 AM Guido van Rossum 
> wrote:
>
>> On Wed, Feb 27, 2019 at 8:50 AM Rhodri James 
>> wrote:
>>
>>> On 27/02/2019 16:25, João Matos wrote:
>>> > I would like to propose that instead of using this (applies to Py3.5
>>> and upwards)
>>> > dict_a = {**dict_a, **dict_b}
>>> >
>>> > we could use
>>> > dict_a = dict_a + dict_b
>>> >
>>> > or even better
>>> > dict_a += dict_b
>>>
>>> While I don't object to the idea of concatenating dictionaries, I feel
>>> obliged to point out that this last is currently spelled
>>> dict_a.update(dict_b)
>>>
>>
>> This is likely to be controversial. But I like the idea. After all, we
>> have `list.extend(x)` ~~ `list += x`. The key conundrum that needs to be
>> solved is what to do for `d1 + d2` when there are overlapping keys. I
>> propose to make d2 win in this case, which is what happens in
>> `d1.update(d2)` anyways. If you want it the other way, simply write `d2 +
>> d1`.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 and long string literals

[David Mertz]

I've sort of lost the threads about who recommends what.  I do not think
that PEP8 needs to include a sentence like "Better tooling would be really
cool" ... notwithstanding that I think that is a true sentence.

On Mon, Feb 25, 2019 at 2:50 PM Jonathan Fine  wrote:

> Hi David
>
> Thank you for sharing your experience.
>
> I'd be most grateful if you could tell us, are you happy with the
> interpretation and additions I've suggested for PEP 8?
>
> And the revisions to pep8 and other linting tools?
>
> Jonathan
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 and long string literals

[David Mertz]

I find the main pain point of line width limits to be string literals that
call out to some *other* code-like thing.  Long URLs, user messages, and
long SQL are three common examples.

It's actually less an issue with SQL since that is itself more readable
across multiple lines, plus SQL itself doesn't care about newlines
embedded.  So just using triple-quoted strings works best there.  However,
sometimes it's just a quick one-liner SQL I want to include, but that can
easily grow longer than 80 characters (or really even 50 chars if the SQL
is being assigned to something inside some nested logic, or being an
argument to a function call).

URLs can get long, and are whitespace sensitive.  I wrote something like
this just today:

url = (f'{URL_BASE}'
   f'&arg1={row["column_name_1"]}'
   f'&something_else={1+row["that value"]}'
   f'&more_args={USER_ID}'
   f'&some_other_stuff={translate[row["foo"]]}'
   f'&yep_this={row["foo"]}'
   f'&arg17=123456')

I don't think that's terrible, and it's a lot nicer to be able to visually
separate what are really arguments to a REST call.  But it's definitely
super-easy to mess up this sort of thing, including by letting a little
whitespace sneak in where I don't want it.

But if the URL wasn't quite so convoluted (this example is pretty directly
taken from a real thing today, just a few names munged to protect the
client :-)), I might want it on one line.  And I'd probably want my linters
to stop nagging me about the fact I do that.

However, what I care about more than that is my editor.  It would be really
nice if my editor provided something like "vertical folding" for things
like this. I do not know of any editors that do that, but it would be easy
to imagine.  E.g. I might have an editor display:

url = f'{URL_BASE}&arg1={row["column_*(..93 chars..)*rg17=123456'

Does anyone know of editors that do that sort of thing? Obviously, you'd
need some way to fold/unfold the hidden text.  But for many purposes, the
first and last few characters probably convey the purpose of the line.

On Mon, Feb 25, 2019 at 4:53 AM Jonathan Fine  wrote:

> I've started this thread, because I think long string literals are
> somewhat special, and may have an easy resolution.
>
> According to PEP 8 a good reason to ignore the line-length (or any
> other) guideline is that "applying the guideline would make the code
> less readable, even for someone who is used to reading code that
> follows this PEP."
>
> PEP 8 also says "Limit all lines to a maximum of 79 characters."
>
> So perhaps it is enough, for long string literals, to add a note to
> PEP 8: When sensible to do so, lines containing a long string literal
> can exceed this length.
>
> It might also help to add to PEP 8 some Programming Recommendations
> for Long String Literals. (PEP 8 already has recommendations for
> Function and Variable Annotations.)
>
> Finally, tools such as pep8 and pylint would need some changes, to
> treat differently lines that contain a long string constant.
>
> --
> Jonathan
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 update on line length

[David Mertz]

As a human, and one who reads and writes code even, I know that MY ability
to understands the meaning of a line of code starts to plummet when it
reaches about 65-70 characters in length.

Yes, of course there are some "it depends" caveats that make some lines
easier and some harder. But an 80 characters "limit" is an excellent rule
of thumb, as long as most coders usually "err" on the short side.

My brain is a lot like those of people whom typesetters have typeset for
for hundred of years, and whom scribes have written for for thousands. It's
easy to be sophistical and discover that code is so very much different
that roles of intelligibility don't apply. It's not though, and they do
still.

Or maybe everyone else in the thread is simply smarter and better able to
handle complexity than I am. More power to you, but that definitely doesn't
feel like "computer programming for everyone."

If you want to write 100 or 120 characters lines, Python won't complain,
but you've walled off your code from me as a potential reader of it.

On Mon, Feb 25, 2019, 12:25 AM Raymond Hettinger <
raymond.hettin...@gmail.com> wrote:

>
>
> > On Feb 22, 2019, at 1:10 PM, Greg Ewing 
> wrote:
>
> >> “Typesetters hundreds of years ago used less than 80 chars per line, so
> that’s what we should do for Python code now” is a pretty weak argument.
> >
> > But that's not the entire argument -- the point it is that typesetters
> > had the goal of making lines of text readable, which is similar (if not
> > quite the same) as the goal of making lines of program code readable.
> > It's a lot closer than, for example, the goal of fitting in an
> > accountant's spreadsheet.
>
>
> The issue with reference to typesetter rules is that they were targeted at
> blocks of prose rather than heavily nested hanging indents with non-trivial
> string literals or a dotted attribute notation.  Typesetters were also
> dealing with fixed page widths and need to leave gutter space for binding.
>
> The "rules" aren't comparable at all.
>
>
> > I would say it the other way around. Once you've reduced the complexity
> > of a line to something a human can handle, *most* of the time 80 chars
> > is enough.
>
> That would make sense if we started at column 0; however, if you have your
> prefix your thoughts with something like
>
> '''
> class TestRemote(unittest.TestCase):
> def test_heartbeat(self):
> ...
> self.assertIsInstance(...
> '''
>
> then the meant of the part "a human can handle" starts at column 30.  Then
> if you need good variable names and/or have to module.function prefixes,
> there is sometimes little to left to work with.
>
>
> Raymond
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 update on line length

[David Mertz]

Wow... Sorry about all my typos from autocorrect. I should not write a
longish reply on my tablet, or should proofread and correct.

On Tue, Feb 19, 2019, 9:30 AM David Mertz  On Tue, Feb 19, 2019, 9:07 AM simon.bordeyne  wrote:
>
>> I find that 100 to 120 characters is ideal as far as line length goes.
>>
>
> I very much hope no one else where had to read it review your code.
>
> In my opinion, splitting lines is much worse in terms of cognitive burden
>> than just a longer line. On top of that, it's not uncommon to reach 6, even
>> 7 indentation levels.
>>
>
> In 20+ years of programming in Python (writing some will known books about
> it too), I have never once indented a block 7 levels. Very, very rarely 6.
> If you do that, your code has serious problems beyond line length.
>
> As far as monitor size goes, I agree that on a laptop is not ideal for
>> 100+ characters per line if you want two pieces of code side by side.
>>
>
> In a readable size and font, I get two vertical panes of 71 characters on
> my 15" Retina display. I can read a smaller font character by character,
> but not easily scan a line.
>
> Here I'm just taking about visual accuity, not the cognitive issue. My
> vision in particular remains 20/20 for near vision (but sadly decayed with
> age for distance). It used to be more like 20/15, but I never used lines
> longer than 80 characters in any programming language (yes, individual
> lines, but not as a rule). I did used to use a somewhat smaller font size
> though.
>
> I guarantee that your code quidditch would improve if you use both shorter
> lines and a larger font. The latter only affects your interaction with your
> screens, the former is essential to everyone who needs to look at your code.
>
> And, fyi, I work on a 23" monitor at home and on a 15.6" monitor on the
>> go. Both are perfectly fine with 100 characters a line.
>>
>> A good middle ground would be to enforce customization of that rule in
>> the most used python linters. A simple setting to set the number of
>> characters before a linting warning occurs would be acceptable.
>>
>
> Every linter I know about is customizable this way.
>
>
>>
>>
>> Envoyé depuis mon smartphone Samsung Galaxy.
>>
>>  Message d'origine 
>> De : David Mertz 
>> Date : 19/02/2019 07:28 (GMT+01:00)
>> À : Anders Hovmöller 
>> Cc : Simon , python-ideas <
>> python-ideas@python.org>
>> Objet : Re: [Python-ideas] PEP 8 update on line length
>>
>> 60, or 68, or 80 characters, is not per se a cognitive limit. Yes, sure
>> whitespace counts much less towards that burden than do regular characters.
>> And to a significant degrees, punctuation vs. letters vs. numbers matter.
>> And sure familiar words such as keywords scan a bit easier than unfamiliar
>> names of variables. And so on.
>>
>> But 80 characters in general is already too wide most of the time. 65 is
>> a better goal as a rule of thumb, with 80 already being for exceptionally
>> long lines. Python keywords are already short, there's not much improvement
>> to be had there. Saving two characters for acryptic WHL isn't better than
>> the word 'while', for example.
>>
>> Pretty much the only time my code how's more than 80 characters is when
>> it includes string literally that occupy a large chunk of the width. But if
>> that 50 character string is the last argument of a function call, the
>> reader can mostly stop scanning at it's beginning, so it's not terrible.
>> When I have many keyword arguments, I break them into multiple physical
>> lines using parents too continue the logical line. Or if I have complex
>> compound conditions, I give the subclauses short but descriptive names
>> before the if/elif.
>>
>> On Tue, Feb 19, 2019, 1:11 AM Anders Hovmöller > wrote:
>>
>>>
>>>
>>> > On 19 Feb 2019, at 05:48, David Mertz  wrote:
>>> >
>>> > You either have much better eyes to read tiny fonts than I do, or
>>> maybe a much larger monitor (it's hard for me to fit a 30"" monitor in my
>>> laptop bag).
>>> >
>>> > But that's not even the real issue. If the characters were in giant
>>> letters on billboards, I still would never want more than 80 of them on a
>>> line (well, rarely, I violate PEP 8 sometimes). Centuries if not millennia
>>> of experience with writing show that cognitive burden goes up
>>> exponentially, not linearly, as lines get to be more than about 60

Re: [Python-ideas] PEP 8 update on line length

[David Mertz]

On Tue, Feb 19, 2019, 9:07 AM simon.bordeyne  I find that 100 to 120 characters is ideal as far as line length goes.
>

I very much hope no one else where had to read it review your code.

In my opinion, splitting lines is much worse in terms of cognitive burden
> than just a longer line. On top of that, it's not uncommon to reach 6, even
> 7 indentation levels.
>

In 20+ years of programming in Python (writing some will known books about
it too), I have never once indented a block 7 levels. Very, very rarely 6.
If you do that, your code has serious problems beyond line length.

As far as monitor size goes, I agree that on a laptop is not ideal for 100+
> characters per line if you want two pieces of code side by side.
>

In a readable size and font, I get two vertical panes of 71 characters on
my 15" Retina display. I can read a smaller font character by character,
but not easily scan a line.

Here I'm just taking about visual accuity, not the cognitive issue. My
vision in particular remains 20/20 for near vision (but sadly decayed with
age for distance). It used to be more like 20/15, but I never used lines
longer than 80 characters in any programming language (yes, individual
lines, but not as a rule). I did used to use a somewhat smaller font size
though.

I guarantee that your code quidditch would improve if you use both shorter
lines and a larger font. The latter only affects your interaction with your
screens, the former is essential to everyone who needs to look at your code.

And, fyi, I work on a 23" monitor at home and on a 15.6" monitor on the go.
> Both are perfectly fine with 100 characters a line.
>
> A good middle ground would be to enforce customization of that rule in the
> most used python linters. A simple setting to set the number of characters
> before a linting warning occurs would be acceptable.
>

Every linter I know about is customizable this way.


>
>
> Envoyé depuis mon smartphone Samsung Galaxy.
>
>  Message d'origine 
> De : David Mertz 
> Date : 19/02/2019 07:28 (GMT+01:00)
> À : Anders Hovmöller 
> Cc : Simon , python-ideas <
> python-ideas@python.org>
> Objet : Re: [Python-ideas] PEP 8 update on line length
>
> 60, or 68, or 80 characters, is not per se a cognitive limit. Yes, sure
> whitespace counts much less towards that burden than do regular characters.
> And to a significant degrees, punctuation vs. letters vs. numbers matter.
> And sure familiar words such as keywords scan a bit easier than unfamiliar
> names of variables. And so on.
>
> But 80 characters in general is already too wide most of the time. 65 is a
> better goal as a rule of thumb, with 80 already being for exceptionally
> long lines. Python keywords are already short, there's not much improvement
> to be had there. Saving two characters for acryptic WHL isn't better than
> the word 'while', for example.
>
> Pretty much the only time my code how's more than 80 characters is when it
> includes string literally that occupy a large chunk of the width. But if
> that 50 character string is the last argument of a function call, the
> reader can mostly stop scanning at it's beginning, so it's not terrible.
> When I have many keyword arguments, I break them into multiple physical
> lines using parents too continue the logical line. Or if I have complex
> compound conditions, I give the subclauses short but descriptive names
> before the if/elif.
>
> On Tue, Feb 19, 2019, 1:11 AM Anders Hovmöller 
>>
>>
>> > On 19 Feb 2019, at 05:48, David Mertz  wrote:
>> >
>> > You either have much better eyes to read tiny fonts than I do, or maybe
>> a much larger monitor (it's hard for me to fit a 30"" monitor in my laptop
>> bag).
>> >
>> > But that's not even the real issue. If the characters were in giant
>> letters on billboards, I still would never want more than 80 of them on a
>> line (well, rarely, I violate PEP 8 sometimes). Centuries if not millennia
>> of experience with writing show that cognitive burden goes up
>> exponentially, not linearly, as lines get to be more than about 60
>> characters.
>>
>> If that is the issue then we should be talking about non-space
>> characters, not a 68 column limit right? No way does 40 spaces due to
>> indent count towards the cognitive burden :)
>>
>> Also, if the cognitive burden is the issue then we should talk about
>> short forms for keyword arguments again. The way the 68 column limit works
>> in practice is that people avoid keyword arguments because of line length,
>> plus the issues already mentioned.
>>
>> / Anders
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 update on line length

[David Mertz]

60, or 68, or 80 characters, is not per se a cognitive limit. Yes, sure
whitespace counts much less towards that burden than do regular characters.
And to a significant degrees, punctuation vs. letters vs. numbers matter.
And sure familiar words such as keywords scan a bit easier than unfamiliar
names of variables. And so on.

But 80 characters in general is already too wide most of the time. 65 is a
better goal as a rule of thumb, with 80 already being for exceptionally
long lines. Python keywords are already short, there's not much improvement
to be had there. Saving two characters for acryptic WHL isn't better than
the word 'while', for example.

Pretty much the only time my code how's more than 80 characters is when it
includes string literally that occupy a large chunk of the width. But if
that 50 character string is the last argument of a function call, the
reader can mostly stop scanning at it's beginning, so it's not terrible.
When I have many keyword arguments, I break them into multiple physical
lines using parents too continue the logical line. Or if I have complex
compound conditions, I give the subclauses short but descriptive names
before the if/elif.

On Tue, Feb 19, 2019, 1:11 AM Anders Hovmöller 
>
> > On 19 Feb 2019, at 05:48, David Mertz  wrote:
> >
> > You either have much better eyes to read tiny fonts than I do, or maybe
> a much larger monitor (it's hard for me to fit a 30"" monitor in my laptop
> bag).
> >
> > But that's not even the real issue. If the characters were in giant
> letters on billboards, I still would never want more than 80 of them on a
> line (well, rarely, I violate PEP 8 sometimes). Centuries if not millennia
> of experience with writing show that cognitive burden goes up
> exponentially, not linearly, as lines get to be more than about 60
> characters.
>
> If that is the issue then we should be talking about non-space characters,
> not a 68 column limit right? No way does 40 spaces due to indent count
> towards the cognitive burden :)
>
> Also, if the cognitive burden is the issue then we should talk about short
> forms for keyword arguments again. The way the 68 column limit works in
> practice is that people avoid keyword arguments because of line length,
> plus the issues already mentioned.
>
> / Anders
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 update on line length

[David Mertz]

You either have much better eyes to read tiny fonts than I do, or maybe a
much larger monitor (it's hard for me to fit a 30"" monitor in my laptop
bag).

But that's not even the real issue. If the characters were in giant letters
on billboards, I still would never want more than 80 of them on a line
(well, rarely, I violate PEP 8 sometimes). Centuries if not millennia of
experience with writing show that cognitive burden goes up exponentially,
not linearly, as lines get to be more than about 60 characters.

This is why it's not just code on screen. Magazines, books, wall signs,
billboards, and every other written form, insists on similar limits on line
length. In printed matter this often means using multiple columns to avoid
overly long lines. The issue isn't resolution or size, it's that the human
brain simply doesn't process long lines of text will.

On Mon, Feb 18, 2019, 11:38 PM Simon  Hello,
>
> I'd like to propose an update to PEP8. Indeed, the 80 characters per line
> guideline is, I feel, outdated.
>
> I understand the need for it, back when monitors were small, and everyone
> coded on terminals, but nowadays, I feel like that guideline is more of a
> hinderance, and that it promotes poor code.
>
> Indeed, people abiding by this rule tend to choose shorter variable names,
> reduce the amount of indentation, and other tricks to just keep the
> character count under 80. I think a 100 or even 120 suggested characters
> per line count is much more realistic by today's standards. It still allow
> for the code to be completely displayed, even on just half of a screen.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Fri, Feb 8, 2019 at 3:17 PM Christopher Barker 
wrote:

> >vec_seq = Vector(seq)
> >(vec_seq * 2).name.upper()
> ># ... bunch more stuff
> >seq = vec_seq.unwrap()
>
> what type would .unwrap() return?
>

The idea—and the current toy implementation/alpha—has .unwrap return
whatever type went into the Vector creation.  Might be a tuple, list, set,
deque, or it might be an iterator.  It might even be some custom collection
that isn't in the standard library.

But you can also explicitly make a Vector into something else by using that
constructor.  Pretty much as I gave example before:

set(Vector(a_list)) # Get a set
Vector(a_list)).unwrap()# Get a list (without needing to know type
to call .unwrap())

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Thu, Feb 7, 2019 at 6:48 PM Steven D'Aprano  wrote:

> I'm sorry, I did not see your comment that you thought new syntax was a
> bad idea. If I had, I would have responded directly to that.
>

Well... I don't think it's the worst idea ever.  But in general adding more
operators is something I am generally wary about.  Plus there's the "grit
on Uncle Timmy's screen" test.

Actually, if I wanted an operator, I think that @ is more intuitive than
extra dots.  Vectorization isn't matrix multiplication, but they are sort
of in the same ballpark, so the iconography is not ruined.


> We can perform thought-experiments and
> we don't need anything but a text editor for that. As far as I'm
> concerned, the thought experiment of comparing these two snippets:
>
> ((seq .* 2)..name)..upper()
>
> versus
>
> map(str.upper, map(operator.attrgetter('name'), map(lambda a: a*2,
> seq)))
>

OK... now compare:

(Vec(seq) * 2).name.upper()

Or:

vec_seq = Vector(seq)
(vec_seq * 2).name.upper()
# ... bunch more stuff
seq = vec_seq.unwrap()

I'm not saying the double dots are terrible, but they don't read *better*
than wrapping (and optionally unwrapping) to me.

If we were to take @ as "vectorize", it might be:

(seq @* 2) @.name @.upper()

I don't hate that.

demonstrates conclusively that even with the ugly double dot syntax,
> infix syntax easily and conclusively beats map.
>

Agreed.


> If I recall correctly, the three maps here were originally proposed by
> you as examples of why map() alone was sufficient and there was no
> benefit to the Julia syntax.


Well... your maps are kinda deliberately ugly.  Even in that direction, I'd
write:

map(lambda s: (s*2).name.upper(), seq)

I don't *love* that, but it's a lot less monstrous than what you wrote.  A
comprehension probably even better:

[(s*2).name.upper() for s in seq]

Again, I apologise, I did not see where you said that this was intended
> as a proof-of-concept to experiment with the concept.
>

All happy.  Puppies and flowers.


> If the Vector class is only a proof of concept, then we surely don't
> need to care about moving things in and out of "vector mode". We can
> take it as a given that "the real thing" will work that way: the syntax
> will be duck-typed and work with any iterable,


Well... I at least moderately think that a wrapper class is BETTER than new
syntax. So I'd like the proof-of-concept to be at least moderately
functional.  In any case, there is ZERO code needed to move in/out of
"vector mode." The wrapped thing is simply an attribute of the object.
When we call vectorized methods, it's just `getattr(type(item), attr)` to
figure out the method in a duck-typed way.

one of the things which lead me to believe that you thought that a
> wrapper class was in and of itself a solution to the problem. If you had
> been proposing this Vector class as a viable working solution (or at
> least a first alpha version towards a viable solution) then worrying
> about round-tripping would be important.
>

Yes, I consider the Vector class a first alpha version of a viable
solution.  I haven't seen anything that makes me prefer new syntax.  I feel
like a wrapper makes it more clear that we are "living in vector land" for
a while.

The same is true for NumPy, in my mind.  Maybe it's just familiarity, but I
LIKE the fact that I know that when my object is an ndarray, operations are
going to be vectorized ones.  Maybe 15 years ago different decisions could
have been made, and some "vectorize this operation syntax" could have made
the ndarray structure just a behavior of lists instead.  But I think the
separation is nice.


> But as a proof-of-concept of the functionality, then:
>
> set( Vector(set_of_stuff) + spam )
> list( Vector(list_of_stuff) + spam )
>

That's fine.  But there's no harm in the class *remembering* what it wraps
either.  We might want to distinguish:

set(Vector(some_collection) + spam) # Make it a set after
the operations
(Vector(some_collection) + spam).unwrap()  # Recover whatever type it
was before


> Why do you care about type uniformity or type-checking the contents of
> the iterable?
>

Because some people have said "I want my vector to be specifically a
*sequence of strings* not of other stuff"

And MAYBE there is some optimization to be had if we know we'll never have
a non-footype in the sequence (after all, NumPy is hella optimized).
That's why the `stringpy` name that someone suggested.  Maybe we'd bypass
most of the Python-land calls when we did the vectorized operations, but
ONLY if we assume type uniformity.

But yes, I generally care about duck-typing only.


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
__

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

Many apologies if people got one or more encrypted versions of this.

On 2/7/19 12:13 AM, Steven D'Aprano wrote:

It wasn't a concrete proposal, just food for thought. Unfortunately the
thinking seems to have missed the point of the Julia syntax and run off
with the idea of a wrapper class.

I did not miss the point! I think adding new syntax à la Julia is a bad
idea—or at very least, not something we can experiment with today (and
wrote as much).

Therefore, something we CAN think about and experiment with today is a
wrapper class.  This approach is pretty much exactly the same thing I
tried in a discussion of PEP 505 a while back (None-aware operators).
In the same vein as that—where I happen to dislike PEP 505 pretty
strongly—one approach to simulate or avoid new syntax is precisely to
use a wrapper class.

As a footnote, I think my demonstration of PEP 505 got derailed by lots
of comments along the lines of "Your current toy library gets the
semantics of the proposed new syntax wrong in these edge cases."  Those
comments were true (and I think I didn't fix all the issues since my
interest faded with the active thread)... but none of them were
impossible to fix, just small errors I had made.

With my *very toy* stringpy.Vector class, I'm just experimenting with
usage ideas.  I have shown a number of uses that I think could be useful
to capture most or all of what folks want in "string vectorization."
Most of what I've but in this list is what the little module does
already, but some is just ideas for what it might do if I add the code
(or someone else makes a PR at https://github.com/DavidMertz/stringpy).

One of the principles I had in mind in my demonstration is that I want
to wrap the original collection type (or keep it an iterator if it
started as one).  A number of other ideas here, whether for built-in
syntax or different behaviors of a wrapper, effectively always reduce
every sequence to a list under the hood.  This makes my approach less
intrusive to move things in and out of "vector mode."  For example:

  v1 = Vector(set_of_strings)
  set_of_strings = v1.lower().apply(my_str_fun)._it  # Get a set back
  v2 = Vector(list_of_strings)
  list_of_strings = v2.lower().apply(my_str_fun)._it # Get a list back
  v3 = Vector(deque_of_strings)
  deque_of_strings = v3.lower().apply(my_str_fun)._it # Get a deque back
  v4 = Vector(iter_of_strings)
  iter_of_strings = v4.lower().apply(my_str_fun)._it  # stays lazy!

So this is round-tripping through vector-land.

Small note: I use the attribute `._it` to store the "sequential thing."
 That feels internal, so maybe some better way of spelling "get the
wrapped thing" would be desirable.

I've also lost track of whether anyone is proposing a "vector of strings'
as opposed to a vector of arbitrary objects.

Nothing I wrote is actually string-specific.  That is just the main use
case stated.  My `stringpy.Vector` might be misnamed in that it is happy
to contain any kind of items.  But we hope they are all items with the
particular methods we want to vectorize.  I showed an example where a
list might contain a custom string-like object that happens to have
methods like `.lower()` as an illustration.

Inasmuch as I want to handle iterator here, it is impossible to do any
type check upon creating a Vector.  For concrete
`collections.abc.Sequence` objects we could check, in principle.  But
I'd rather it be "we're all adults here" ... or at most provide some
`check_type_uniformity()` function or method that had to be called
explicitly.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Mon, Feb 4, 2019 at 7:14 AM Kirill Balunov 
wrote:

> len(v)   # -> 12
>
> v[len]   # -> 
>
>
> In this case you can apply any function, even custom_linked_list from
> my_inhouse_module.py.
>

I think I really like this idea.  Maybe as an extra spelling but still
allow .apply() to do the same thing. It feels reasonably intuitive to me.
Not *identical to* indexing in NumPy and Pandas, but sort of in the same
spirit as predicative or selection based indices.

What do other people on this thread think? Would you learn that easily?
Could you teach it?


> >>> v[1:]
>  'Nov', 'Dec']>
> >>> v[i[1:]] # some helper class `i`
>  'ov', 'ec']>
>
>
This feels more forced, unfortunately.  Something short would be good, but
not sure I like this.  This is really just a short spelling of
pandas.IndexSlice or numpy.s_  It came up in another thread some months
ago, but there is another proposal to allow the obvious spelling
`slice[start:stop:sep]` as a way of creating slices.

Actually, I guess that's all halfway for the above.  We'd need to do this
still:

v[itemgetter(IndexSlicer[1:])]


That's way too noisy.  I guess I just don't find the lowercase `i` to be
iconic enough.  I think with a better SHORT name, I'd like:

v[Item[1:]]


 Maybe that's not the name?

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Mon, Feb 4, 2019, 12:47 AM Christopher Barker

> I've lost track if who is advocating what, but:
>

Well, I made a toy implementation of a Vector class. I'm not sure what that
means I advocate other than the existence of a module on GitHub.

FWIW, I called the repo 'stringpy' as a start, so that expressed some
interest in it being about vectors of strings. But so-far, I haven't found
anything that actually needs to be string-like.  In general, methods get
passed through to their underlying objects and deliberately duck typed,
like:

v.replace("a", "b")
>>
>
As an extra, we could enforce homogeneity, or even string-nesss
specifically. I don't really know what homogeneity means though, once we
consider ABCs, subclasses, and duck types that don't use inheritance on r
ABC registration. At least so far, I haven't coded anything that would get
a performance gain from enforcing the string-nesss of items (but all pure
Python so far, no Cython or C)

This is adding something - maybe just compactness, but I also think
> readability.
>

I think with changed methods the win gets greater:

v.replace("a", "b").upper().apply(myfun)

If you want to do any generic items, it becomes a lot harder.
>

So far, generic has been a lot easier to code than hand-rolled methods.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

How would you spell these with funcoperators?

v.replace("a","b").upper().count("B")
vec1.replace("PLACEHOLDER", vec2)
concat = vec1 + vec2

On Sun, Feb 3, 2019, 6:40 PM Robert Vanden Eynde 
>
> On Sat, 2 Feb 2019, 21:46 Brendan Barnwell 
> Yeah, it's called pip install funcoperators :
>
>>  some_list @ str.lower @ tokenize @ remove_stopwords
>>
>
> → some_list @ to(str.lower) @ to(tokenize) @ to(remove_stopwords)
>
> Where from funcoperators import postfix as to
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Sun, Feb 3, 2019, 6:36 PM Greg Ewing

> But they only cover the special case of a function that takes
> elements  from just one input vector. What about one that takes
> coresponding elements from two or more vectors?
>

What syntax would you like? Not necessarily new syntax per se, but what
calling convention.

I can think of a few useful cases.

vec1.replace("PLACEHOLDER", vec2)

Maybe that would transform one vector using the corresponding strings from
another vector.

What should happen if the vector length mismatch? I think this should
probably be an exception... unlike what zip() and itertools.zip_longest()
do. But maybe not.

concat = vec1 + vec2

Again the vector length question is there. But assuming the same length,
this seems like a reasonable way to get a new vector concatenating each
corresponding element.

Other uses? Are they different in general pattern?

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

>
> >>> len(v)  # Number of elements in the Vector `v`
>

Agreed, this should definitely be the behavior.  So how do we get a vector
of lengths of each element?


> >>> # Compute the length of each element of the Vector `v`
> >>> v.apply(len)
> >>> v @ len
>

Also possible is:

v.len()

We couldn't do that for every possible function, but this one is special
inasmuch as we expect the items each to have a .__len__() but don't want to
spell the dunders. Likewise for just a handful of other methods/functions.

The key different though is that *I* would want to a way to use both
methods already attached to the objects/items. in a vector and also a
generic user-provided function that operates on the items. I guess you
disagree about "method pass-through" but it reads more elegantly to me:

>>> # Replace all "a" by "b"
> >>> v.apply(lambda s: s.replace("a", "b"))
> >>> v @ (lambda s: s.replace("a", "b"))
>

Compare these with:

v.replace("a", "b")

Since we already know v is a Vector, we kinda expect methods to be
vectorized.  This feels like the "least surprise" and also the least extra
code.  Moreover, spelling chained methods with many .appy() calls (even if
spelled '@') feels very cumbersome:

(A) v.apply(lambda s: s.replace("a", "b")).apply(str.upper).apply(lambda s:
s.count("B"))

(B) v @ lambda s: s.replace("a", "b") @ str.upper  @ lambda s: s.count("B")

(C) v.replace("a","b").upper().count("B")

Between these, (C) feels a heck of a lot more intuitive and readable to me.

Here we put an emphasis on the methods already attached to objects.  But
this isn't terrible:

def double(x):
return x*2
v.apply(double).replace("a","b").upper().count("B")

In @ notation it would be:

v @ double @ lambda s: s.replace("a", "b") @ str.upper  @ lambda s:
s.count("B")

The 'double' is slightly easier, but the method calls are much worse.

MOREOVER, the model of "everything is apply/@" falls down terribly once we
have duck typing.

This is a completely silly example, but it's one that apply/@ simply cannot
address because it assumes it is the SAME function/method applied to each
object:

>>> class CaseInsensitiveStr(str):
... def replace(self, old, new):
... return str.upper(self).replace(old.upper(), new.upper())
...
>>> l = ['Monday', CaseInsensitiveStr('Tuesday'), 'Wednesday']
>>> v = Vector(l)
>>> v.replace('day', 'time')





-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Sun, Feb 3, 2019 at 3:16 PM Ronald Oussoren 
wrote:

> The @ operator is meant for matrix multiplication (see PEP 465) and is
> already used for that in NumPy. IMHO just that is a good enough reason for
> not using @ as an elementwise application operator (ignoring if having an
> such an operator is a good idea in the first place).
>

Co-opting operators is pretty common in Python.  For example, the
`.__div__()` operator spelled '/' is most often used for some kind of
numeric division.  Some variations on that, for example vectorized in
NumPy.  And different numeric types operate a bit differently.  The name of
the magic method obvious suggests division.

And yet, in the standard library we have pathlib which we can use like this
(from the module documentation):

>>> p = Path('/etc')>>> q = p / 'init.d' / 'reboot'

That use is reasonable and iconic, even if it is nothing like division.

The `.__mod__()` operator spelled '%' means something very different in
relation to a float or int object versus a string object.  I.e. modulo
division versus string interpolation.

I've even seen documentation of some library that coopts `.__matmul__()` to
do something with email addresses.  It's not a library I use, just
something I once saw the documentation on, so I'm not certain of details.
But you can imagine that e.g. :

email = user @ domain


Could be helpful and reasonable (exact behavior and purpose could vary, but
it's "something about email" iconically).

In other words, I'm not opposed to using the @ operator in my
stringpy.Vector class out of purity about the meaning of operators.  I just
am not convinced that it actually adds anything that is not easier without
it.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What factors led Guido to quit?

[David Mertz]

On Sun, Feb 3, 2019 at 1:32 PM Ryan Gonzalez  wrote:

> - I seriously doubt he's going to come back.
>

The election for Steering Council is underway, and Guido is one of the
candidates. He may or may not be one of 5 members of the SC, but that's up
to the voters among the core committers.  But if Guido wins, he will be one
of several serving for a finite terms (but obviously with great wisdom and
experience to lend to the SC).

Let's just respect Guido's decision to modify his role, and give great
thanks to his huge contributions to the world over decades.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Sun, Feb 3, 2019 at 1:38 PM Adrien Ricocotam  wrote:

> I honestly don’t understand what you don’t like the @ syntax.
>

Can you show any single example that would work with the @ syntax that
would not work in almost exactly the same way without it? I have not seen
any yet, and none seem obvious.  Adding new syntax for its own sake is
definitely to be avoided when possible (even though technically the
operator exists, so it wouldn't be actual new syntax).


> My idea is using functions that takes on argument : an object of the type
> of the vector. That’s actually how map works.
>

I do not understand this.  Spell my simple example using @ notation.  I.e.

my_vec @ replace {something? here for 'foo' with 'bar'}



> What I understood from your previous message is that there’s ambiguity
> when using magic functions on whether it’s applied to each element of the
> vector or the vector itself. That was the first thing I saw.
>

I decided there really isn't.  I think that any function applied to the
vector should operate on the sequence as a whole.  E.g. what length does it
have? Cast it to a different kind of sequence. Print it out. Serialize it.
Etc.

The things that are vectorized should always be methods of the vector
instead.  And ALMOST every method should in fact be a vectorized
operation.  In most cases, those will be a "pass through" to the methods of
the items inside of the vector. We won't write every possible method in the
Vector class.

My toy so far only works with methods that the items actually have.  In the
examples, string methods.  But actually, I should add one method like this:

my_vec.apply(lambda x: x*2)


That is, we might want to vectorize custom functions also.  Maybe in that
example we should name the function 'double' for clarity: '
my_vec.apply(double)'.  I do think that just a few methods need to be
custom programmed because they correspond to magic methods of the items
rather than regular names (or not even directly to magic methods, but more
machinery).  So:

my_vec.list()  #-> cast each item to a list

my_vec.tuple() #-> cast each item to a tuple

my_vec.set()   #-> cast each item to a set


Maybe that's doing too much though.  We could always do that with map() or
comprehensions; it's not clear it's a common enough use case.

Functions that could be used are then the same we can use in map. But I do
> agree it’s not easy to have functions with parameters. That’s why I used
> functools.partial
>

I really did not understand how that was meant to work.  But it was a whole
lot of lines to accomplish something very small either way.


> On Sun 3 Feb 2019 at 19:23, David Mertz  wrote:
>
>> On Sun, Feb 3, 2019 at 3:54 AM Adrien Ricocotam 
>> wrote:
>>
>>> I think all the issues you have right now would go of using another
>>> operation. I proposed the @ notation that is clear and different from
>>> everything else,
>>>
>> plus the operator is called "matmul" so it completely makes sense. The
>>> the examples would be :
>>>
>>
>>
>>> >>> l = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
>>> >>> v = Vector(l)
>>> >>> len(v)
>>> 12
>>> >>> v @ len
>>> 
>>>
>>
>> I cannot really see how using the @ operator helps anything here.  If
>> this were a language that isn't Python (or conceivably some future version
>> of Python, but that doesn't feel likely or desirable to me), I could
>> imagine @ as an operator to vectorize any arbitrary sequence (or
>> iterator).  But given that we've already made the sequence into a Vector,
>> there's no need for extra syntax to say it should act in a vectorized way.
>>
>> Moreover, your syntax is awkward for methods with arguments.  How would I
>> spell:
>>
>> v.replace('foo', 'bar')
>>
>>
>> In the @ syntax? I actually made an error on my first pass where simply
>> naming a method was calling it.  I thought about keeping it for a moment,
>> but that really only allows zero argument calls.
>>
>> I think the principled thing to do here is add the minimal number of
>> methods to Vector itself, and have everything else pass through as
>> vectorized calls.  Most of that minimal number are "magic method":
>> __len__(), __contains__(), __str__(), __repr__(), __iter__(),
>> __reversed__().  I might have forgotten a couple.  All of those should not
>> be called directly, normally, but act as magic for operators or built-in
>> functions.
>>
>> I think I should then create regular methods of the same name t

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Sun, Feb 3, 2019 at 3:54 AM Adrien Ricocotam  wrote:

> I think all the issues you have right now would go of using another
> operation. I proposed the @ notation that is clear and different from
> everything else,
>
plus the operator is called "matmul" so it completely makes sense. The the
> examples would be :
>


> >>> l = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
> >>> v = Vector(l)
> >>> len(v)
> 12
> >>> v @ len
> 
>

I cannot really see how using the @ operator helps anything here.  If this
were a language that isn't Python (or conceivably some future version of
Python, but that doesn't feel likely or desirable to me), I could imagine @
as an operator to vectorize any arbitrary sequence (or iterator).  But
given that we've already made the sequence into a Vector, there's no need
for extra syntax to say it should act in a vectorized way.

Moreover, your syntax is awkward for methods with arguments.  How would I
spell:

v.replace('foo', 'bar')


In the @ syntax? I actually made an error on my first pass where simply
naming a method was calling it.  I thought about keeping it for a moment,
but that really only allows zero argument calls.

I think the principled thing to do here is add the minimal number of
methods to Vector itself, and have everything else pass through as
vectorized calls.  Most of that minimal number are "magic method":
__len__(), __contains__(), __str__(), __repr__(), __iter__(),
__reversed__().  I might have forgotten a couple.  All of those should not
be called directly, normally, but act as magic for operators or built-in
functions.

I think I should then create regular methods of the same name that perform
the vectorized version.  So we would have:

len(v)   # -> 12

v.len()  # -> 

list(v)  # -> ["Jan", "Feb", "Mar", "Apr", "May", "Jul" ...]
v.list() # -> 


I can't implement every single constructor that users might
conceivably want, of course, but I can do it for the basic types in
builtins and common standard library.  E.g. I might do:

v.deque() # -> 


But I certainly won't manually add:

v.custom_linked_list()  # From my_inhouse_module.py


Hmm... maybe even I could look at names of maybe-constructors in the
current namespace and try them.  That starts to feel too magic.  Falling
back to this feels better:

map(custom_linked_list, v)  # From my_inhouse_module.py



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What factors led Guido to quit?

[David Mertz]

Guido left for a variety of personal reasons, only some of which I know.
Soon we will have a wonderful new Steering Council who will act as a
collective BDFL (except each for finite terms, and not dictator since it's
a council... but benevolent :-)).

On Sun, Feb 3, 2019, 12:33 PM James Lu  Are you a moderator or committed?
>
> I disagree with you. This, like the communication thread, is about
> improving the ability of the Python community to work together. If you
> don’t think others can have a respectful discussion about that, that’s the
> community’s fault, not mine.
>
> Again, the goal of this thread is for constructive discussion.
>
> Sent from my iPhone
>
> > On Feb 3, 2019, at 9:44 AM, Antoine Pitrou  wrote:
> >
> >
> > James, please don't post such topics on python-ideas.  This is
> > completely off-topic and has strong flamebait potential.  You may write
> > on your own blog or Facebook account if you feel so strongly about it.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > On Sat, 2 Feb 2019 11:04:40 -0500
> > James Lu  wrote:
> >> I want y’all to think about this very carefully. What factors led Guido
> to quit? And I don’t want you to just reply with the first thing that comes
> off your head. The purpose of this question/discussion is to identify
> problems with the Python community so we can fix them.
> >>
> >> That is the only real chance of having a successful Python language
> that doesn’t get left behind by other languages, or of getting Guido to
> come back.
> >> ___
> >> Python-ideas mailing list
> >> Python-ideas@python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >
> >
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

>
> I think it should follow the pre-existing behaviour of list, set, tuple,
> etc.
>
>  >>> Vector("hello")
> 
>

I try to keep the underlying datatype of the wrapped collection as much
as possible.  Casting a string to a list changes that.

>>> Vector(d)

>>> Vector(tuple(d))

>>> Vector(set(d))

>>> from collections import deque
>>> Vector(deque(d))



Strings are already a Collection, there is not firm need cast them to a
list to live inside a Vector.  I like the idea of maintaining the original
type if someone wants it back later (possibly after transformations of the
values).

Why is it pointless for a vector, but not for a list?
>

I guess it really isn't.  I was thinking of just .upper() and .lower()
where upper/lower-casing each individual letter is the same as doing so to
the whole string.  But for .replace() or .count() or .title() or
.swapcase() the meaning is very different if it is letter-at-a-time.

I guess a string gets unstringified pretty quickly no matter what though.
E.g. this seems like right behavior once we transform something:

>>> vstr = Vector('Monday')
>>> vstr

>>> vstr.upper()



I dunno... I suppose I *could* do `self._it = "".join(self._it)` whenever I
do a transform on a string to keep the underlying iterable as a string.
But the point of a Vector really is sequences of strings not sequences of
characters.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Sat, Feb 2, 2019 at 10:00 PM MRAB  wrote:

> Perhaps a reserved attribute that let's you refer to the vector itself
> instead of its members, e.g. '.self'?
>
> >>> len(v)
> 
>  >>> len(v.self)
> 12
>

I like that! But I'm not sure if '.self' is misleading.  I use an attribute
called '._it' already that does exactly this.  But since we're asking the
length of the list or tuple or set or deque or etc that the Vector wraps,
does it feel like it would be deceptive to call them all '.self'?

I'm really not sure.  I could just rename '._it' to '.self' and get the
behavior you show (well, I still need a little checking whether the thing
wrapped is a collection or an iterator ... I guess a '.self' property.  Or
some other name to do that).

You remind me that I need to at .__getitem__() too so I can slice and index
Vectors.  But I know how to do that easily enough.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

Trying to make iterators behave in a semi-nice way also. I kinda like this
(example remains silly, but it shows idea).

>>> for n, mon in enumerate(vi.upper().replace('J','_').title()):
... print(mon)
... if n>3: break
...
...
_An
Feb
Mar
Apr
May
>>> vi
>
>>> list(vi)
['Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']
>>> vi
>
>>> list(vi)
[]


On Sat, Feb 2, 2019 at 9:03 PM David Mertz  wrote:

> Slightly more on my initial behavior:
>
> >>> Vector({1:2,3:4})
> TypeError: Ambiguity vectorizing a map, perhaps try it.keys(),
> it.values(), or it.items()
>
> >>> Vector(37)
> TypeError: Vector can only be initialized with an iterable
>
> >>> Vector("hello")
> 
>
>
> I'm wondering if maybe making a vector out of a scalar should simply be a
> length-one vector. What do you think?
>
> Also, should a single string be treated like a vector of characters or
> like a scalar? It feels kinda pointless to make a vector of characters
> since I cannot think of anything it would do better than a plain string
> already does (largely just the same thing slower).
>
> On Sat, Feb 2, 2019 at 8:54 PM David Mertz  wrote:
>
>> Here is a very toy proof-of-concept:
>>
>> >>> from vector import Vector
>> >>> l = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
>> >>> v = Vector(l)
>> >>> v
>> > 'Sep', 'Oct', 'Nov', 'Dec']>
>> >>> v.strip().lower().replace('a','X')
>> > 'sep', 'oct', 'nov', 'dec']>
>> >>> vt = Vector(tuple(l))
>> >>> vt
>> > 'Sep', 'Oct', 'Nov', 'Dec')>
>> >>> vt.lower().replace('o','X')
>> > 'sep', 'Xct', 'nXv', 'dec')>
>>
>>
>> My few lines are at https://github.com/DavidMertz/stringpy
>>
>> One thing I think I'd like to be different is to have some way of
>> accessing EITHER the collection being held OR each element.  So now I just
>> get:
>>
>> >>> v.__len__()
>> 
>>
>>
>> Yes, that's an ugly spelling of `len(v)`, but let's bracket that for the
>> moment.  It would be nice also to be able to ask "what's the length of the
>> vector, in a non-vectorized way" (i.e. 12 in this case).  Maybe some naming
>> convention like:
>>
>> >>> v.collection__len__()
>> 12
>>
>>
>> This last is just a possible behavior, not in the code I just uploaded.
>>
>>
>> On Sat, Feb 2, 2019 at 6:47 PM Chris Angelico  wrote:
>>
>>> On Sun, Feb 3, 2019 at 10:36 AM Ben Rudiak-Gould 
>>> wrote:
>>> >
>>> > On Sat, Feb 2, 2019 at 3:23 PM Christopher Barker 
>>> wrote:
>>> >>
>>> >> a_list_of_strings.strip().lower().title()
>>> >>
>>> >> is a lot nicer than:
>>> >>
>>> >> [s.title() for s in (s.lower() for s in [s.strip(s) for s in
>>> a_list_of_strings])]
>>> >>
>>> >> or
>>> >>
>>> >> list(map(str.title, (map(str.lower, (map(str.strip,
>>> a_list_of_strings # untested
>>> >
>>> > In this case you can write
>>> >
>>> > [s.strip().lower().title() for s in a_list_of_strings]
>>>
>>> What if it's a more complicated example?
>>>
>>> len(sorted(a_list_of_strings.casefold())[:100])
>>>
>>> where the len() is supposed to give back a list of the lengths of the
>>> first hundred strings, sorted case insensitively? (Okay so it's a
>>> horrible contrived example. Bear with me.)
>>>
>>> With current syntax, this would need multiple map calls or
>>> comprehensions:
>>>
>>> [len(s) for s in sorted(s.casefold() for s in a_list_of_strings)[:100]]
>>>
>>> (Better examples welcomed.)
>>>
>>> ChrisA
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>> --
>> Keeping medicines from the bloodstreams of the sick; food
>> from the bellies of the hungry; books from the hands of the
>> uneducated; technology from the underdeveloped; and putting
>> advocates of freedom in prisons.  Intellectual property is
>> to the 21st century what the slave trade was to the 16th.
>>
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

Slightly more on my initial behavior:

>>> Vector({1:2,3:4})
TypeError: Ambiguity vectorizing a map, perhaps try it.keys(), it.values(),
or it.items()

>>> Vector(37)
TypeError: Vector can only be initialized with an iterable

>>> Vector("hello")



I'm wondering if maybe making a vector out of a scalar should simply be a
length-one vector. What do you think?

Also, should a single string be treated like a vector of characters or like
a scalar? It feels kinda pointless to make a vector of characters since I
cannot think of anything it would do better than a plain string already
does (largely just the same thing slower).

On Sat, Feb 2, 2019 at 8:54 PM David Mertz  wrote:

> Here is a very toy proof-of-concept:
>
> >>> from vector import Vector
> >>> l = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
> >>> v = Vector(l)
> >>> v
>  'Oct', 'Nov', 'Dec']>
> >>> v.strip().lower().replace('a','X')
>  'oct', 'nov', 'dec']>
> >>> vt = Vector(tuple(l))
> >>> vt
>  'Oct', 'Nov', 'Dec')>
> >>> vt.lower().replace('o','X')
>  'Xct', 'nXv', 'dec')>
>
>
> My few lines are at https://github.com/DavidMertz/stringpy
>
> One thing I think I'd like to be different is to have some way of
> accessing EITHER the collection being held OR each element.  So now I just
> get:
>
> >>> v.__len__()
> 
>
>
> Yes, that's an ugly spelling of `len(v)`, but let's bracket that for the
> moment.  It would be nice also to be able to ask "what's the length of the
> vector, in a non-vectorized way" (i.e. 12 in this case).  Maybe some naming
> convention like:
>
> >>> v.collection__len__()
> 12
>
>
> This last is just a possible behavior, not in the code I just uploaded.
>
>
> On Sat, Feb 2, 2019 at 6:47 PM Chris Angelico  wrote:
>
>> On Sun, Feb 3, 2019 at 10:36 AM Ben Rudiak-Gould 
>> wrote:
>> >
>> > On Sat, Feb 2, 2019 at 3:23 PM Christopher Barker 
>> wrote:
>> >>
>> >> a_list_of_strings.strip().lower().title()
>> >>
>> >> is a lot nicer than:
>> >>
>> >> [s.title() for s in (s.lower() for s in [s.strip(s) for s in
>> a_list_of_strings])]
>> >>
>> >> or
>> >>
>> >> list(map(str.title, (map(str.lower, (map(str.strip,
>> a_list_of_strings # untested
>> >
>> > In this case you can write
>> >
>> > [s.strip().lower().title() for s in a_list_of_strings]
>>
>> What if it's a more complicated example?
>>
>> len(sorted(a_list_of_strings.casefold())[:100])
>>
>> where the len() is supposed to give back a list of the lengths of the
>> first hundred strings, sorted case insensitively? (Okay so it's a
>> horrible contrived example. Bear with me.)
>>
>> With current syntax, this would need multiple map calls or comprehensions:
>>
>> [len(s) for s in sorted(s.casefold() for s in a_list_of_strings)[:100]]
>>
>> (Better examples welcomed.)
>>
>> ChrisA
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

Here is a very toy proof-of-concept:

>>> from vector import Vector
>>> l = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
>>> v = Vector(l)
>>> v

>>> v.strip().lower().replace('a','X')

>>> vt = Vector(tuple(l))
>>> vt

>>> vt.lower().replace('o','X')



My few lines are at https://github.com/DavidMertz/stringpy

One thing I think I'd like to be different is to have some way of accessing
EITHER the collection being held OR each element.  So now I just get:

>>> v.__len__()



Yes, that's an ugly spelling of `len(v)`, but let's bracket that for the
moment.  It would be nice also to be able to ask "what's the length of the
vector, in a non-vectorized way" (i.e. 12 in this case).  Maybe some naming
convention like:

>>> v.collection__len__()
12


This last is just a possible behavior, not in the code I just uploaded.


On Sat, Feb 2, 2019 at 6:47 PM Chris Angelico  wrote:

> On Sun, Feb 3, 2019 at 10:36 AM Ben Rudiak-Gould 
> wrote:
> >
> > On Sat, Feb 2, 2019 at 3:23 PM Christopher Barker 
> wrote:
> >>
> >> a_list_of_strings.strip().lower().title()
> >>
> >> is a lot nicer than:
> >>
> >> [s.title() for s in (s.lower() for s in [s.strip(s) for s in
> a_list_of_strings])]
> >>
> >> or
> >>
> >> list(map(str.title, (map(str.lower, (map(str.strip,
> a_list_of_strings # untested
> >
> > In this case you can write
> >
> > [s.strip().lower().title() for s in a_list_of_strings]
>
> What if it's a more complicated example?
>
> len(sorted(a_list_of_strings.casefold())[:100])
>
> where the len() is supposed to give back a list of the lengths of the
> first hundred strings, sorted case insensitively? (Okay so it's a
> horrible contrived example. Bear with me.)
>
> With current syntax, this would need multiple map calls or comprehensions:
>
> [len(s) for s in sorted(s.casefold() for s in a_list_of_strings)[:100]]
>
> (Better examples welcomed.)
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

My double dot was a typo on my tablet, not borrowing Julia syntax, in this
case.

On Sat, Feb 2, 2019, 6:43 PM David Mertz  On Sat, Feb 2, 2019, 6:23 PM Christopher Barker
>
>> a_list_of_strings.strip().lower().title()
>>
>> is a lot nicer than:
>>
>> [s.title() for s in (s.lower() for s in [s.strip(s) for s in
>> a_list_of_strings])]
>>
>> or
>>
>> list(map(str.title, (map(str.lower, (map(str.strip, a_list_of_strings
>> # untested
>>
>
> I'm warming up some. But is this imagined as vectors of strings, or as
> generically homogeneous objects? And what is homogeneity exactly in the
> face of duck typing?
>
> Absent the vector wrapping, I think I might write this for your example:
>
> map(lambda s: s..strip().lower().title(), a_list_of_strings)
>
> That's slightly longer, but just by the length of the word lambda.
>
> One could write a wrapper to vectorize pretty easily. So maybe:
>
> Vector(a_list_of_strings).strip().lower().title()
>
> This would just pass along the methods to the individual items, and
> wouldn't need to think about typing per se. Maybe other objects happen to
> have those three methods, so are string-like in a duck way.
>
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Sat, Feb 2, 2019, 6:23 PM Christopher Barker

> a_list_of_strings.strip().lower().title()
>
> is a lot nicer than:
>
> [s.title() for s in (s.lower() for s in [s.strip(s) for s in
> a_list_of_strings])]
>
> or
>
> list(map(str.title, (map(str.lower, (map(str.strip, a_list_of_strings
> # untested
>

I'm warming up some. But is this imagined as vectors of strings, or as
generically homogeneous objects? And what is homogeneity exactly in the
face of duck typing?

Absent the vector wrapping, I think I might write this for your example:

map(lambda s: s..strip().lower().title(), a_list_of_strings)

That's slightly longer, but just by the length of the word lambda.

One could write a wrapper to vectorize pretty easily. So maybe:

Vector(a_list_of_strings).strip().lower().title()

This would just pass along the methods to the individual items, and
wouldn't need to think about typing per se. Maybe other objects happen to
have those three methods, so are string-like in a duck way.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

Beyond possibly saving 3-5 characters, I continue not to see anything
different from map in this discussion.

list(vector) applies list to the vector itself.
> list.(vector) applies list to each component of vector.
>

In Python:

list(seq) applies list to the sequence itself
map(list, seq) applies list to each component of seq

In terms of other examples:

map(str.upper, seq) uppercases each item
map(operator.attrgetter('name'), seq) gets the name attribute of each item
map(lambda a: a*2, seq) doubles each item
(lambda a: a*2)(seq) doubles the sequence itself

... Last two might enjoy named function 'double'




> > The problem, of course, is that list() now has to understand Vector
> > specially, and so does any function you think of applying to it.
>
> *The whole point* of the Julia syntax is that no function has to
> understand any sequence. When we write:
>
> for item in vector:
> func(item)
>
> func only has to understand item, not vector. The same applies to the
> Julia syntax
>
> func.(vector)
>
> There's no puzzle here, no tricky cases, because it is completely
> deterministic and explicit: func(x) always calls func with x as
> argument, func.(x) always calls func with each of x's items as
> arguments.
>
>
>
> > Operators are easier (even those like [1:]) because Vector can make its
> > own definition of each through (a finite set of) dunder methods. To make
> > a Vector accept an arbitrarily-named method call like my_strings.upper()
> > to mean:
>
> With the Julia syntax, there is no need for vectors (or lists, or
> generators, or tuples, or sets, or any other iterator...) to accept
> arbitrary method calls. So long as vectors can be iterated over,
> func.(vector) will work.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

I still haven't seen any examples that aren't already spelled 'map(fun, it)'

On Sat, Feb 2, 2019, 3:17 PM Jeff Allen  On 02/02/2019 18:44, MRAB wrote:
>
> On 2019-02-02 17:31, Adrien Ricocotam wrote:
> > I personally would the first option to be the case. But then vectors
> shouldn't be list-like but more generator like.
> >
> OK, here's another one: if you use 'list(...)' on a vector, does it apply
> to the vector itself or its members?
>
> >>> list(my_strings)
>
> You might be wanting to convert a vector into a list:
>
> ['one', 'two', 'three']
>
> or convert each of its members onto lists:
>
> Vector([['one'], ['two'], ['three']])
>
> More likely you mean:
>
> >>> [list(i) for i in ['one', 'two', 'three']]
> [['o', 'n', 'e'], ['t', 'w', 'o'], ['t', 'h', 'r', 'e', 'e']]
>
> The problem, of course, is that list() now has to understand Vector
> specially, and so does any function you think of applying to it. Operators
> are easier (even those like [1:]) because Vector can make its own
> definition of each through (a finite set of) dunder methods. To make a
> Vector accept an arbitrarily-named method call like my_strings.upper() to
> mean:
>
> >>> [i.upper() for i in ['one', 'two', 'three']]
> ['ONE', 'TWO', 'THREE']
>
> is perhaps just about possible by manipulating __getattribute__ to resolve
> names matching methods on the underlying type to a callable that loops over
> the content.
>
> Jeff
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Option of running shell/console commands inside the REPL

[David Mertz]

On Sat, Feb 2, 2019, 8:15 AM Oleg BroytmanFor the question "Does Python REPL need more batteries?" is your
> answer "No, just point people to IDLE"?
>If it is - well, I disagree. I implemented a lot of enhancements for
> REPL myself, and I don't like and avoid GUI programs
>

IPython and BPython and Xonsh are projects that exist. I doubt the Python
REPL will ever be as full featured as those, and I don't really think it
should hope to. Similarly for IDLE or PyCharm or Spyder or Jupyter for
folks who like GUIs.

Apart from the maintenance work required for CPython (and for other Python
implementations), the feature cadence is likely to be faster on those
projects than is the Python core. That said, I definitely appreciate some
small improvements in the REPL sooner the years, probably ones you have
added, Oleg. So it's a balancing act, I recognize.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Fri, Feb 1, 2019, 6:16 PM Adrien Ricocotam  A thing I thought about but I'm not satisfy is using the new
> matrix-multiplication operator:
>
> my_string_vector @ str.lower
>
> def compute_grad(a_student):
> return "you bad"
> my_student_vector @ compute_grad
>

This is certainly doable. But why would it be better than:

map(str.lower, my_string_vector)
map(compute_grad, my_student_vector)

These latter seem obvious, clear, and familiar.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

[David Mertz]

To be clear. Most of what I don't want is a system where OTHER people are
relying on ratings rather than careful reading. I want to communicate on
changes where posts cannot be "voted up" or edited, etc.

Mind you, I do know this other than this sort of discussion had other
needs. GitHub issues are very useful. Occasionally I'll even go back and
edit a prior comment rather than add another. But there I'm trying to make
the issue genuinely correctly describe the issue at hand.

Stack Overflow or Stack Exchange are very useful for finding technical
answers. Voting up best response there is extremely useful.

I do not want python-ideas to resemble those. It is simply not the
appropriate kind of discussion.

On Fri, Feb 1, 2019, 7:30 PM Abe Dillon  [David Mertz]
>
>> I have absolutely no interest in any system that arranges comments in
>> anything but related thread and chronological order. I DO NOT want any
>> rating or evaluation of comments of any kind other than my own evaluation
>> based on reading them. Well, also in reading the informed opinions of other
>> readers.
>
>
>
> I would find it useless if not actively counterproductive to follow any
>> system where such ratings of comments existed.
>
>
> Then just sort by chronological order.
>
> On Fri, Feb 1, 2019 at 6:19 PM David Mertz  wrote:
>
>> I have absolutely no interest in any system that arranges comments in
>> anything but related thread and chronological order. I DO NOT want any
>> rating or evaluation of comments of any kind other than my own evaluation
>> based on reading them. Well, also in reading the informed opinions of other
>> readers.
>>
>> I would find it useless if not actively counterproductive to follow any
>> system where such ratings of comments existed.
>>
>> There is one property that every system invented to supercede email have
>> in common. They are all dramatically worse in almost every way.
>>
>> On Fri, Feb 1, 2019, 5:09 PM Abe Dillon >
>>>  [Dan Sommers]
>>>
>>>> Another point in favor of email clients over web pages is
>>>> that there are many of them, and *you* control the display
>>>> and other preferences rather than whoever wrote the forum
>>>> or owns the server.
>>>
>>>
>>> There is a tool called the Reddit Enhancement Suite or RES (and probably
>>> others)
>>> That lets you control a great deal of the display and other preferences,
>>> however; I'm not
>>> sure how that control compares to something like Thunderbird.
>>>
>>> One thing that's nice about Reddit is you can link to posts, so if
>>> you've already discussed something at length in another thread,
>>> you can simply refer to that discussion.
>>>
>>>  [Dan Sommers]
>>>
>>>> In an optimal technical discussion, opinions from users
>>>> don't count for anything.  The ideas stand on their own
>>>> merits and research and metrics; users only serve to
>>>> confirm the methodology.
>>>
>>>
>>> A lot can be said about how an ideal world would work. Ideally, we could
>>> define the meaning of life and good and evil and we wouldn't need this
>>> clumsy system of laws and courts to approximate the whole mess.
>>>
>>> I don't think it's that crazy to think that a voting system might
>>> approximate merit a little better than the timestamp on a post.
>>> It's not going to be perfect, but perfect shouldn't be the enemy of
>>> better.
>>>
>>> On Fri, Feb 1, 2019 at 3:17 PM Dan Sommers <
>>> 2qdxy4rzwzuui...@potatochowder.com> wrote:
>>>
>>>> On 2/1/19 2:58 PM, Abe Dillon wrote:
>>>> > [Dan Sommers]
>>>> >
>>>> >> A mailing list is not a feed... Dan, a decades and decades long fan
>>>> of
>>>> >> mailing lists and real email clients.
>>>> >
>>>> >
>>>> > I'm only familiar with Gmail which keeps reply chains coherent and
>>>> moves
>>>> > each chain to the top of my "forums" tab based on who responded last.
>>>> > I haven't explored the various email clients available, can you
>>>> suggest one?
>>>>
>>>> I used mutt for a long time, and then claws-mail, and now
>>>> thunderbird.  They all met my needs, although I did give
>>>> up on claws-mail when I got a hidpi display (claws-mail
>>>> based on gtk2, which doesn't g

Re: [Python-ideas] Clearer communication

[David Mertz]

I have absolutely no interest in any system that arranges comments in
anything but related thread and chronological order. I DO NOT want any
rating or evaluation of comments of any kind other than my own evaluation
based on reading them. Well, also in reading the informed opinions of other
readers.

I would find it useless if not actively counterproductive to follow any
system where such ratings of comments existed.

There is one property that every system invented to supercede email have in
common. They are all dramatically worse in almost every way.

On Fri, Feb 1, 2019, 5:09 PM Abe Dillon   [Dan Sommers]
>
>> Another point in favor of email clients over web pages is
>> that there are many of them, and *you* control the display
>> and other preferences rather than whoever wrote the forum
>> or owns the server.
>
>
> There is a tool called the Reddit Enhancement Suite or RES (and probably
> others)
> That lets you control a great deal of the display and other preferences,
> however; I'm not
> sure how that control compares to something like Thunderbird.
>
> One thing that's nice about Reddit is you can link to posts, so if you've
> already discussed something at length in another thread,
> you can simply refer to that discussion.
>
>  [Dan Sommers]
>
>> In an optimal technical discussion, opinions from users
>> don't count for anything.  The ideas stand on their own
>> merits and research and metrics; users only serve to
>> confirm the methodology.
>
>
> A lot can be said about how an ideal world would work. Ideally, we could
> define the meaning of life and good and evil and we wouldn't need this
> clumsy system of laws and courts to approximate the whole mess.
>
> I don't think it's that crazy to think that a voting system might
> approximate merit a little better than the timestamp on a post.
> It's not going to be perfect, but perfect shouldn't be the enemy of better.
>
> On Fri, Feb 1, 2019 at 3:17 PM Dan Sommers <
> 2qdxy4rzwzuui...@potatochowder.com> wrote:
>
>> On 2/1/19 2:58 PM, Abe Dillon wrote:
>> > [Dan Sommers]
>> >
>> >> A mailing list is not a feed... Dan, a decades and decades long fan of
>> >> mailing lists and real email clients.
>> >
>> >
>> > I'm only familiar with Gmail which keeps reply chains coherent and moves
>> > each chain to the top of my "forums" tab based on who responded last.
>> > I haven't explored the various email clients available, can you suggest
>> one?
>>
>> I used mutt for a long time, and then claws-mail, and now
>> thunderbird.  They all met my needs, although I did give
>> up on claws-mail when I got a hidpi display (claws-mail
>> based on gtk2, which doesn't grok hidpi displays).
>>
>> Another point in favor of email clients over web pages is
>> that there are many of them, and *you* control the display
>> and other preferences rather than whoever wrote the forum
>> or owns the server.
>>
>> > [Dan Sommers]
>> >
>> >> Whoever posted last ends up at the bottom of the thread, so that I can
>> >> read threads from top to bottom in chronological order.  Getting the
>> >> last word in shouldn't earn a spot at the top of the list.
>> >
>> >
>> > That doesn't like any closer an approximation to a merit-based solution
>> to
>> > me.
>> Perhaps not all by itself.  Many/most email clients allow
>> individual users to "score" emails by various criteria, and
>> then to display higher scoring messages "above" the others,
>> or not display certain messages at all.  Personally, I don't
>> use the automated systems, but they're very comprehensive
>> (arguably too complicated), and again, *user* adjustable.
>>
>> In an optimal technical discussion, opinions from users
>> don't count for anything.  The ideas stand on their own
>> merits and research and metrics; users only serve to
>> confirm the methodology.
>>
>> Dan
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

[David Mertz]

If any non-email system is adopted, it will exclude me, and probably many
other contributors to this list.  A mailing list is an appropriate and
useful format. "Discussion systems" are not.

On Fri, Feb 1, 2019 at 1:36 PM Abe Dillon  wrote:

> I've pitched this before but gotten little feedback (especially positive
> feedback), but I think a Reddit-style forum would be a pretty vast
> improvement. We could easily start a python_ideas subreddit to try it out.
>
> I know the google group presents threaded conversations, but I've run into
> enough bugs trying to use that platform that I now only interact with
> python-ideas via my gmail account, and threads are flattened here. Also, a
> Reddit-style forum has voting built in. As a bonus, we can write moderation
> bots and present useful info in the side-bar.
>
> If people find Reddit distasteful or otherwise a bad idea, maybe we can
> find some forum software that replicates the feature set of Reddit and host
> it ourselves?
>
> On Fri, Feb 1, 2019 at 10:41 AM James Lu  wrote:
>
>> A lot of the traffic on this email list is people saying “I don’t
>> understand” or “that’s not what I meant” or trying to re-explain. A lot of
>> “-1”s are really “I don’t see the usefulness of this”.
>>
>>
>> So I want an open discussion on: How can we communicate clearer?
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

[David Mertz]

On Fri, Feb 1, 2019 at 12:43 PM Adrien Ricocotam 
wrote:

> What I think is bad using mailing list it's the absence of votes. I'd
> often like to just hit a "+1" button for some mails just to say to the
> author I'm with him and I think s.he's ideas are great.
>

I feel like the strongest virtue of mailing lists is the absence of votes.

Various chat systems where you can add a "thumbs up" or "smile" or the like
encourage laziness and content-free interaction.  On a mailing list, for
the most part, we encourage people to formulate complete opinions supported
by reasons and arguments.  That is what python-ideas should be.  It should
not be a democracy or a voting system.

That said, obviously sometimes people do just reply with +1 to something.
I do that myself at times.  There are times when that really is an
appropriate reply, but I think it should be discouraged as the default
answer style.

Yours, David...

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

On Thu, Jan 31, 2019 at 12:52 PM Chris Barker via Python-ideas <
python-ideas@python.org> wrote:

> I know that when I'm used to working with numpy and then need to do some
> string processing or some such, I find myself missing this "vectorization"
> -- if I want to do the same operation on a whole bunch of strings, why do I
> need to write a loop or comprehension or map?
>

Isn't what you want called "Pandas"? E.g.:

>>> type(strs)


>>> strs
0 Jan
1 Feb
2 Mar
3 Apr
4 May
5 Jun
6 Jul
7 Aug
8 Sep
9 Oct
10Nov
11Dec

>>> strs.str.upper()
0 JAN
1 FEB
2 MAR
3 APR
4 MAY
5 JUN
6 JUL
7 AUG
8 SEP
9 OCT
10NOV
11DEC

>>> strs.str.upper().str.count('A')
0 1
1 0
2 1
3 1
4 1
5 0
6 0
7 1
8 0
9 0
100
110

>>> strs.str.replace('[aA]','X')
0 JXn
1 Feb
2 MXr
3 Xpr
4 MXy
5 Jun
6 Jul
7 Xug
8 Sep
9 Oct
10Nov
11Dec


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] AMEND PEP-8 TO DISCOURAGE ALL CAPS

[David Mertz]

On Wed, Jan 30, 2019, 4:23 PM Abe Dillon  Consider that math.pi and math.e are constants that are not all caps,
> have you ever been tempted to re-bind those variables?
>

I generally use 'from math import pi as PI' because the lower case is
confusing and misnamed.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

I really don't get the "two different signatures" concern. The two
functions do different things, why would we expect them to automatically
share a signature.

There are a zillion different open() functions or methods in the standard
library, and far more in third party software. They each have various
different signatures and functionality because they "open" different
things. So what? Use the interface to the function you are using, not to
something else that happens to share a name (in a different namespace).

On Wed, Jan 30, 2019, 5:06 AM Jamesie Pic  I'm not disagreeing by any mean. I'm just saying assembling strings is
> a common programing task and that we have two different methods with
> the same name and inconsistent signatures and that it's error-prone.
> I'm most certainly *not* advocating for breaking compatibility or
> whatnot.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

The point really is that something called 'stringify()' could do a lot of
different reasonable and useful things. None of them are universally what
users would want. Unless you have to function scads if optional keyword
arguments, is behavior would surprise many users and not for their purpose.

On Tue, Jan 29, 2019, 10:46 PM David Mertz  Of course not! The request was for something that worked on Python
> *collections*. If the OP wanted something that worked on iterables in
> general, we'd need a different function with different behavior.
>
> Of course, it also doesn't work on dictionaries. I don't really have any
> ideas what the desired behavior might be for dicts. Various things are
> conceivable, none obvious. But it's fine on lists, sets, tuples, deques,
> and some other things that are roughly sequence-like.
>
>
>
> On Tue, Jan 29, 2019, 10:38 PM Robert Vanden Eynde  wrote:
>
>>
>> stringify = lambda it: type(it)(map(str, it))
>>>
>>
>> stringify(range(5)) doesn't work ^^
>>
>> One advantage or having a standard function is that it has been designed
>> by a lot of persons for all possible use cases :)
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

Of course not! The request was for something that worked on Python
*collections*. If the OP wanted something that worked on iterables in
general, we'd need a different function with different behavior.

Of course, it also doesn't work on dictionaries. I don't really have any
ideas what the desired behavior might be for dicts. Various things are
conceivable, none obvious. But it's fine on lists, sets, tuples, deques,
and some other things that are roughly sequence-like.



On Tue, Jan 29, 2019, 10:38 PM Robert Vanden Eynde 
> stringify = lambda it: type(it)(map(str, it))
>>
>
> stringify(range(5)) doesn't work ^^
>
> One advantage or having a standard function is that it has been designed
> by a lot of persons for all possible use cases :)
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

"Not every five line function needs to be in the standard library"

... even more true for every one line function.  I can think of a few dozen
variations of similar but not quite identical behavior to my little
stringify() that "could be useful."  Python gives us easy composition to
create each of them.  It's not PHP, after all.

On Tue, Jan 29, 2019 at 8:52 PM Alex Shafer  wrote:

> That would be strongly preferred to duplication across hundreds of use
> cases and thousands (millions?) of users. Not all of them are likely to
> come up with the most efficient implementation either.
>  Original Message ----
> On Jan 29, 2019, 18:44, David Mertz < me...@gnosis.cx> wrote:
>
>
> stringify = lambda it: type(it)(map(str, it))
>
> Done! Does that really need to be in the STDLIB?
>
> On Tue, Jan 29, 2019, 7:11 PM Alex Shafer via Python-ideas <
> python-ideas@python.org wrote:
>
>> 1) I'm in favor of adding a stringify method to all collections
>>
>> 2) strings are special and worthy of a "special case" because strings
>> tend to be human readable and are used in all kinds of user interface.
>>
>>
>>
>>  Original Message 
>> On Jan 29, 2019, 16:04, Steven D'Aprano < st...@pearwood.info> wrote:
>>
>>
>> On Tue, Jan 29, 2019 at 10:51:26PM +0100, Jamesie Pic wrote:
>>
>> > What do you think of list.stringify(delim) ?
>>
>> What's so special about lists? What do you think of:
>>
>> tuple.stringify
>> deque.stringify
>> iterator.stringify
>> dict.keys.stringify
>>
>> etc. And what's so special about strings that lists have to support a
>> stringify method and not every other type?
>>
>> list.floatify
>> list.intify
>> list.tuplify
>> list.setify
>> list.iteratorify
>>
>> Programming languages should be more about composable, re-usable general
>> purpose components more than special cases.
>>
>> --
>> Steve
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

stringify = lambda it: type(it)(map(str, it))

Done! Does that really need to be in the STDLIB?

On Tue, Jan 29, 2019, 7:11 PM Alex Shafer via Python-ideas <
python-ideas@python.org wrote:

> 1) I'm in favor of adding a stringify method to all collections
>
> 2) strings are special and worthy of a "special case" because strings tend
> to be human readable and are used in all kinds of user interface.
>
>
>
>  Original Message 
> On Jan 29, 2019, 16:04, Steven D'Aprano < st...@pearwood.info> wrote:
>
>
> On Tue, Jan 29, 2019 at 10:51:26PM +0100, Jamesie Pic wrote:
>
> > What do you think of list.stringify(delim) ?
>
> What's so special about lists? What do you think of:
>
> tuple.stringify
> deque.stringify
> iterator.stringify
> dict.keys.stringify
>
> etc. And what's so special about strings that lists have to support a
> stringify method and not every other type?
>
> list.floatify
> list.intify
> list.tuplify
> list.setify
> list.iteratorify
>
> Programming languages should be more about composable, re-usable general
> purpose components more than special cases.
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

On Tue, Jan 29, 2019, 12:22 AM Brendan Barnwell

>  What would you expect to happen with this line:
> >
> > ['foo', b'foo', 37, re.compile('foo')].join('_')
>
> That problem already exists with str.join though.  It's just
> currently spelled this way:
>
> ','.join(['foo', b'foo', 37, re.compile('foo')])
>
> . . . and the result is an error.  I don't see how it's
> semantically
> any less sensible to call list.join on a list of non-string things than
> it is to pass a list of non-string things to str.join.


This feels like an important asymmetry to me. There is a difference between
to object itself being the wrong kind of thing and the arguments to a
method being wrong.

In the first case, the object (a heterogenous list) can NEVER support a
.join() method. It's simply the wrong kind of object. Of course, it's right
as far as the basic type system goes, but its deeper (maybe "structural")
type cannot support that method.

On the other hand, sure, almost any function, including methods, will choke
on bad arguments. But no string *object* rules out joining if good
arguments can be found.

I am sure readers will immediately reply, "what about list.sort()?"
Unfortunately, that really will simply fail on lists of the wrong "type."
After all these years, I still think that change in Python 2.3 or so was
the wrong choice (for those with fewer gray hairs: when the hills were
young, Python objects were arbitrarily comparable under inequality, even
when the answer didn't "mean" anything).

I actually agree that a 'cast_to_string_and_join()' function sounds useful.
Of course, you can write one easily enough, it doesn't need to be a method.
For that matter, I think I'd probably rather that str.join() was simply a
function in the string module or somewhere similar, with a signature like
'join(delim, iter_of_strings)'
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add list.join() please

[David Mertz]

On Mon, Jan 28, 2019 at 8:44 PM Jamesie Pic  wrote:

> ['cancel', name].join('_')
>

This is a frequent suggestion.  It is also one that makes no sense
whatsoever if you think about Python's semantics.  What would you expect to
happen with this line:

['foo', b'foo', 37, re.compile('foo')].join('_')

 List are not restricted to containing only strings (or things that are
string-like enough that they might play well with joining).  Growing a
method that pertains only to that specialized sort of list breaks the
mental model of Python.  Moreover, there is no way to TELL if a particular
list is a "list of strings" other than checking each item inside it (unlike
in many languages).

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] kwargs for return

[David Mertz]

I'm not certain of memory usage. But using 'make_dataclass' makes the
"noise" pretty much no worse than namedtuple.

Person = namedtuple("Person", "name age address")
Person = make_dataclass("Person", "name age address".split())

Unless you have millions of there's objects, memory probably isn't that
important. But I guess you might... and namedtuple did sell itself as "less
memory than small dictionaries"

On Sat, Jan 26, 2019, 1:26 PM Christopher Barker  On Sat, Jan 26, 2019 at 10:13 AM David Mertz  wrote:
>
>> Indeed! I promise to use dataclass next time I find myself about to use
>> namedtuple. :-)
>>
>
> Indeed IIUC, namedtuple was purposely designed to be able to replace
> tuples as well as adding the named access.
>
> But that does indeed cause potential issues. However, dataclasses see kind
> of heavyweight to me -- am I imagining that, or could one make a
> named_not_tuple that was appreciably lighter weight? (in creation time,
> memory use, that sort of thing)
>
> -CHB
>
>
>
>
>
>
>
>>
>> I'm pretty sure that virtually all my uses will allow that.
>>
>> On Sat, Jan 26, 2019, 1:09 PM Eric V. Smith >
>>>
>>>
>>> On 1/26/2019 12:30 PM, David Mertz wrote:
>>> > On Sat, Jan 26, 2019 at 10:31 AM Steven D'Aprano >> > <mailto:st...@pearwood.info>> wrote:
>>> >
>>> > In what way is it worse, given that returning a namedtuple with
>>> named
>>> >
>>> > fields is backwards compatible with returning a regular tuple? We
>>> can
>>> > have our cake and eat it too.
>>> > Unless the caller does a type-check, there is no difference.
>>> Sequence
>>> > unpacking will still work, and namedtuples unlike regular tuples
>>> can
>>> > support optional attributes.
>>> >
>>> >
>>> > I suppose the one difference is where someone improperly relies on
>>> tuple
>>> > unpacking.
>>> >
>>> > Old version:
>>> >
>>> > def myfun():
>>> >  # ...
>>> >  return a, b, c
>>> >
>>> > # Call site
>>> > val1, val2, val3 = myfun()
>>> >
>>> >
>>> > New version:
>>> >
>>> > def myfun():
>>> >  # ...
>>> >  return a, b, c, d
>>> >
>>> >
>>> > Now the call site will get "ValueError: too many values to unpack".
>>> > Namedtuples don't solve this problem, of course.  But they don't make
>>> > anything worse either.
>>> >
>>> > The better approach, of course, is to document the API as only using
>>> > attribute access, not positional.  I reckon dataclasses from the start
>>> > could address that concern... but so can documentation alone.  E.g.:
>>> >
>>> > Old version (improved):
>>> >
>>> > def myfun():
>>> >
>>> >  mydata = namedtuple("mydata", "a b c")
>>> >
>>> >  # ...
>>> >  return mydata(a, b, c)
>>> >
>>> > # Call site
>>> > ret = myfun()
>>> >
>>> > val1, val2, val3 = ret.a, ret.b, ret.c
>>> >
>>> >
>>> > New version (improved)
>>> >
>>> > def myfun():
>>> >
>>> >  mydata = namedtuple("mydata", "a b c d e")
>>> >
>>> >  # ...
>>> >  return mydata(a, b, c, d, e)
>>> >
>>> > Now the call site is completely happy with no changes (assuming it
>>> > doesn't need to care about what values 'ret.d' or 'ret.e' contain...
>>> but
>>> > presumably those extra values are optional in some way.
>>> >
>>> > Moreover, we are even perfectly fine if we had created
>>> > namedtuple("mydata", "e d c b a") for some reason, completely changing
>>> > the positions of all the named attributes in the improved namedtuple.
>>>
>>> Preventing this automatic unpacking (and preventing iteration in
>>> general) was one of the motivating factors for dataclasses:
>>> https://www.python.org/dev/peps/pep-0557/#id47
>>>
>>> Eric
>>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Christopher Barker, PhD
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] kwargs for return

[David Mertz]

On Sat, Jan 26, 2019, 1:21 PM Christopher Barker

> As I understand it, functions return either a single value, or a tuple of
> values -- there is nothing special about how assignment is happening when a
> function is called.
>

No. It's simpler than that! Functions return a single value, period.

That single value might happen to be a tuple or something else unpackable.
This makes it feel like we have multiple return values, but we never
actually do. The fact that "tuples are spelled by commas not by
parentheses" makes this distinction easy to ignore most of the time.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] kwargs for return

[David Mertz]

Indeed! I promise to use dataclass next time I find myself about to use
namedtuple. :-)

I'm pretty sure that virtually all my uses will allow that.

On Sat, Jan 26, 2019, 1:09 PM Eric V. Smith 
>
> On 1/26/2019 12:30 PM, David Mertz wrote:
> > On Sat, Jan 26, 2019 at 10:31 AM Steven D'Aprano  > <mailto:st...@pearwood.info>> wrote:
> >
> > In what way is it worse, given that returning a namedtuple with named
> >
> > fields is backwards compatible with returning a regular tuple? We can
> > have our cake and eat it too.
> > Unless the caller does a type-check, there is no difference. Sequence
> > unpacking will still work, and namedtuples unlike regular tuples can
> > support optional attributes.
> >
> >
> > I suppose the one difference is where someone improperly relies on tuple
> > unpacking.
> >
> > Old version:
> >
> > def myfun():
> >  # ...
> >  return a, b, c
> >
> > # Call site
> > val1, val2, val3 = myfun()
> >
> >
> > New version:
> >
> > def myfun():
> >  # ...
> >  return a, b, c, d
> >
> >
> > Now the call site will get "ValueError: too many values to unpack".
> > Namedtuples don't solve this problem, of course.  But they don't make
> > anything worse either.
> >
> > The better approach, of course, is to document the API as only using
> > attribute access, not positional.  I reckon dataclasses from the start
> > could address that concern... but so can documentation alone.  E.g.:
> >
> > Old version (improved):
> >
> > def myfun():
> >
> >  mydata = namedtuple("mydata", "a b c")
> >
> >  # ...
> >  return mydata(a, b, c)
> >
> > # Call site
> > ret = myfun()
> >
> > val1, val2, val3 = ret.a, ret.b, ret.c
> >
> >
> > New version (improved)
> >
> > def myfun():
> >
> >  mydata = namedtuple("mydata", "a b c d e")
> >
> >  # ...
> >  return mydata(a, b, c, d, e)
> >
> > Now the call site is completely happy with no changes (assuming it
> > doesn't need to care about what values 'ret.d' or 'ret.e' contain... but
> > presumably those extra values are optional in some way.
> >
> > Moreover, we are even perfectly fine if we had created
> > namedtuple("mydata", "e d c b a") for some reason, completely changing
> > the positions of all the named attributes in the improved namedtuple.
>
> Preventing this automatic unpacking (and preventing iteration in
> general) was one of the motivating factors for dataclasses:
> https://www.python.org/dev/peps/pep-0557/#id47
>
> Eric
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] kwargs for return

[David Mertz]

On Sat, Jan 26, 2019 at 10:31 AM Steven D'Aprano 
wrote:

> In what way is it worse, given that returning a namedtuple with named
>
fields is backwards compatible with returning a regular tuple? We can
> have our cake and eat it too.
> Unless the caller does a type-check, there is no difference. Sequence
> unpacking will still work, and namedtuples unlike regular tuples can
> support optional attributes.
>

I suppose the one difference is where someone improperly relies on tuple
unpacking.

Old version:

def myfun():
# ...
return a, b, c

# Call site
val1, val2, val3 = myfun()


New version:


def myfun():
# ...
return a, b, c, d


Now the call site will get "ValueError: too many values to unpack".
Namedtuples don't solve this problem, of course.  But they don't make
anything worse either.

The better approach, of course, is to document the API as only using
attribute access, not positional.  I reckon dataclasses from the start
could address that concern... but so can documentation alone.  E.g.:

Old version (improved):

def myfun():

mydata = namedtuple("mydata", "a b c")

# ...
return mydata(a, b, c)

# Call site
ret = myfun()

val1, val2, val3 = ret.a, ret.b, ret.c


New version (improved)

def myfun():

mydata = namedtuple("mydata", "a b c d e")

# ...
return mydata(a, b, c, d, e)

Now the call site is completely happy with no changes (assuming it doesn't
need to care about what values 'ret.d' or 'ret.e' contain... but presumably
those extra values are optional in some way.

Moreover, we are even perfectly fine if we had created namedtuple("mydata",
"e d c b a") for some reason, completely changing the positions of all the
named attributes in the improved namedtuple.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] kwargs for return

[David Mertz]

I was going to write exactly they're same idea Steven did.

Right now you can simply design APIs to return dictionaries or, maybe
better, namedtuples. Namedtuples are really nice since you can define new
attributes when you upgrade an API without breaking any old coffee that
used the prior attributes... Of course, you can only add more, not remove
old ones, to assure compatibility. Unlike dictionaries, namedtuples cannot
contain arbitrary "keywords" at runtime, which is either good or bad
depending on your purposes.

Recently, dataclasses are also an option. They are cool, but I haven't yet
had a reason to use them. They feel heavier than namedtuples though (as a
programming construct, not talking about memory usage or speed or whatever).

On Sat, Jan 26, 2019, 8:52 AM Steven D'Aprano  On Sat, Jan 26, 2019 at 02:04:12PM +0100, Thomas Güttler Lists wrote:
>
> > Example:
> >
> > status = backend.transmit_data()
> >
> > But later you want to add something to the API.
> [...]
> > How could kwargs for return look like?
>
> return {'status': True, 'messages': []}
>
> Or perhaps better:
>
> return ResultObject(status=True, messages=[])
>
>
> I don't see anything here that can't be done by returning a dict, a
> namedtuple (possibly with optional fields), or some other object with
> named fields. They can be optional, they can have defaults, and you can
> extend the object by adding new fields without breaking backwards
> compatibility.
>
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Potential PEP: with/except

[David Mertz]

You could write a context manager that used an arbitrary callback passed in
to handle exceptions (including re-raising as needed). This doesn't require
new syntax, just writing a custom CM.

On Tue, Jan 22, 2019, 4:20 PM Barry Scott 
>
> On 22 Jan 2019, at 20:31, Michael Selik  wrote:
>
> On Tue, Jan 22, 2019, 12:11 PM Paul Ferrell 
>> I see this as the natural evolution of what 'with' is all about -
>> replacing necessary try-finally blocks with something more elegant. We just
>> didn't include the 'except' portion.
>>
>
> The time machine strikes again. In fact, you can handle exceptions with a
> context manager object. Whatever you're with-ing must have a dunder exit
> method, which received any exceptions raised in the block as an argument.
> Return true and the exception is suppressed.
>
>
> Suppressing the exception is not the general case.
> And will not work for the example given.
>
> Barry
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/