Re: [Python-ideas] Adding Python interpreter info to "pip install"

[Pradyun Gedam]

On Fri, 20 Jul 2018, 09:52 Nathaniel Smith,  wrote:

> On Thu, Jul 19, 2018 at 5:45 PM, Al Sweigart  wrote:
> > The goal of this idea is to make it easier to find out when someone has
> > installed packages for the wrong python installation. I'm coming across
> > quite a few StackOverflow posts and emails where beginners are using pip
> to
> > install a package, but then finding they can't import it because they
> have
> > multiple python installations and used the wrong pip.
>
> This sounds like a great idea to me, but pip is developer separately
> from python itself, and I don't think the pip maintainers monitor
> python-ideas.


At least one (me) does lurk here. ;)

I'd suggest filing a feature request on the pip tracker:
>
> https://github.com/pypa/pip/issues/new?template=feature-request.md


I second this suggestion to file a feature request for this on the tracker.

Additionally, you should also take a look to see if this has been filed
already since I do remember having a discussion where this came. :)

--
Pradyun


> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Steven D'Aprano]

On Thu, Jul 19, 2018 at 05:32:48PM +0900, Stephen J. Turnbull wrote:

> To be honest, code transformations like this
> 
>  >  class BaseUploadObject(object):
>  >  def find_content_type(self, filename):
>  >  ctype, encoding = mimetypes.guess_type(filename)
>  >  if ctype is None:
>  >  return 'application/octet-stream'
>  >  else:
>  >  return ctype
> 
> to this
> 
>  >  class BaseUploadObject(object):
>  >  def find_content_type(self, filename):
>  >  ctype, encoding = mimetypes.guess_type(filename)
>  >  return ctype ?? 'application/octet-stream'
> 
> make me cringe. 

I don't think that's a good use of ?? and that method should 
either remain in the original form, or at most, be re-written using

return ctype if ctypes is not None else 'application/octet-stream'

(although I prefer the original form). There doesn't seem to be any 
advantage to ?? in this example except to transform a four-line 
imperative-style return statement to a one-line return statement, 
trading off vertical space for horizontal. But there's no improvement in 
expressiveness that I can see.


> Exactly one of two things is true:
> 
> 1.  mimetypes.guess_type guarantees that the only falsie it will ever
> return is None, or
> 
> 2.  it doesn't.
> 
> In case 1, "ctype or 'application/octet-stream'" ALSO does the right
> thing. 

It does the right thing, but it doesn't look like it does the right 
thing.

The ?? operator promises to only transform None, and nothing but None. 
To the reader, the `or` operator promises to transform any falsey value. 
Is `or` too broad and do too much? There's no way of knowing.

Instead of treating `guess_type` as a black-box, the reader now has to 
dig-deep into its documentation and/or implementation to find out which 
falsey values it might return and whether or not they ought to be 
treated the same as None.


> In case 2, ONLY "ctype or 'application/octet-stream'" does the
> right thing, as few callers of BaseUploadObject.find_content_type will
> be prepared for "", (), [], or any variety of 0 as the return value.

I doubt that guess_type (and hence find_content_type) will return 0 or 
[] etc as part of their documented interface (as opposed to a bug), but 
if they did, surely we ought to assume that the caller is prepared to 
handle its output.

More likely is that it might return the empty string "". At face value I 
would expect that a better interface for guess_type would be to return 
the empty string rather than None.

But perhaps there is a legitimate reason to treat the empty string as 
distinct from None, it should *not* be transformed.

On perhaps the function is avoiding the "failure looks like success" 
anti-pattern. Who hasn't been bitten by writing something like this?

chunk = text[text.find('spam')+4:]

It looks okay at a glance: return the chunk of text immediately 
following "spam". But if "spam" isn't present at all, it returns the 
entire text minus the first three characters, which is probably not what 
you wanted.

So there is a good argument to be made that guess_type ought to promise:

- return None if it cannot guess the type;
- otherwise return a non-empty string.

(or that the empty string is itself a legitimate output that ought to be 
considered distinct from None and should *not* be transformed).

In that case, `or` reads wrongly, because it either wrongly transforms 
"" into the default when it should not, or it suggests to the reader 
that it might.

Either way, an explicit test for None -- whether we use "if ... is None" 
or the proposed ?? operator -- is better than a test for falsey.



> So I would like to see various examples of code where
> 
> 1.  in the original code, treating a falsie that is not None the same
> way that None is treated is a detectable bug; and

I don't see that this is relevant. We already had this discussion, when 
the ternary if was introduced, and a million times when discussing why 
it is better to write "if spam is None" rather than "if not spam".

In other words, we ought to be comparing the expressiveness of 

process(spam ?? something)

versus:

   process(something if spam is None else spam)

(and similar variations), not against 

   process(spam if spam else something)
   process(spam or something)

both of which do something different from the ?? operator.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding Python interpreter info to "pip install"

[Wes Turner]

`python -m site` and its docs may also be useful for troubleshooting
end-user installations.

A diagnostic script with e.g. these commands could also be helpful:

which python
which pip
python -m site  # sys.path, USER_SITE (pip --user)
python -m pip --version

>>> print((os.name, sys.platform, platform.system()))


https://docs.python.org/3/library/site.html

"""
The site module also provides a way to get the user directories from the
command line:

$ python3 -m site --user-site
/home/user/.local/lib/python3.3/site-packages

If it is called without arguments, it will print the contents of sys.path
on the standard output, followed by the value of USER_BASE and whether the
directory exists, then the same thing for USER_SITE, and finally the value
of ENABLE_USER_SITE.

--user-base

Print the path to the user base directory.

--user-site

Print the path to the user site-packages directory.

If both options are given, user base and user site will be printed (always
in this order), separated by os.pathsep.

If any option is given, the script will exit with one of these values: O if
the user site-packages directory is enabled, 1 if it was disabled by the
user, 2 if it is disabled for security reasons or by an administrator, and
a value greater than 2 if there is an error.
"""


On Friday, July 20, 2018, Nathaniel Smith  wrote:

> On Thu, Jul 19, 2018 at 5:45 PM, Al Sweigart  wrote:
> > The goal of this idea is to make it easier to find out when someone has
> > installed packages for the wrong python installation. I'm coming across
> > quite a few StackOverflow posts and emails where beginners are using pip
> to
> > install a package, but then finding they can't import it because they
> have
> > multiple python installations and used the wrong pip.
>
> This sounds like a great idea to me, but pip is developer separately
> from python itself, and I don't think the pip maintainers monitor
> python-ideas. I'd suggest filing a feature request on the pip tracker:
>
> https://github.com/pypa/pip/issues/new?template=feature-request.md
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding Python interpreter info to "pip install"

[Nathaniel Smith]

On Thu, Jul 19, 2018 at 5:45 PM, Al Sweigart  wrote:
> The goal of this idea is to make it easier to find out when someone has
> installed packages for the wrong python installation. I'm coming across
> quite a few StackOverflow posts and emails where beginners are using pip to
> install a package, but then finding they can't import it because they have
> multiple python installations and used the wrong pip.

This sounds like a great idea to me, but pip is developer separately
from python itself, and I don't think the pip maintainers monitor
python-ideas. I'd suggest filing a feature request on the pip tracker:

https://github.com/pypa/pip/issues/new?template=feature-request.md

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding Python interpreter info to "pip install"

[Abdur-Rahmaan Janhangeer]

as it requires only a log, i think it is worth it as pip acts covertly
meaning you don't see which pip is being used in a multi-py environment,
yes you can have version pip info

Abdur-Rahmaan Janhangeer
https://github.com/Abdur-rahmaanJ
Mauritius


>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephen J. Turnbull]

Terry Reedy writes:
 > On 7/19/2018 4:32 AM, Stephen J. Turnbull wrote:

 > > make me cringe.  Exactly one of two things is true:
 > 
 > It seems to me that the problem is returning None.

Exactly, but this seems to indicate that the problem I'm talking about
is the code.  It's not; my issue is with the use of THIS code as an
example justifying "??".

 > The premise of returning None as default is that users can follow
 > with conditional code.  If we now think that this is too much of a
 > burden, we should stop returning None, at least by default.

Note that in the case of an MUA, code might look like something like
this:

class BaseSaveObject(object):# NOTE NAME CHANGE!
def find_content_type(self, filename):
ctype, encoding = mimetypes.guess_type(filename)
while ctype is None:
ctype = input('Couldn't recognize MIME type. Ideas?")
ctype = self.validate_mimetype(ctype)  # returns None on bad
return ctype

So returning "application/octet-stream" is probably inappropriate if
parsing the message header for Content-Type fails.

I'm not sure why you say "burdensome".  Using "x() or y()" *because*
it short-circuits is idiomatic (although some deprecate it).  The main
point of "??" is that "or" is not good enough because it does the same
thing with other falsies that it does with None.  There are other
tweaks (such as very high precedence), but that's the main thing.

What I'm looking for is a set of examples where "or" won't do, and
explanations of *why* that is true and why programmers *won't* abuse
"??" to create less resilient code, to balance against the avalanche
of examples where "??" leads to less resilient code, such as this
case.

Note that if you want to use "??" rather than "or", there's at least
one other falsie that you want to *avoid* a default for, and then in
cases like this one, the question is "why aren't you checking for it
to prevent exceptions in a module written long long ago in a galaxy
far far away?"

 > > 1.  mimetypes.guess_type guarantees that the only falsie it will ever
 > >  return is None, or
 > > 
 > > 2.  it doesn't.
 > 
 > or
 > 3. it never returns anything other than a non-blank string unless the 
 > user so requests.

That would be nice, but that's off-topic.  The point here is that
Steve ripped a page from Tim's book and transformed a pile of real
code.  (This deserves a standing ovation IMO!)  So he has to take the
API as it is, warts and all.

Steve

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephen J. Turnbull]

Steve Dower writes:

 > But for me, the ?? operator makes its strongest case when you are 
 > getting attributes from potentially None values:
 > 
 >  x = maybe_get()?.get_int() ?? 0
 >  y = maybe_get()?.get_list() ?? []
 > 
 > In this case, using 'or' may replace an intentionally falsie value
 > with your own,

I wish the PEP's examples had such cases marked and explained.  I
didn't understand the context for most examples (ie, *why* the left
operand could be None, and why that needed to be distinguished from
"intentionally falsie values").

In the "find_content_type" example, None is just a way of saying
"oops, you handle it" that is less "cute" than "".  "or" is far more
resilient in that context than "??" would be.  I am a mail geek, and
*much* more likely to encounter that code than the others.  I would be
distressed if the email module or Mailman grew a lot of "??".  Email
is a domain where resilience really matters because you can receive
arbitrary data over SMTP, there's no provision for rejecting garbage,
and all too often you do get garbage. :-/

 > while using a default parameter is still going to leave you 
 > with None if the first maybe_get() returned nothing.
 > 
 > "?." without "??" feels like a trap, while "??" without "?." feels 
 > largely unnecessary. But both together lets you turn many lines of code 
 > into a much shorter snippet that reads left-to-right and lets you assume 
 > success until you reach the "??".

I find that pretty persuasive.  Perhaps that should be added to PEP 8.

I would advocate write those 

x = maybe_get() ?. get_int() ?? 0
y = maybe_get() ?. get_list() ?? []

to emphasize that this is an operator that "does something" to the
operand, rather than "just" extracting an attribute.  (No, I'm not
going to try to make that more precise, so feel free to ignore.)  I'm
not sure about the ?[ operator:

x = a?[b] ?? c
x = a ?[b] ?? c
x = a ?[ b] ?? c
x = a ?[ b ] ?? c

I guess I like the second line best (and

x = a?[b]??c

is right out, of course).

I do think Terry Reedy's criticism that in many cases having
maybe_get() return None rather than an appropriate default object is
bad API design may be important.  I suspect that the "a ?. b ?? c" 
pattern will discourage refactoring of that code in favor of writing
short workarounds.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding Python interpreter info to "pip install"

[Al Sweigart]

 Sorry, I meant "pip list", rather than "pip info".

I thought about the fact that "pip --version" provides this info, but 1) it
provides the location of pip, not the python interpreter it installs
packages for and 2) it would be an additional step for the question-asker
to go through after posting the output of "pip install".

It would be nice to display output that the question-asker can compare
directly with the output of "which python". And I'd like to shorten the
potential amount of back-and-forth before the helper can get the
information they need.

Additionally, while they could always just run "python -m pip install spam"
but most tutorials tell them to run pip directly, so I still see the need
for this.


On Thu, Jul 19, 2018 at 5:50 PM, Chris Angelico  wrote:

> On Fri, Jul 20, 2018 at 10:45 AM, Al Sweigart  wrote:
> > The goal of this idea is to make it easier to find out when someone has
> > installed packages for the wrong python installation. I'm coming across
> > quite a few StackOverflow posts and emails where beginners are using pip
> to
> > install a package, but then finding they can't import it because they
> have
> > multiple python installations and used the wrong pip.
> >
> > For example, this guy has this problem:
> > https://stackoverflow.com/questions/37662012/which-pip-is-
> with-which-python
> >
> > I'd propose adding a simple line to the output of "pip install" that
> changes
> > this:
> >
> > user@user:~$ pip3 install pyperclip
> > Collecting pyperclip
> > Installing collected packages: pyperclip
> > Successfully installed pyperclip-1.6.2
> >
> > ...to something like this:
> >
> > user@user:~$ pip3 install pyperclip
> > Running pip for /usr/bin/python3
> > Collecting pyperclip
> > Installing collected packages: pyperclip
> > Successfully installed pyperclip-1.6.2
> >
> > This way, when they copy/paste their output to StackOverflow, it'll be
> > somewhat more obvious to their helper that they used pip for the wrong
> > python installation.
> >
> > This info would also be useful for the output of "pip info", but that
> would
> > break scripts that reads that output.
> >
> > Any thoughts?
>
> You can get some very useful information from "pip3 --version". As
> well as pip's own version, it tells you the version of Python that
> it's running under, AND what directory it's being run from. If you
> want to request that similar info be added to other commands, I would
> strongly recommend lifting the exact format of --version and using
> that.
>
> (I'm not sure what "pip info" is, btw. My pip doesn't seem to have that.)
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[David Mertz]

On Thu, Jul 19, 2018, 9:39 AM Steven D'Aprano  wrote:

> *Its just spelling*. If it is a useful and well-defined feature, we'll
> get used to the spelling soon enough.
>
> That's not to say that spelling is not important *at all*, or that we
> should never prefer words to symbols. But if the only objection we have is
> "this is useful but I don't like the spelling so -1" then that's
> usually a pretty weak argument against the feature.
>

I could not disagree more. Spelling is extremely important to readability,
and that only goes away partially with familiarity. As a teaching language,
moreover, there will always be a late share of Python users who haven't
become familiar, and for whom "executable pseudo-code" is a big advantage.

I don't believe Python should be ONLY for those who do not already know it,
but neither should it be even half so arcane as this PEP would make it.

Here's are well defined features of the J programming language, for
comparison... very compact too:

For example, in the following contrived expression the exclamation point
 !refers to three
distinct functions:

2!!7!4

The function below can be used to list all of the prime numbers between 1
and R with:

2_&{&/x!/:2_!x}'!R
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Michael Selik]

On Thu, Jul 19, 2018 at 8:40 PM Michael Lee 
wrote:

>
>1. ...
>
>2. We can already easily get the same functionality using standard
>Python. E.g., instead of doing foo?["bar"]?[0]?["baz"], we could do 
> lookup(foo,
>"bar", 0, "baz") where lookup is a function that looks roughly like
>this:
>
>def lookup(item, *parts):
>for part in parts:
>if item is None:
>return None
>item = item[parts]
>return item
>
>
Try/except also looks decent.

try:
x = foo['bar'][0]
except TypeError:
x = 'default'
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Greg Ewing]

David Mertz wrote:
"APL and Perl territory" means "use lots of punctuation characters in 
somewhat cryptic ways, often combining several for a distinct 
semantics."


And APL takes it a step further by freely making up new
characters when it runs out of existing ones. At least
nobody has suggested doing that in Python yet...

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding Python interpreter info to "pip install"

[Chris Angelico]

On Fri, Jul 20, 2018 at 10:45 AM, Al Sweigart  wrote:
> The goal of this idea is to make it easier to find out when someone has
> installed packages for the wrong python installation. I'm coming across
> quite a few StackOverflow posts and emails where beginners are using pip to
> install a package, but then finding they can't import it because they have
> multiple python installations and used the wrong pip.
>
> For example, this guy has this problem:
> https://stackoverflow.com/questions/37662012/which-pip-is-with-which-python
>
> I'd propose adding a simple line to the output of "pip install" that changes
> this:
>
> user@user:~$ pip3 install pyperclip
> Collecting pyperclip
> Installing collected packages: pyperclip
> Successfully installed pyperclip-1.6.2
>
> ...to something like this:
>
> user@user:~$ pip3 install pyperclip
> Running pip for /usr/bin/python3
> Collecting pyperclip
> Installing collected packages: pyperclip
> Successfully installed pyperclip-1.6.2
>
> This way, when they copy/paste their output to StackOverflow, it'll be
> somewhat more obvious to their helper that they used pip for the wrong
> python installation.
>
> This info would also be useful for the output of "pip info", but that would
> break scripts that reads that output.
>
> Any thoughts?

You can get some very useful information from "pip3 --version". As
well as pip's own version, it tells you the version of Python that
it's running under, AND what directory it's being run from. If you
want to request that similar info be added to other commands, I would
strongly recommend lifting the exact format of --version and using
that.

(I'm not sure what "pip info" is, btw. My pip doesn't seem to have that.)

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Adding Python interpreter info to "pip install"

[Al Sweigart]

 The goal of this idea is to make it easier to find out when someone has
installed packages for the wrong python installation. I'm coming across
quite a few StackOverflow posts and emails where beginners are using pip to
install a package, but then finding they can't import it because they have
multiple python installations and used the wrong pip.

For example, this guy has this problem:
https://stackoverflow.com/questions/37662012/which-pip-is-with-which-python

I'd propose adding a simple line to the output of "pip install" that
changes this:

user@user:~$ pip3 install pyperclip
Collecting pyperclip
Installing collected packages: pyperclip
Successfully installed pyperclip-1.6.2

...to something like this:

user@user:~$ pip3 install pyperclip
Running pip for /usr/bin/python3
Collecting pyperclip
Installing collected packages: pyperclip
Successfully installed pyperclip-1.6.2

This way, when they copy/paste their output to StackOverflow, it'll be
somewhat more obvious to their helper that they used pip for the wrong
python installation.

This info would also be useful for the output of "pip info", but that would
break scripts that reads that output.

Any thoughts?

-Al
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Greg Ewing]

Calvin Spealman wrote:

Operators that only vary by case would be... confusing.


Also I'd prefer to see AND and OR (or maybe And and Or) reserved for
possible use as overridable, non-short-circuiting forms of "and"
and "or" for use by DSLs.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Greg Ewing]

Nathan wrote:


How many programmer will read "||" or "OR" as a classic "or" operator?


It will certainly confuse someone who has any C reflexes, since
"||" in C is just a boolean "or".

--
Greg

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Todd]

On Thu, Jul 19, 2018, 08:47 Rhodri James  wrote:

> On 19/07/18 09:33, Antoine Pitrou wrote:
> > There is a use case I sympathize with: the argument-is-None case.  For
> > that I would suggest a simpler form:  "A else B" which would evaluate
> > to A if A is not None, otherwise to B (parentheses may be mandatory).
> >
> > So e.g. one of the examples would read:
> >
> >   def insort_right(a, x, lo=0, hi=None):
> >   # ...
> >   hi = hi else len(a)
> >   # ...
>
> Much as I would like a keyword, "else" is the wrong one.  It implies we
> are dealing with truthiness, which we aren't, and lays a subtle semantic
> trap as a consequence.
>
> If anyone can think of a good word for "if it isn't None, otherwise",
> I'd be all for it :-)
>

What about "nelse"?

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Greg Ewing]

Rhodri James wrote:

On 19/07/18 07:06, Greg Ewing wrote:


There's no such tradition for the new
operators being proposed.


There is, actually, it's just not a long one.  C# has had null-aware 
operators for a while, for example.


THere's a precedent, yes, but I wouldn't call it a tradition.
A substantial period of time is part of the definition of the
word.

Wikipedia:

"A tradition is a belief or behavior passed down within a group
or society with symbolic meaning or special significance with
origins in the past."

Merriam-Webster:

"the handing down of information, beliefs, or customs from one
generation to another."

I don't think C# has been around long enought to span multiple
generations of programmers.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Chris Angelico]

On Fri, Jul 20, 2018 at 10:03 AM, Greg Ewing
 wrote:
> Rhodri James wrote:
>>
>> If anyone can think of a good word for "if it isn't None, otherwise", I'd
>> be all for it :-)
>
>
> I don't think there's any single Engish word that captures
> all of that, so we'd have to invent one.
>
> Some suggestions:
>
> inno (If Not None, Otherwise)
>
> oft  (Or, Failing That)

Iunno, that'd oft be confusing.

Kappa

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Greg Ewing]

Rhodri James wrote:
If anyone can think of a good word for "if it isn't None, otherwise", 
I'd be all for it :-)


I don't think there's any single Engish word that captures
all of that, so we'd have to invent one.

Some suggestions:

inno (If Not None, Otherwise)

oft  (Or, Failing That)

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Greg Ewing]

Jonathan Fine wrote:


Perhaps "argue" is not the right word here. It sounds too much for or
against. And it has implications of being heated and angry.


At least arguing about the nature of argument is a very
Pythonic thing to do.

"That's not arguing, it's just contradiction!"
"No, it isn't!"

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Michael Selik]

On Thu, Jul 19, 2018 at 5:29 PM Steve Dower  wrote:

> * "It makes it more complex"
> * "It's harder to follow the flow"
>
> Depends on your measure of complexity. For me, I prioritise "area under
> the indentation" as my preferred complexity metric (more lines*indents
> == more complex), as well as left-to-right reading of each line (more
> random access == more complex). By these measures, ?. significantly
> reduces the complexity over any of the current or future alternatives::
>
> def f(a=None):
>  name = 'default'
>  if a is not None:
>  user = a.get_user()
>  if user is not None:
>  name = user.name
>  print(name)
>


You left one out, I think, that looks decent. I converted the print to a
return, because I think it's more common to return than print.

def f(a=None):
try:
return a.get_user().name
except AttributeError:
return 'default'


def f(a=None):
>  print(a?.get_user()?.name ?? 'none')
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Michael Selik]

On Thu, Jul 19, 2018 at 2:19 PM Terry Reedy  wrote:

> It seems to me that the problem is returning None.


If functions raised errors instead of returning None, we wouldn't have so
much trouble. Much of the benefit of PEP-572 was handling Nones and similar
sentinel values.

Instead of providing better tools for handling Nones, I'd rather encourage
folks to raise exceptions with a nice tool like PEP 463, exception-catching
expressions. Re-reading Guido's explanation for rejecting PEP 463, I now
understand the acceptance of PEP 572 better, as he also disagrees with the
preference for EAFP over LBYL.

For what it's worth, I agree with the folks that think ?. and ?[ create too
much visual clutter and are too easy to overlook or misunderstand. The ??=
and ?? operators will have spaces around them, so they're easier to read.
However, their benefit is so minor, if any, that it's better to stick with
the status quo.

I'd rather write one of these patterns:

# when x was assigned elsewhere
if x is None:
x = value

# when x is typically None
value if x is None else x

# when x is typically not None
x if x is not None else value

# when I'm playing code golf
x or value


In many of these cases I'd rather write 4 lines of code. It reads how I'd
speak the logic to another person, though out-loud I'd probably add a
"then" and say "otherwise" instead of "else":

if x is None:
y = a
else:
y = b
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Steven D'Aprano]

On Thu, Jul 19, 2018 at 11:39:50AM -0700, Brendan Barnwell wrote:
> On 2018-07-19 06:38, Steven D'Aprano wrote:
> >*Its just spelling*. If it is a useful and well-defined feature, we'll
> >get used to the spelling soon enough.
> >
> >That's not to say that spelling is not important *at all*, or that we
> >should never prefer words to symbols. But if the only objection we have
> >is "this is useful but I don't like the spelling so -1" then that's
> >usually a pretty weak argument against the feature.
> 
>   But we already have a spelling for the most common case.  It is:
> 
> x = a if a is not None else b
> 
>   That is the only use case of any of these operators that is actually 
> common enough for me to care about --- but it's still not common enough 
> to warrant the creation of a new operator, let alone multiple new operators.


That's a reasonable argument: "there's no need for this operator 
because...", albeit it is a subjective argument. (There's no objective 
rule about how common an operation should be before allowing it to be an 
operator.)

What's not a reasonable argument is "I see that there could be a need 
for this operator, but I don't like the spelling so -1 on the entire 
proposal", which was my point. (One of my points.)


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Steve Dower]

Thanks everyone for the feedback and discussion so far. I want to 
address some of the themes, so apologies for not quoting individuals and 
for doing this in one post instead of twenty.


--


* "It looks like line noise"

Thanks for the feedback. There's nothing constructive for me to take 
from this.


* "I've never needed this"

Also not very actionable, but as background I'll say that this was 
exactly my argument against adding them to C#. But my coding style has 
adapted to suit (for example, I'm more likely to use "null" as a default 
value and have a single function implementation than two 
mostly-duplicated overloads).


* "It makes it more complex"
* "It's harder to follow the flow"

Depends on your measure of complexity. For me, I prioritise "area under 
the indentation" as my preferred complexity metric (more lines*indents 
== more complex), as well as left-to-right reading of each line (more 
random access == more complex). By these measures, ?. significantly 
reduces the complexity over any of the current or future alternatives::


def f(a=None):
name = 'default'
if a is not None:
user = a.get_user()
if user is not None:
name = user.name
print(name)

def f(a=None):
if a is not None:
user = a.get_user()
name = user.name if user is not None else 'default'
print(name)
else
print('default')

def f(a=None):
user = a.get_user() if a is not None else None
name = user.name if user is not None else 'default'
print(name)

def f(a=None):
print(user.name
  if (user := a.get_user() if a is not None else None) is not None
  else 'default')

def f(a=None):
print(a?.get_user()?.name ?? 'none')

* "We have 'or', we don't need '??'"

Nearly-agreed, but I think the tighter binding on ?? makes it more 
valuable and tighter test make it valuable in place of 'or'.


For example, compare:

a ** b() or 2 # actual:   (a ** b()) or 2
a ** b() ?? 2 # proposed:  a ** (b() ?? 2)

In the first, the presence of 'or' implies that either b() or __pow__(a, 
b()) could return a non-True value. This is correct (it could return 0 
if a == 0). And the current precedence results in the result of __pow__ 
being used for the check.


In the second one, the presence of the '??' implies that either b() or 
__pow__(a, b()) could return None. The latter should never happen, and 
so the choices are to make the built-in types propagate Nones when 
passed None (uhh... no) or to make '??' bind to the closer part of the 
expression.


(If you don't think it's likely enough that a function could return 
[float, None], then assume 'a ** b?.c ?? 2' instead.)


* "We could have '||', we don't need '??'"

Perhaps, though this is basically just choosing the bikeshed colour. In 
the absence of a stronger argument, matching existing languages 
equivalent operators instead of operators that do different things in 
those languages should win.


* "We could have 'else', we don't need '??'"

This is the "a else 'default'" rather than "a ?? 'default'" proposal, 
which I do like the look of, but I think it will simultaneously mess 
with operator precedence and also force me to search for the 'if' that 
we actually need to be comparing "(a else 'default')" vs. "a ?? 'default'"::


x = a if b else c else d
x = a if (b else c) else d
x = a if b else (c else d)

* "It's not clear whether it's 'is not None' or 'hasattr' checks"

I'm totally sympathetic to this. Ultimately, like everything else, this 
is a concept that has to be taught/learned rather than known intrinsically.


The main reasons for not having 'a?.b' be directly equivalent to 
getattr(a, 'b', ???) is that you lose the easy ability to find typos, 
and we also already have the getattr() approach.


(Aside: in this context, why should the result be 'None' if an attribute 
is missing? For None, the None value propagates (getattr(a, 'b', a)), 
while for falsies you could argue the same thing applies. But for a 
silently handled AttributeError? You still have to make the case that 
None is special here, just special as a return value vs. special as a test.)


* "The semantics of this example changed from getattr() with ?."

Yes, this was a poor example. On re-reading, all of the checks are 
indeed looking for optional attributes, rather than looking them up on 
optional targets. I'll find a better one (I've certainly seen and/or 
written code like this that was intended to avoid crashing on None, but 
I stopped my search of the stdlib too soon after finding this example).


* "Bitwise operators"

Uh... yeah. Have fun over there :)

* "Assumes the only falsie ever returned [in some context] is None"

I argue that it assumes the only falsie you want to replace with a 
different value is None. In many cases, I'd expect the None to be 
replaced with a falsie of the intended type:


x = maybe_get_int() ?? 0
y = maybe_get_list() ?? []

Particularly for the 

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[MRAB]

On 2018-07-19 20:23, Terry Reedy wrote:

On 7/19/2018 8:30 AM, Jonathan Fine wrote:

Hi


There is a formatted version of this PEP at
https://www.python.org/dev/peps/pep-0505/


I've taken a look at this, and have some comments on the first two
examples drawn from standard library code. (And a very grateful +10
for writing a script to find such examples.)

I've started a subthread, just to discuss the ?= and ?? operators. And
something newish, that I call OR.

FIRST EXAMPLE
The first example is
---
 From bisect.py:
def insort_right(a, x, lo=0, hi=None):
 # ...
 if hi is None:
 hi = len(a)
---

Here, None is a sentinel value. The simpler code
---
   hi = hi or len(a)
---
fails when hi is zero (or any other value that is False in the boolean context).

This can be fixed by introducing a new operator OR which is similar to
'or' but has the semantics this example requires. Thus, given OR we
can write
---
   hi = hi OR len(a)


Antoine Pitrou proposed the same thing, using the existing 'else' as the
spelling.  'hi else len(s)' abbreviates the existing
'hi if hi is not None else len(s)'.
with the boilerplate 'if x is not None' removed.


---
where (A OR B) returns A if A is not None, otherwise it returns B.

(Recall that (A or B) returns A if bool(A), otherwise it returns B.)


To make the parallel more obvious, (A or B) also abbreviates
tem = A; tem if bool(tem) is not False else B


If you want a not-too-long keyword, how about "anders"? :-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Terry Reedy]

On 7/19/2018 8:30 AM, Jonathan Fine wrote:

Hi


There is a formatted version of this PEP at
https://www.python.org/dev/peps/pep-0505/


I've taken a look at this, and have some comments on the first two
examples drawn from standard library code. (And a very grateful +10
for writing a script to find such examples.)

I've started a subthread, just to discuss the ?= and ?? operators. And
something newish, that I call OR.

FIRST EXAMPLE
The first example is
---
 From bisect.py:
def insort_right(a, x, lo=0, hi=None):
 # ...
 if hi is None:
 hi = len(a)
---

Here, None is a sentinel value. The simpler code
---
   hi = hi or len(a)
---
fails when hi is zero (or any other value that is False in the boolean context).

This can be fixed by introducing a new operator OR which is similar to
'or' but has the semantics this example requires. Thus, given OR we
can write
---
   hi = hi OR len(a)


Antoine Pitrou proposed the same thing, using the existing 'else' as the 
spelling.  'hi else len(s)' abbreviates the existing

'hi if hi is not None else len(s)'.
with the boilerplate 'if x is not None' removed.


---
where (A OR B) returns A if A is not None, otherwise it returns B.

(Recall that (A or B) returns A if bool(A), otherwise it returns B.)


To make the parallel more obvious, (A or B) also abbreviates
tem = A; tem if bool(tem) is not False else B

--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Chris Angelico]

On Fri, Jul 20, 2018 at 4:45 AM, Stephan Houben  wrote:
> Let me just address this point:
>
> 2018-07-19 20:36 GMT+02:00 Brendan Barnwell :
>>
>> As far as I can see, these null-coalescing operators would break
>> that model.  The PEP doesn't seem to provide for a "real" magic method
>> allowing users to override the actual behavior of the method.  (You can only
>> override __has_value__ to hook into it, but not define by fiat what A ?? B
>> does, as you can with other operators.)  And I think the reason for this is
>> that the operator itself is too specific, much more specific in semantics
>> than other operators.  (I had similar doubts about adding the
>> matrix-multiplication operator @.)
>
>
> I think the actual reason is that it is a short-cutting operator, and none
> of
> the shortcutting operators (and, or,  if/else) have an associated method.
> They cannot have, since they induce a non-standard evaluation order,
> hence their effect cannot be emulated with a method invocation.

Also for the same reason that the 'is' operator doesn't have a
corresponding dunder. You can't ask an object if it's the same object
as another; it either is or is not, intrinsically. It's the same here;
it either is None, or is not None, intrinsically. These new operators
have very clearly defined semantics involving the special object None,
and they short-circuit; two good reasons NOT to have them overridable.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephan Houben]

Let me just address this point:

2018-07-19 20:36 GMT+02:00 Brendan Barnwell :

> As far as I can see, these null-coalescing operators would break
> that model.  The PEP doesn't seem to provide for a "real" magic method
> allowing users to override the actual behavior of the method.  (You can
> only override __has_value__ to hook into it, but not define by fiat what A
> ?? B does, as you can with other operators.)  And I think the reason for
> this is that the operator itself is too specific, much more specific in
> semantics than other operators.  (I had similar doubts about adding the
> matrix-multiplication operator @.)
>

I think the actual reason is that it is a short-cutting operator, and none
of
the shortcutting operators (and, or,  if/else) have an associated method.
They cannot have, since they induce a non-standard evaluation order,
hence their effect cannot be emulated with a method invocation.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Elazar]

On Thu, Jul 19, 2018 at 11:37 AM Brendan Barnwell 
wrote:

>
> As far as I can see, these null-coalescing operators would break
> that
> model.  The PEP doesn't seem to provide for a "real" magic method
> allowing users to override the actual behavior of the method.  (You can
> only override __has_value__ to hook into it, but not define by fiat what
> A ?? B does, as you can with other operators.)  And I think the reason
> for this is that the operator itself is too specific, much more specific
> in semantics than other operators.  (I had similar doubts about adding
> the matrix-multiplication operator @.)
>

It is just as specific as the `is` operator, and for the same reason.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Terry Reedy]

On 7/19/2018 4:33 AM, Antoine Pitrou wrote:


This is adding a whole range of new operators without enough of a use
case.  It is also making code harder to read, as evaluation can stop at
any of the "?*" operators.  And it looks like noise (or like Perl 6,
which is the same).

There is a use case I sympathize with: the argument-is-None case.  For
that I would suggest a simpler form:  "A else B" which would evaluate
to A if A is not None, otherwise to B (parentheses may be mandatory).

So e.g. one of the examples would read:

  def insort_right(a, x, lo=0, hi=None):
  # ...
  hi = hi else len(a)
  # ...


I like this.  (A or B) and (A and B)  could now* be explained as an 
abbreviations of


A if A else B
A if not A else B

but with A only evaluated once, as in

tem = A; tem if tem else B
tem = A; tem if not A else B

(A if A else B) is equivalent to (A if bool(A) is not False else B)

(A else B) is then easily explained as an abbreviation of

A if A is not None else B

that only evaluates A once.


* In the future, tem if (tem := A) else B

--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Brendan Barnwell]

On 2018-07-19 06:38, Steven D'Aprano wrote:

*Its just spelling*. If it is a useful and well-defined feature, we'll
get used to the spelling soon enough.

That's not to say that spelling is not important *at all*, or that we
should never prefer words to symbols. But if the only objection we have
is "this is useful but I don't like the spelling so -1" then that's
usually a pretty weak argument against the feature.


But we already have a spelling for the most common case.  It is:

x = a if a is not None else b

	That is the only use case of any of these operators that is actually 
common enough for me to care about --- but it's still not common enough 
to warrant the creation of a new operator, let alone multiple new operators.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Brendan Barnwell]

On 2018-07-19 02:11, Chris Angelico wrote:

Okay. What about bitwise operators, then? They don't have centuries of
mathematical backing to support them, yet it isn't considered
"unpythonic" to have &|^~ peppering our code. Judging by the current
levels of backlash against symbolic operators, it would have been
better to use "more explicit" function calls for all bitwise
operations. Coalescing None to a value is_at least_  as common as
performing bit manipulations in integers.


	The use of & to mean "and" does indeed have centuries of backing to 
support it.  The other operators, and the use of & to specifically mean 
bitwise-and, are newer, but still go back decades in other programming 
languages.  Moreover, the fact that these are bitwise versions of more 
general logical operations (conjunction, disjunction, etc.) means that 
they can be overridden for custom types in a way that is still 
intuitive.  For instance, sets override & to mean "intersection" because 
it means "everything that is in set A AND in set B", and similarly for | 
and ^.


	So basically I do not regard these operators as bitwise operators. 
They are symbols standing for logical operations.  Their default 
implementations on built-in numeric types do bitwise operations, but 
that is not what the symbols "mean".  Their meaning is more general and 
bitwise operations are one reasonable narrowing of that meaning for use 
in the context of builtin numeric types, but there are many other uses 
of these symbols that are equally consistent with their meaning.


	To me, this notion of symbols with an abstract meaning which can be 
narrowed for particular types by overriding magic methods is one of 
Python's greatest strengths.  + doesn't mean "add two numbers", it means 
"add", and that makes sense for many types, and they can override 
__add__ to define sensible behavior for their own purposes.  [] doesn't 
mean "get list item" it means "get", and types can override __getitem__ 
to define sensible behavior for their own purposes.


	As far as I can see, these null-coalescing operators would break that 
model.  The PEP doesn't seem to provide for a "real" magic method 
allowing users to override the actual behavior of the method.  (You can 
only override __has_value__ to hook into it, but not define by fiat what 
A ?? B does, as you can with other operators.)  And I think the reason 
for this is that the operator itself is too specific, much more specific 
in semantics than other operators.  (I had similar doubts about adding 
the matrix-multiplication operator @.)


	People keep saying that this null-coalescing behavior is so common and 
useful, etc., but that hasn't been my experience at all.  In my 
experience, the desire to shortcut this kind of logic is more often a 
sign of corner-cutting and insufficiently specified data formats, and is 
likely to cause bugs later on.  Eventually it has to actually matter 
whether something is None or not, and these operators just kick that can 
down the road.  In terms of their abstract meaning, they are not 
remotely close to as common or useful as operators like & and |.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Terry Reedy]

On 7/19/2018 4:32 AM, Stephen J. Turnbull wrote:

Chris Angelico writes later in thread:
  > On Thu, Jul 19, 2018 at 9:55 AM, Giampaolo Rodola'  
wrote:

  > Personally, I'm +0 on this. It'd be a few small wins here and there,
  > nothing huge, and I could easily live without it; but it's something
  > that I know some people will love.

I am 100% in sync with the reasoning, but -0 on the PEP (and only that
high because the advocates are so passionate).

To be honest, code transformations like this

  >  class BaseUploadObject(object):
  >  def find_content_type(self, filename):
  >  ctype, encoding = mimetypes.guess_type(filename)
  >  if ctype is None:
  >  return 'application/octet-stream'
  >  else:
  >  return ctype

to this

  >  class BaseUploadObject(object):
  >  def find_content_type(self, filename):
  >  ctype, encoding = mimetypes.guess_type(filename)
  >  return ctype ?? 'application/octet-stream'

make me cringe.  Exactly one of two things is true:


It seems to me that the problem is returning None.  Guess_type should 
have default='application/octet-stream' (or whatever *is* the default) 
in its signature.


The premise of returning None as default is that users can follow with 
conditional code.  If we now think that this is too much of a burden, we 
should stop returning None, at least by default.



1.  mimetypes.guess_type guarantees that the only falsie it will ever
 return is None, or

2.  it doesn't.


or
3. it never returns anything other than a non-blank string unless the 
user so requests.



--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Performance improvements via static typing

[Michael Hall]

On Thu, Jul 19, 2018 at 10:07 AM, Stephan Houben 
wrote:

> You are aware of numba?
>
> https://numba.pydata.org/
>
> Stephan
>
> Op do 19 jul. 2018 16:03 schreef Eric Fahlgren :
>
>> On Thu, Jul 19, 2018 at 6:52 AM Michael Hall
>>  wrote:
>>
>>> While I am aware of projects like Cython and mypy, it seems to make
>>> sense for CPython to allow optional enforcement of type hints, with
>>> compiler optimizations related to it to be used. While this would not
>>> receive the same level of performance benefits as using ctypes directly,
>>> there do appear to be various gains available here.
>>>
>>
>> ​Just to make sure I understand:  In other words, they would no longer be
>> "hints" but "guarantees".  This would allow an optimizer pass much greater
>> latitude in code generation, somehow or other.​
>>
>> For purposes of illustration (this is not a proposal, just for
>> clarification):
>>
>> @guaranteed_types
>> def my_sqrt(x:c_double) -> c_double:
>> ...
>>
>> would tell the compiler that it's now possible to replace the general
>> PyObject marshalling of this function with a pure-C one that only accepts
>> doubles and woe be unto those who use it otherwise.
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
Less so than I probably should have been given the idea I'm pitching. I've
given it a quick look, and at a glance, it seems to already be capable of
the desired behavior specifically centered around performance. I do still
think it *may* be beneficial to have in the CPython reference
implementation alongside a standard grammar, which could further enable the
various libraries and python implementations to make use of the additional
knowledge that the type hint is intended as a more than a hint.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Performance improvements via static typing

[Michael Hall]

On Thu, Jul 19, 2018 at 10:02 AM, Eric Fahlgren 
wrote:

> On Thu, Jul 19, 2018 at 6:52 AM Michael Hall 
> wrote:
>
>> While I am aware of projects like Cython and mypy, it seems to make sense
>> for CPython to allow optional enforcement of type hints, with compiler
>> optimizations related to it to be used. While this would not receive the
>> same level of performance benefits as using ctypes directly, there do
>> appear to be various gains available here.
>>
>
> ​Just to make sure I understand:  In other words, they would no longer be
> "hints" but "guarantees".  This would allow an optimizer pass much greater
> latitude in code generation, somehow or other.​
>
> For purposes of illustration (this is not a proposal, just for
> clarification):
>
> @guaranteed_types
> def my_sqrt(x:c_double) -> c_double:
> ...
>
> would tell the compiler that it's now possible to replace the general
> PyObject marshalling of this function with a pure-C one that only accepts
> doubles and woe be unto those who use it otherwise.
>
>
Yes, Though this would likely require introducing new grammar of some sort
to have it treated by the compiler as a guarantee, rather than a hint.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Paul Moore]

On 19 July 2018 at 16:25, Eric V. Smith  wrote:
> It currently does something: it replaces all instances, just as if you
> hadn't supplied a count (see my example below). You can't change its
> behavior.

... without a deprecation cycle. Which is of course not worth it for
something which could much more easily be done by adding an rreplace
function - which is the real point of the comment.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Eric V. Smith]

On 7/19/2018 11:22 AM, Calvin Spealman wrote:
If its treated as a missing parameter, and currently doesn't do 
anything, then it wouldn't be used... right? and it could be safe to add 
behavior for it... right?


It currently does something: it replaces all instances, just as if you 
hadn't supplied a count (see my example below). You can't change its 
behavior.


Eric



On Thu, Jul 19, 2018 at 11:17 AM, Eric V. Smith > wrote:


On 7/19/2018 10:01 AM, Calvin Spealman wrote:

As an alternative suggestion: What if the count parameter to
str.replace() counted from the right with negative values? That
would be consistent with other things like indexing and slicing.


We couldn't make this change because negative values already have a
meaning: it's interpreted as a missing parameter:

 >>> 'abab'.replace('a', 'z')
'zbzb'
 >>> 'abab'.replace('a', 'z', 0)
'abab'
 >>> 'abab'.replace('a', 'z', 1)
'zbab'
 >>> 'abab'.replace('a', 'z', -1)
'zbzb'
 >>> 'abab'.replace('a', 'z', -2)
'zbzb'
 >>> 'abab'.replace('a', 'z', -100)
'zbzb'

I think .rreplace() is the better design.

Eric



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[MRAB]

On 2018-07-19 16:22, Calvin Spealman wrote:
If its treated as a missing parameter, and currently doesn't do 
anything, then it wouldn't be used... right? and it could be safe to add 
behavior for it... right?



Are you sure that it wouldn't break some existing code?

Plus, we already have .find/.rfind, .index/.rindex, etc, so 
.replace/.rreplace would be a better fit.


On Thu, Jul 19, 2018 at 11:17 AM, Eric V. Smith > wrote:


On 7/19/2018 10:01 AM, Calvin Spealman wrote:

As an alternative suggestion: What if the count parameter to
str.replace() counted from the right with negative values? That
would be consistent with other things like indexing and slicing.


We couldn't make this change because negative values already have a
meaning: it's interpreted as a missing parameter:

 >>> 'abab'.replace('a', 'z')
'zbzb'
 >>> 'abab'.replace('a', 'z', 0)
'abab'
 >>> 'abab'.replace('a', 'z', 1)
'zbab'
 >>> 'abab'.replace('a', 'z', -1)
'zbzb'
 >>> 'abab'.replace('a', 'z', -2)
'zbzb'
 >>> 'abab'.replace('a', 'z', -100)
'zbzb'

I think .rreplace() is the better design.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Multi-core reference count garbage collection

[MRAB]

On 2018-07-19 11:53, Jonathan Fine wrote:

Hi Stephan

Thank you for the extract from the GC Handbook, which I think I may have 
seen before. Yes, it is GOOD that it's an already known idea.


Searching for "buffered reference counting" I found
https://mail.python.org/pipermail/python-dev/2016-October/146696.html
in which Larry Hastings says that C-python "plays games with reference 
counts" which makes implementing "buffered reference counting" harder. 
And he gives examples.


Larry also writes [loc cit] about resurrecting objects. I don't know 
what he means by this. It may be something to do with weak references. 
Larry's post gives some details, in which difficulties may lie. In 
particular, he writes

===
https://mail.python.org/pipermail/python-dev/2016-October/146604.html
It's my contention that this API [for weak references] is simply 
untenable under the

Gilectomy, and that it needs to change to returning a new (strong)
reference.


[snip]

When an object's refcount drops to 0, the object's __del__ method (if 
defined) is called, and then the object's memory can be reclaimed.


But what if the __del__ method creates a new reference to the object?

The object's refcount is no longer 0, so the object is no longer garbage.

That's called "resurrecting an object".
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[MRAB]

On 2018-07-19 14:26, Stephan Houben wrote:

FWIW, I like ??

It is short and distinctive.
There is prior art in this spelling in c#.
It requires no new keyword, nor does it give new meaning to an existing one.

I understand why  ?[ needs to be spelled using only a single ?, but I am 
afraid it will be used infrequently, and people will accidentally write

  a??[x]
which is legal but different.

I found the example code in the PEP using ?. and ?[ hard to read.
?? and ??= are compelling, though.

One more question: what does this do?

del x
x ??= 42


Well, what does this do?

del x
if x is None:
x = 42

[snip]
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Calvin Spealman]

If its treated as a missing parameter, and currently doesn't do anything,
then it wouldn't be used... right? and it could be safe to add behavior for
it... right?

On Thu, Jul 19, 2018 at 11:17 AM, Eric V. Smith  wrote:

> On 7/19/2018 10:01 AM, Calvin Spealman wrote:
>
>> As an alternative suggestion: What if the count parameter to
>> str.replace() counted from the right with negative values? That would be
>> consistent with other things like indexing and slicing.
>>
>
> We couldn't make this change because negative values already have a
> meaning: it's interpreted as a missing parameter:
>
> >>> 'abab'.replace('a', 'z')
> 'zbzb'
> >>> 'abab'.replace('a', 'z', 0)
> 'abab'
> >>> 'abab'.replace('a', 'z', 1)
> 'zbab'
> >>> 'abab'.replace('a', 'z', -1)
> 'zbzb'
> >>> 'abab'.replace('a', 'z', -2)
> 'zbzb'
> >>> 'abab'.replace('a', 'z', -100)
> 'zbzb'
>
> I think .rreplace() is the better design.
>
> Eric
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Eric V. Smith]

On 7/19/2018 10:01 AM, Calvin Spealman wrote:
As an alternative suggestion: What if the count parameter to 
str.replace() counted from the right with negative values? That would be 
consistent with other things like indexing and slicing.


We couldn't make this change because negative values already have a 
meaning: it's interpreted as a missing parameter:


>>> 'abab'.replace('a', 'z')
'zbzb'
>>> 'abab'.replace('a', 'z', 0)
'abab'
>>> 'abab'.replace('a', 'z', 1)
'zbab'
>>> 'abab'.replace('a', 'z', -1)
'zbzb'
>>> 'abab'.replace('a', 'z', -2)
'zbzb'
>>> 'abab'.replace('a', 'z', -100)
'zbzb'

I think .rreplace() is the better design.

Eric
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Calvin Spealman]

The concatenated name reads a little weird. Alternatively, extend the
conditional expression syntax.

a = b if None c

equivalent to
a = b if b is None else c


On Thu, Jul 19, 2018 at 9:35 AM, Robert Vanden Eynde 
wrote:

> If we're about to use a new keyword, it could be infix too:
>
> a = b ifnone c
>
> Although the assignment version looks unusual:
>
> b ifnone= c
>
> Then with the "default b = c" would look like this:
>
> ifnone b = c
>
> Le jeu. 19 juil. 2018 à 15:30, Calvin Spealman  a
> écrit :
>
>> Operators that only vary by case would be... confusing. I'm not super
>> keen on the other syntax (either the ?? or .? operators) but I do think
>> they read well in C# where they come from. Different things work in
>> different languages, some times.
>>
>> What about a new keyword: default
>>
>> So you'd write the above examples like this:
>>
>> default hi = len(a)  # Only executes the assignment if the left-hand is
>> None
>> default encoding = sys.getdefaultencoding()
>>
>> On Thu, Jul 19, 2018 at 9:06 AM, Pål Grønås Drange > > wrote:
>>
>>> > I've started a subthread, just to discuss the ?= and ?? operators. And
>>> > something newish, that I call OR.
>>>
>>> I would think `||` would be much better.
>>>
>>> It could be a kind of "semantic or" which could use the aforementioned
>>> dunder has_value.
>>>
>>> -1, though, but to the general None-awareness.
>>>
>>> Pål
>>>
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Calvin Spealman]

It would be consistent to apply it to other functions and I'd be in favour
of that, yes.

On Thu, Jul 19, 2018 at 10:06 AM, Eric Fahlgren 
wrote:

> On Thu, Jul 19, 2018 at 7:01 AM Calvin Spealman 
> wrote:
>
>> As an alternative suggestion: What if the count parameter to
>> str.replace() counted from the right with negative values? That would be
>> consistent with other things like indexing and slicing.
>>
>
> ​That could certainly be made to work, but I think it fails the
> discoverability test.  You'd have the [:]-syntax and replace working one
> way, and split/rsplit, find/rfind and so on working another way.  Unless
> you're proposing the negative index for the latter ones also?​
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Performance improvements via static typing

[Stephan Houben]

You are aware of numba?

https://numba.pydata.org/

Stephan

Op do 19 jul. 2018 16:03 schreef Eric Fahlgren :

> On Thu, Jul 19, 2018 at 6:52 AM Michael Hall 
> wrote:
>
>> While I am aware of projects like Cython and mypy, it seems to make sense
>> for CPython to allow optional enforcement of type hints, with compiler
>> optimizations related to it to be used. While this would not receive the
>> same level of performance benefits as using ctypes directly, there do
>> appear to be various gains available here.
>>
>
> ​Just to make sure I understand:  In other words, they would no longer be
> "hints" but "guarantees".  This would allow an optimizer pass much greater
> latitude in code generation, somehow or other.​
>
> For purposes of illustration (this is not a proposal, just for
> clarification):
>
> @guaranteed_types
> def my_sqrt(x:c_double) -> c_double:
> ...
>
> would tell the compiler that it's now possible to replace the general
> PyObject marshalling of this function with a pure-C one that only accepts
> doubles and woe be unto those who use it otherwise.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Eric Fahlgren]

On Thu, Jul 19, 2018 at 7:01 AM Calvin Spealman  wrote:

> As an alternative suggestion: What if the count parameter to str.replace()
> counted from the right with negative values? That would be consistent with
> other things like indexing and slicing.
>

​That could certainly be made to work, but I think it fails the
discoverability test.  You'd have the [:]-syntax and replace working one
way, and split/rsplit, find/rfind and so on working another way.  Unless
you're proposing the negative index for the latter ones also?​
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Performance improvements via static typing

[Eric Fahlgren]

On Thu, Jul 19, 2018 at 6:52 AM Michael Hall 
wrote:

> While I am aware of projects like Cython and mypy, it seems to make sense
> for CPython to allow optional enforcement of type hints, with compiler
> optimizations related to it to be used. While this would not receive the
> same level of performance benefits as using ctypes directly, there do
> appear to be various gains available here.
>

​Just to make sure I understand:  In other words, they would no longer be
"hints" but "guarantees".  This would allow an optimizer pass much greater
latitude in code generation, somehow or other.​

For purposes of illustration (this is not a proposal, just for
clarification):

@guaranteed_types
def my_sqrt(x:c_double) -> c_double:
...

would tell the compiler that it's now possible to replace the general
PyObject marshalling of this function with a pure-C one that only accepts
doubles and woe be unto those who use it otherwise.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Calvin Spealman]

As an alternative suggestion: What if the count parameter to str.replace()
counted from the right with negative values? That would be consistent with
other things like indexing and slicing.

On Thu, Jul 19, 2018 at 9:47 AM, Eric Fahlgren 
wrote:

> On Wed, Jul 18, 2018 at 8:20 PM Graham Gott  com> wrote:
>
>>
>> Thoughts? Support/oppose?
>>
>
> ​
> +1, along with an overall rework of str methods to make them more
> consistent.
>
> The find, replace and split families should all gain the same
> tuple-as-search-string that endswith and startswith use.  (Discussed in the
> last year, I'm too lazy to look up references...)
>
> ​
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Performance improvements via static typing

[Michael Hall]

While I am aware of projects like Cython and mypy, it seems to make sense
for CPython to allow optional enforcement of type hints, with compiler
optimizations related to it to be used. While this would not receive the
same level of performance benefits as using ctypes directly, there do
appear to be various gains available here.

My main concern with this as a thought was how to specify type hints as
optional, as for maximum benefit, this shouldn't prevent the ability to
type hint functions that you don't want to be treated in this manner. While
I don't have an answer for that yet, I thought I'd toss the idea out there
first. If it needs to be seen in action before deciding if it makes sense
to add, I can work on a potential implementation soon, but right now, this
is just an idea.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[David Mertz]

"APL and Perl territory" means "use lots of punctuation characters in
somewhat cryptic ways, often combining several for a distinct semantics." I
did not mean "APL and Perl use those specific characters with the proposed
meaning."

On Thu, Jul 19, 2018, 9:39 AM Steven D'Aprano  wrote:

> On Wed, Jul 18, 2018 at 08:05:56PM -0400, David Mertz wrote:
>
> > '??' and '?.' and ?[]' are really just marching
> > into APL and Perl territory. Yes, I know such operators exist in other
> > languages, but it feels very unpythonic.
>
> If these features exist in "other languages", and *don't* exist in APL
> or Perl, how are they marching into APL and Perl territory?
>
> Perl territory, like this perhaps?
>
> print "hello world\n";
> @days = ("Monday", "Tuesday", "Wednesday");
> print $days[0]
>
> Yes, I can see why we use "Perl syntax" as an insult *wink*
>
> Okay, okay, my examples are a bit unfair. I deliberately chose examples
> where the syntax is almost identical to Python's. Aside from the array
> and scalar sigils @ and $ the above could be Python.
>
> Tens of thousands of non-English speakers have had to learn the meaning
> of what might as well be meaningless, random sets of symbols (to them)
> like "class", "import", "while" and "True". If they can do so, perhaps
> we English-speakers should stop complaining about how hard it is to
> memorise the meaning of a couple of symbols like ??. Surely its no more
> difficult than learning the various meanings of ** and [] which we've
> already done.
>
> *Its just spelling*. If it is a useful and well-defined feature, we'll
> get used to the spelling soon enough.
>
> That's not to say that spelling is not important *at all*, or that we
> should never prefer words to symbols. But if the only objection we have
> is "this is useful but I don't like the spelling so -1" then that's
> usually a pretty weak argument against the feature.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting str.rreplace()

[Eric Fahlgren]

On Wed, Jul 18, 2018 at 8:20 PM Graham Gott 
wrote:

>
> Thoughts? Support/oppose?
>

​
+1, along with an overall rework of str methods to make them more
consistent.

The find, replace and split families should all gain the same
tuple-as-search-string that endswith and startswith use.  (Discussed in the
last year, I'm too lazy to look up references...)

​
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Steven D'Aprano]

On Wed, Jul 18, 2018 at 08:05:56PM -0400, David Mertz wrote:

> '??' and '?.' and ?[]' are really just marching
> into APL and Perl territory. Yes, I know such operators exist in other
> languages, but it feels very unpythonic.

If these features exist in "other languages", and *don't* exist in APL 
or Perl, how are they marching into APL and Perl territory?

Perl territory, like this perhaps?

print "hello world\n";
@days = ("Monday", "Tuesday", "Wednesday");
print $days[0]

Yes, I can see why we use "Perl syntax" as an insult *wink*

Okay, okay, my examples are a bit unfair. I deliberately chose examples 
where the syntax is almost identical to Python's. Aside from the array 
and scalar sigils @ and $ the above could be Python.

Tens of thousands of non-English speakers have had to learn the meaning 
of what might as well be meaningless, random sets of symbols (to them) 
like "class", "import", "while" and "True". If they can do so, perhaps 
we English-speakers should stop complaining about how hard it is to 
memorise the meaning of a couple of symbols like ??. Surely its no more 
difficult than learning the various meanings of ** and [] which we've 
already done.

*Its just spelling*. If it is a useful and well-defined feature, we'll 
get used to the spelling soon enough.

That's not to say that spelling is not important *at all*, or that we 
should never prefer words to symbols. But if the only objection we have 
is "this is useful but I don't like the spelling so -1" then that's 
usually a pretty weak argument against the feature.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Robert Vanden Eynde]

If we're about to use a new keyword, it could be infix too:

a = b ifnone c

Although the assignment version looks unusual:

b ifnone= c

Then with the "default b = c" would look like this:

ifnone b = c

Le jeu. 19 juil. 2018 à 15:30, Calvin Spealman  a
écrit :

> Operators that only vary by case would be... confusing. I'm not super keen
> on the other syntax (either the ?? or .? operators) but I do think they
> read well in C# where they come from. Different things work in different
> languages, some times.
>
> What about a new keyword: default
>
> So you'd write the above examples like this:
>
> default hi = len(a)  # Only executes the assignment if the left-hand is
> None
> default encoding = sys.getdefaultencoding()
>
> On Thu, Jul 19, 2018 at 9:06 AM, Pål Grønås Drange 
> wrote:
>
>> > I've started a subthread, just to discuss the ?= and ?? operators. And
>> > something newish, that I call OR.
>>
>> I would think `||` would be much better.
>>
>> It could be a kind of "semantic or" which could use the aforementioned
>> dunder has_value.
>>
>> -1, though, but to the general None-awareness.
>>
>> Pål
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Calvin Spealman]

Operators that only vary by case would be... confusing. I'm not super keen
on the other syntax (either the ?? or .? operators) but I do think they
read well in C# where they come from. Different things work in different
languages, some times.

What about a new keyword: default

So you'd write the above examples like this:

default hi = len(a)  # Only executes the assignment if the left-hand is None
default encoding = sys.getdefaultencoding()

On Thu, Jul 19, 2018 at 9:06 AM, Pål Grønås Drange 
wrote:

> > I've started a subthread, just to discuss the ?= and ?? operators. And
> > something newish, that I call OR.
>
> I would think `||` would be much better.
>
> It could be a kind of "semantic or" which could use the aforementioned
> dunder has_value.
>
> -1, though, but to the general None-awareness.
>
> Pål
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Nathan]

Hello.

Even if I'm -1 on the general idea of a None-aware, for a lot of reason already 
mentionned in this thread, the syntax "??" proposed (that I find much less 
readable than the current version) is still better IMHO.
How many programmer will read "||" or "OR" as a classic "or" operator? 
Programmers needs to learn the language, but making it easy for them to 
understand and learn is important in python.
More than that, if you use "||" or "OR" thinking you use "or", nothing will 
help you to understand what isn't working in your code.
So -1 on that idea even if PEP 505 were accepted.

Nathan Mugnier
 
 
 
 
> Message du 19/07/18 15:06
> De : "Pål Grønås Drange" 

> A : "Jonathan Fine" 
> Copie à : "Python-Ideas" 

> Objet : Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? 
> and OR
> 
>
> I've started a subthread, just to discuss the ?= and ?? operators. And
> > something newish, that I call OR.
I would think `||` would be much better.
It could be a kind of "semantic or" which could use the aforementioned dunder 
has_value.
-1, though, but to the general None-awareness.
Pål



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephan Houben]

FWIW, I like ??

It is short and distinctive.
There is prior art in this spelling in c#.
It requires no new keyword, nor does it give new meaning to an existing one.

I understand why  ?[ needs to be spelled using only a single ?, but I am
afraid it will be used infrequently, and people will accidentally write
 a??[x]
which is legal but different.

I found the example code in the PEP using ?. and ?[ hard to read.
?? and ??= are compelling, though.

One more question: what does this do?

del x
x ??= 42

Stephan


Op do 19 jul. 2018 15:00 schreef Judah Levy :

> On Thu, Jul 19, 2018 at 8:47 AM Rhodri James  wrote:
>
>> On 19/07/18 09:33, Antoine Pitrou wrote:
>> > There is a use case I sympathize with: the argument-is-None case.  For
>> > that I would suggest a simpler form:  "A else B" which would evaluate
>> > to A if A is not None, otherwise to B (parentheses may be mandatory).
>> >
>> > So e.g. one of the examples would read:
>> >
>> >   def insort_right(a, x, lo=0, hi=None):
>> >   # ...
>> >   hi = hi else len(a)
>> >   # ...
>>
>> Much as I would like a keyword, "else" is the wrong one.  It implies we
>> are dealing with truthiness, which we aren't, and lays a subtle semantic
>> trap as a consequence.
>>
>> If anyone can think of a good word for "if it isn't None, otherwise",
>> I'd be all for it :-)
>>
>> --
>> Rhodri James *-* Kynesim Ltd
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> I think that it may look better with the order switched and the word
> unless, as in
>
> def insort_right(a, x, lo=0 hi=None):
> # ...
> hi = len(a) unless hi
> # ...
>
> Unfortunately, this does maybe feel more like checking for truthiness than
> non-Null value
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Paul Moore]

On 19 July 2018 at 13:39, Rhodri James  wrote:

>> After updating to use the ``?[]`` and ``??`` operators::
>>
>>  def _get_const_info(const_index, const_list):
>>  argval = const_list?[const_index] ?? const_index
>>  return argval, repr(argval)
>
>
> Here's where I start to part company.  To me, the updated version is
> markedly harder to read, even if it does (once deciphered) convey the intent
> of the function better than the original :-)  The "?." and "?[]" operators
> just aren't obvious enough not to trip my internal WTF filter; either that
> or I'll overlook the "?" part entirely, which is probably worse.

I completely agree. The semantics of ?. and ?[] is non-intuitive at
best - and particularly when chained. The Groovy language has the ?.
operator (see http://groovy-lang.org/operators.html#_safe_navigation_operator),
and I pretty much always read it as if it were the equivalent
expression without the ? signs, with an added proviso "it'll probably
do something useful if we hit None". But when it comes to what that
"something useful" is, I'm left hoping the original writer knew what
they were doing.

So I'd never write code using ?. or ?[] myself, and I'd be unable to
reasonably review or maintain code containing them. That's a pretty
serious condemnation of the syntax in my view.

Conversely, while I find ?= and ?? ugly, and would avoid them, the
semantics are relatively easy to assess when reading code. Of the two,
I dislike ?? more than ?=, but both are streets ahead of ?. and ?[].

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Pål Grønås Drange]

> I've started a subthread, just to discuss the ?= and ?? operators. And
> something newish, that I call OR.

I would think `||` would be much better.

It could be a kind of "semantic or" which could use the aforementioned
dunder has_value.

-1, though, but to the general None-awareness.

Pål
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Paul Moore]

On 19 July 2018 at 13:30, Jonathan Fine  wrote:
> Hi
>
>> There is a formatted version of this PEP at
>> https://www.python.org/dev/peps/pep-0505/
>
> I've taken a look at this, and have some comments on the first two
> examples drawn from standard library code. (And a very grateful +10
> for writing a script to find such examples.)
>
> I've started a subthread, just to discuss the ?= and ?? operators. And
> something newish, that I call OR.
>
> FIRST EXAMPLE
> The first example is
> ---
> From bisect.py:
> def insort_right(a, x, lo=0, hi=None):
> # ...
> if hi is None:
> hi = len(a)
> ---
>
> Here, None is a sentinel value. The simpler code
> ---
>   hi = hi or len(a)
> ---
> fails when hi is zero (or any other value that is False in the boolean 
> context).
>
> This can be fixed by introducing a new operator OR which is similar to
> 'or' but has the semantics this example requires. Thus, given OR we
> can write
> ---
>   hi = hi OR len(a)
> ---
> where (A OR B) returns A if A is not None, otherwise it returns B.
>
> (Recall that (A or B) returns A if bool(A), otherwise it returns B.)

How does (A OR B) differ from the PEP's (A ?? B)?

I don't particularly like it in either form. But ?? at least has the
advantage of being used in other languages.

[...]
> Comments ?? suggestions. For example, would a None-aware AND operator be 
> useful?

I see no value in a None-aware AND. Real use cases are the only
reasonable argument for such a thing, not arguments based on symmetry
(or generalisation). And I doubt I'd find real-world use cases
particularly compelling.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Judah Levy]

On Thu, Jul 19, 2018 at 8:47 AM Rhodri James  wrote:

> On 19/07/18 09:33, Antoine Pitrou wrote:
> > There is a use case I sympathize with: the argument-is-None case.  For
> > that I would suggest a simpler form:  "A else B" which would evaluate
> > to A if A is not None, otherwise to B (parentheses may be mandatory).
> >
> > So e.g. one of the examples would read:
> >
> >   def insort_right(a, x, lo=0, hi=None):
> >   # ...
> >   hi = hi else len(a)
> >   # ...
>
> Much as I would like a keyword, "else" is the wrong one.  It implies we
> are dealing with truthiness, which we aren't, and lays a subtle semantic
> trap as a consequence.
>
> If anyone can think of a good word for "if it isn't None, otherwise",
> I'd be all for it :-)
>
> --
> Rhodri James *-* Kynesim Ltd
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>

I think that it may look better with the order switched and the word
unless, as in

def insort_right(a, x, lo=0 hi=None):
# ...
hi = len(a) unless hi
# ...

Unfortunately, this does maybe feel more like checking for truthiness than
non-Null value
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Rhodri James]

On 19/07/18 07:06, Greg Ewing wrote:

Chris Angelico wrote:

I'd love to hear an explanation of WHY this doesn't look like Python
any more. For instance, is the + operator somehow wrong for Python,
and it should have been the word "add"?


There's a very long tradition of using the symbol "+" to
represent addition, so it's something most people are
familiar with. There's no such tradition for the new
operators being proposed.


There is, actually, it's just not a long one.  C# has had null-aware 
operators for a while, for example.  I'll admit the syntax sits better 
in a C-like language, though.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Rhodri James]

On 19/07/18 09:33, Antoine Pitrou wrote:

There is a use case I sympathize with: the argument-is-None case.  For
that I would suggest a simpler form:  "A else B" which would evaluate
to A if A is not None, otherwise to B (parentheses may be mandatory).

So e.g. one of the examples would read:

  def insort_right(a, x, lo=0, hi=None):
  # ...
  hi = hi else len(a)
  # ...


Much as I would like a keyword, "else" is the wrong one.  It implies we 
are dealing with truthiness, which we aren't, and lays a subtle semantic 
trap as a consequence.


If anyone can think of a good word for "if it isn't None, otherwise", 
I'd be all for it :-)


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Rhodri James]

On 18/07/18 18:43, Steve Dower wrote:
Possibly this is exactly the wrong time to propose the next big syntax 
change, since we currently have nobody to declare on it, but since we're 
likely to argue for a while anyway it probably can't hurt (and maybe 
this will become the test PEP for whoever takes the reins?).


Many thanks for your work on this, Steve.  Having just written a fair 
bit of code that would have benefited from None-coalescing, I'm +1 for 
"??" and "??=" and +0 for the rest.



Grammar changes
---

The following rules of the Python grammar are updated to read::

     augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | 
'^=' |

     '<<=' | '>>=' | '**=' | '//=' | '??=')

     power: coalesce ['**' factor]
     coalesce: atom_expr ['??' factor]
     atom_expr: ['await'] atom trailer*
     trailer: ('(' [arglist] ')' |
   '[' subscriptlist ']' |
   '?[' subscriptlist ']' |
   '.' NAME |
   '?.' NAME)


I was a bit worried about the priority here, but your explanations 
elsewhere make sense.  Perhaps the PEP would benefit from being a bit 
more explicit about this?



 From ``calendar.py``::

     encoding = options.encoding
     if encoding is None:
     encoding = sys.getdefaultencoding()
     optdict = dict(encoding=encoding, css=options.css)

After updating to use the ``??`` operator::

     optdict = dict(encoding=encoding ?? sys.getdefaultencoding(),
    css=options.css)


ITYM
  optdict = dict(encoding=options.encoding ?? sys.getdefaultencoding(),
 css=options.css)


 From ``dis.py``::

     def _get_const_info(const_index, const_list):
     argval = const_index
     if const_list is not None:
     argval = const_list[const_index]
     return argval, repr(argval)

After updating to use the ``?[]`` and ``??`` operators::

     def _get_const_info(const_index, const_list):
     argval = const_list?[const_index] ?? const_index
     return argval, repr(argval)


Here's where I start to part company.  To me, the updated version is 
markedly harder to read, even if it does (once deciphered) convey the 
intent of the function better than the original :-)  The "?." and "?[]" 
operators just aren't obvious enough not to trip my internal WTF filter; 
either that or I'll overlook the "?" part entirely, which is probably worse.



Just use a conditional expression
-

Another common way to initialize default values is to use the ternary 
operator.

Here is an excerpt from the popular `Requests package
`_::

     data = [] if data is None else data
     files = [] if files is None else files
     headers = {} if headers is None else headers
     params = {} if params is None else params
     hooks = {} if hooks is None else hooks

This particular formulation has the undesirable effect of putting the 
operands
in an unintuitive order: the brain thinks, "use ``data`` if possible and 
use
``[]`` as a fallback," but the code puts the fallback *before* the 
preferred

value.

The author of this package could have written it like this instead::

     data = data if data is not None else []
     files = files if files is not None else []
     headers = headers if headers is not None else {}
     params = params if params is not None else {}
     hooks = hooks if hooks is not None else {}

This ordering of the operands is more intuitive, but it requires 4 extra
characters (for "not "). It also highlights the repetition of identifiers:
``data if data``, ``files if files``, etc.


While the four whole extra characters bothers me not one bit, the 
repetition is more of an issue.  It's much worse if what you have is 
more like:


item = spam.spam.spam.eggs if spam.spam.spam.eggs is not None else beans

--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Jonathan Fine]

Hi

> There is a formatted version of this PEP at
> https://www.python.org/dev/peps/pep-0505/

I've taken a look at this, and have some comments on the first two
examples drawn from standard library code. (And a very grateful +10
for writing a script to find such examples.)

I've started a subthread, just to discuss the ?= and ?? operators. And
something newish, that I call OR.

FIRST EXAMPLE
The first example is
---
>From bisect.py:
def insort_right(a, x, lo=0, hi=None):
# ...
if hi is None:
hi = len(a)
---

Here, None is a sentinel value. The simpler code
---
  hi = hi or len(a)
---
fails when hi is zero (or any other value that is False in the boolean context).

This can be fixed by introducing a new operator OR which is similar to
'or' but has the semantics this example requires. Thus, given OR we
can write
---
  hi = hi OR len(a)
---
where (A OR B) returns A if A is not None, otherwise it returns B.

(Recall that (A or B) returns A if bool(A), otherwise it returns B.)

SECOND EXAMPLE
The second example is
---
>From calendar.py:
encoding = options.encoding
if encoding is None:
encoding = sys.getdefaultencoding()
optdict = dict(encoding=encoding, css=options.css)
---

Once we have OR we would write this as
---
encoding = encoding OR sys.getdefaultencoding()
optdict = dict(encoding=encoding, css=options.css)
---

And from here we can condense into a single (longish) line:
---
optdict = dict(encoding=encoding OR sys.getdefaultencoding(), css=options.css)
--

SUMMARY
Here, for reference, are the suggestions using ?= and ?? in PEP 505.
---
hi ??= len(a)
---
optdict = dict(encoding=encoding ?? sys.getdefaultencoding(), css=options.css)
---

Comments ?? suggestions. For example, would a None-aware AND operator be useful?

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[David Mertz]

On Thu, Jul 19, 2018, 5:12 AM Chris Angelico  wrote:

> On Thu, Jul 19, 2018 at 4:06 PM, Greg Ewing 
> wrote:
> > There's a very long tradition of using the symbol "+" to
> > represent addition, so it's something most people are
> > familiar with. There's no such tradition for the new
> > operators being proposed.
>
> Okay. What about bitwise operators, then? They don't have centuries of
> mathematical backing to support them, yet it isn't considered
> "unpythonic" to have &|^~ peppering our code.
>

I have quite literally NEVER seem Python code with much use of the bitwise
operators. I guess the closest I've come is in some NumPy and Pandas code
where filters cannot use plain 'and' and ' or' but every student would find
it more intuitive if they could. E.g.

myDf[(myDf.field1 > 4) & (myDf.field2 < 2)]

Everyone still gets tripped up by the need for those parentheses because of
operator precedence. But this already yells or "special domain" rather than
"plain Python".

Indeed, it not uncommon or unreasonable to recommend using np.bitwise_and()
to avoid those confusing operators.

In the case where bitwise masking is used in its C manner, the code screams
special domain even more loudly. It definitely feels strongly unPythonic,
but it's often a reasonable compromise for dealing with just a little bit
of binary data without having to write a C extension.

I would ALWAYS want the code that used bitwise operators wrapped in a
separate function that most users and developers didn't need to look at,
but rather they'd call a more Pythonic API for the overall operation.

A huge difference is that bitwise operators do something you simply cannot
do other ways in Python at all. None-aware operators *at best* allow you to
write something with an existing straightforward approach using a couple
fewer lines. I think the result is ALWAYS less clear to read. Code golf is
an anti-goal in Python.

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Giampaolo Rodola']

On Thu, Jul 19, 2018 at 11:22 AM Antoine Pitrou  wrote:
>
> On Thu, 19 Jul 2018 19:11:33 +1000
> Chris Angelico  wrote:
>
> > On Thu, Jul 19, 2018 at 4:06 PM, Greg Ewing  
> > wrote:
> > > Chris Angelico wrote:
> > >>
> > >> I'd love to hear an explanation of WHY this doesn't look like Python
> > >> any more. For instance, is the + operator somehow wrong for Python,
> > >> and it should have been the word "add"?
> > >
> > >
> > > There's a very long tradition of using the symbol "+" to
> > > represent addition, so it's something most people are
> > > familiar with. There's no such tradition for the new
> > > operators being proposed.
> >
> > Okay. What about bitwise operators, then? They don't have centuries of
> > mathematical backing to support them, yet it isn't considered
> > "unpythonic" to have &|^~ peppering our code.
>
> They have decades of widespread presence in other programming
> languages, though.
>
> > Coalescing None to a value is _at least_ as common as
> > performing bit manipulations in integers.
>
> Certainly, but spelling that as a "?*" operator is a syntactical
> novelty.
>
> Consider that for the ternary operator, Python chose "B if A else C"
> over "A ? B : C", even though the latter had precedent in several
> languages.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

^
Agreed. To me even bitwise operators "feel" a bit weird when using
them but fortunately they are rare. On the other hand "?" has the
potential to be used (and abused) much more than bitwise operators.
Also I don't consider "since we have X then let's add Y" a valid
enough reasoning.

-- 
Giampaolo - http://grodola.blogspot.com
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Multi-core reference count garbage collection

[Jonathan Fine]

Hi Stephan

Thank you for the extract from the GC Handbook, which I think I may have
seen before. Yes, it is GOOD that it's an already known idea.

Searching for "buffered reference counting" I found
https://mail.python.org/pipermail/python-dev/2016-October/146696.html
in which Larry Hastings says that C-python "plays games with reference
counts" which makes implementing "buffered reference counting" harder. And
he gives examples.

Larry also writes [loc cit] about resurrecting objects. I don't know what
he means by this. It may be something to do with weak references. Larry's
post gives some details, in which difficulties may lie. In particular, he
writes
===
https://mail.python.org/pipermail/python-dev/2016-October/146604.html
It's my contention that this API [for weak references] is simply untenable
under the
Gilectomy, and that it needs to change to returning a new (strong)
reference.
===

However, I'm focused on the larger picture right now. And it good to know
something of where the hazards lie.

Once again, Stephan, thank you for your contribution to this thread.


On Thu, Jul 19, 2018 at 9:47 AM, Stephan Houben 
wrote:

> Hi Jonathan,
>
> 2018-07-19 8:33 GMT+02:00 Jonathan Fine :
>
>> I call any such scheme BUFFERED multi-core reference count garbage
>> collection. The worker processes know nothing about how garbage collection
>> is managed. Instead, they pass over to the GC process sufficient
>> information to allow it to manage the garbage.
>>
>
> This is actually a known idea (which is GOOD, unless you wanted to apply
> for a patent ;-) ).
>
> It is described, among other places, in
> "The Garbage Collection Handbook: The Art of Automatic Memory Management",
> by Richard Jones, Antony Hosking, Eliot Moss.
>
> In fact, they also call it buffered reference counting.
> Section 18.2: Buffered Reference Counting:
> "...DeTreville[1990] had log mutators the old and new referents of each
> pointer update to a buffer"
>
> By the way, this book describes a ton of other ideas to speed up
> reference counting in general and in the concurrent case, so
> it may be worthwhile to get it.
>
> I should warn you that many of these advanced refcounting ideas have been
> proposed in the past already, although I am unaware of this particular
> technique
> having been implemented.
>
> Stephan
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Antoine Pitrou]

On Thu, 19 Jul 2018 19:11:33 +1000
Chris Angelico  wrote:

> On Thu, Jul 19, 2018 at 4:06 PM, Greg Ewing  
> wrote:
> > Chris Angelico wrote:  
> >>
> >> I'd love to hear an explanation of WHY this doesn't look like Python
> >> any more. For instance, is the + operator somehow wrong for Python,
> >> and it should have been the word "add"?  
> >
> >
> > There's a very long tradition of using the symbol "+" to
> > represent addition, so it's something most people are
> > familiar with. There's no such tradition for the new
> > operators being proposed.  
> 
> Okay. What about bitwise operators, then? They don't have centuries of
> mathematical backing to support them, yet it isn't considered
> "unpythonic" to have &|^~ peppering our code.

They have decades of widespread presence in other programming
languages, though.

> Coalescing None to a value is _at least_ as common as
> performing bit manipulations in integers.

Certainly, but spelling that as a "?*" operator is a syntactical
novelty.

Consider that for the ternary operator, Python chose "B if A else C"
over "A ? B : C", even though the latter had precedent in several
languages.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Chris Angelico]

On Thu, Jul 19, 2018 at 4:06 PM, Greg Ewing  wrote:
> Chris Angelico wrote:
>>
>> I'd love to hear an explanation of WHY this doesn't look like Python
>> any more. For instance, is the + operator somehow wrong for Python,
>> and it should have been the word "add"?
>
>
> There's a very long tradition of using the symbol "+" to
> represent addition, so it's something most people are
> familiar with. There's no such tradition for the new
> operators being proposed.

Okay. What about bitwise operators, then? They don't have centuries of
mathematical backing to support them, yet it isn't considered
"unpythonic" to have &|^~ peppering our code. Judging by the current
levels of backlash against symbolic operators, it would have been
better to use "more explicit" function calls for all bitwise
operations. Coalescing None to a value is _at least_ as common as
performing bit manipulations in integers.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Greg Ewing]

Chris Angelico wrote:

I'd love to hear an explanation of WHY this doesn't look like Python
any more. For instance, is the + operator somehow wrong for Python,
and it should have been the word "add"?


There's a very long tradition of using the symbol "+" to
represent addition, so it's something most people are
familiar with. There's no such tradition for the new
operators being proposed.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Stephan Houben]

Hi Nathaniel,

2018-07-19 1:33 GMT+02:00 Nathaniel Smith :

> Note that this everything you said here also exactly describes the
> programming model for the existing 'multiprocessing' module:
> "structured clone" is equivalent to how multiprocessing uses pickle to
> transfer arbitrary objects, or you can use multiprocessing.Array to
> get a shared view on raw "C"-style data.
>


This is true. In fact, I am a big fan of multiprocessing and I think it is
often
overlooked/underrated. Experience with multiprocessing is also
what has me convinced that share-nothing or share-explicit approach
to concurrency is a useful programming model.

The main limitation of multiprocessing comes when you need to go outside
Python, and you need to interact with C/C++ libraries or operating services
from multiple processes.
The support for this generally varies from "extremely weak" to "none at
all".

For example, things I would like to in parallel with a main thread/process:

* Upload data to the GPU using OpenGL or OpenCL
* Generate a picture in pyqt QImage, then hand over zero-copy to main thread
* interact with a complex scenegraph in C++ (shared with main thread)

This is impossible right now but would be possible if the interpreters were
all in-process.

In addition, there are things which are now hard with "multiprocessing" but
could be fixed.
For example, sharing a Numpy array is possible but very inconvenient.
You need to first allocate the raw data segment, communicate that, then
create in each process
an array which uses this data segment.

Ideally, this would rather work like this:

   ar = numpy.zeros((30, 30), shared=True)

and then "ar" would automatically be shared.

This is fixable but given the other limitations above the question is if it
is worthwhile
to fix it now. It would be a lot simpler to fix if we had the in-process
model.

But yeah, I am actually also very open to ideas on how multiprocessing
could be
made more convenient and powerful. Perhaps there are ways, and I am just
not seeing them.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Multi-core reference count garbage collection

[Stephan Houben]

Hi Jonathan,

2018-07-19 8:33 GMT+02:00 Jonathan Fine :

> I call any such scheme BUFFERED multi-core reference count garbage
> collection. The worker processes know nothing about how garbage collection
> is managed. Instead, they pass over to the GC process sufficient
> information to allow it to manage the garbage.
>

This is actually a known idea (which is GOOD, unless you wanted to apply
for a patent ;-) ).

It is described, among other places, in
"The Garbage Collection Handbook: The Art of Automatic Memory Management",
by Richard Jones, Antony Hosking, Eliot Moss.

In fact, they also call it buffered reference counting.
Section 18.2: Buffered Reference Counting:
"...DeTreville[1990] had log mutators the old and new referents of each
pointer update to a buffer"

By the way, this book describes a ton of other ideas to speed up
reference counting in general and in the concurrent case, so
it may be worthwhile to get it.

I should warn you that many of these advanced refcounting ideas have been
proposed in the past already, although I am unaware of this particular
technique
having been implemented.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Chris Angelico]

On Thu, Jul 19, 2018 at 5:45 PM, Brice Parent  wrote:
> The biggest drawback of this, is that (if I understand it well), it may be
> done quite easily without any change to the language:
>
> def first_set(*elements):  # please don't mind the name of the function,
> it's not the purpose here
> """ Will return the first element that is not None """
> for element in elements:
> if element is not None:
> return element
>
> raise AllNoneException()
>
> first_set(3, 5)  # -> 3
> first_set(None, 5)  # -> 5
> first_set(None, None, 8, 10)  # -> 8
> first_set(None, Car(model="sport")).buy()  # calling
> Car(model="sport").buy()
> first_set(None, ["a", "b", "c"])[1]  # -> "b"
> first_set(None, None)  # -> exception is raised
>
> (note that such function could even accept a "rejected_values" kwarg, like
> `rejected_values=(None, [], "")`, just by replacing the `if` clause by `if
> element not in rejected_values:`)

No it can't, for the same reason that the 'and' and 'or' operators
can't be implemented cleanly as functions: it short-circuits. The
right-hand operator _will not_ be evaluated unless the left is None.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Antoine Pitrou]

This is adding a whole range of new operators without enough of a use
case.  It is also making code harder to read, as evaluation can stop at
any of the "?*" operators.  And it looks like noise (or like Perl 6,
which is the same).

There is a use case I sympathize with: the argument-is-None case.  For
that I would suggest a simpler form:  "A else B" which would evaluate
to A if A is not None, otherwise to B (parentheses may be mandatory).

So e.g. one of the examples would read:

 def insort_right(a, x, lo=0, hi=None):
 # ...
 hi = hi else len(a)
 # ...


Regards

Antoine.


On Wed, 18 Jul 2018 10:43:36 -0700
Steve Dower  wrote:
> Possibly this is exactly the wrong time to propose the next big syntax 
> change, since we currently have nobody to declare on it, but since we're 
> likely to argue for a while anyway it probably can't hurt (and maybe 
> this will become the test PEP for whoever takes the reins?).
> 
> FWIW, Guido had previously indicated that he was generally favourable 
> towards most of this proposal, provided we could figure out coherent 
> semantics. Last time we tried, that didn't happen, so this time I've 
> made the semantics much more precise, have implemented and verified 
> them, and made much stronger statements about why we are proposing these.
> 
> Additional thanks to Mark Haase for writing most of the PEP. All the 
> fair and balanced parts are his - all the overly strong opinions are mine.
> 
> Also thanks to Nick Coghlan for writing PEPs 531 and 532 last time we 
> went through this - if you're unhappy with "None" being treated as a 
> special kind of value, I recommend reading those before you start 
> repeating them.
> 
> There is a formatted version of this PEP at 
> https://www.python.org/dev/peps/pep-0505/
> 
> My current implementation is at 
> https://github.com/zooba/cpython/tree/pep-505 (though I'm considering 
> removing some of the new opcodes I added and just generating more 
> complex code - in any case, let's get hung up on the proposal rather 
> than the implementation :) )
> 
> Let the discussions begin!
> 
> ---
> 
> PEP: 505
> Title: None-aware operators
> Version: $Revision$
> Last-Modified: $Date$
> Author: Mark E. Haase , Steve Dower 
> 
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 18-Sep-2015
> Python-Version: 3.8
> 
> Abstract
> 
> 
> Several modern programming languages have so-called "``null``-coalescing" or
> "``null``- aware" operators, including C# [1]_, Dart [2]_, Perl, Swift, 
> and PHP
> (starting in version 7). These operators provide syntactic sugar for common
> patterns involving null references.
> 
> * The "``null``-coalescing" operator is a binary operator that returns 
> its left
>operand if it is not ``null``. Otherwise it returns its right operand.
> * The "``null``-aware member access" operator accesses an instance 
> member only
>if that instance is non-``null``. Otherwise it returns ``null``. 
> (This is also
>called a "safe navigation" operator.)
> * The "``null``-aware index access" operator accesses an element of a 
> collection
>only if that collection is non-``null``. Otherwise it returns 
> ``null``. (This
>is another type of "safe navigation" operator.)
> 
> This PEP proposes three ``None``-aware operators for Python, based on the
> definitions and other language's implementations of those above. 
> Specifically:
> 
> * The "``None`` coalescing`` binary operator ``??`` returns the left 
> hand side
>if it evaluates to a value that is not ``None``, or else it evaluates and
>returns the right hand side. A coalescing ``??=`` augmented assignment
>operator is included.
> * The "``None``-aware attribute access" operator ``?.`` evaluates the 
> complete
>expression if the left hand side evaluates to a value that is not 
> ``None``
> * The "``None``-aware indexing" operator ``?[]`` evaluates the complete
>expression if the left hand site evaluates to a value that is not 
> ``None``
> 
> Syntax and Semantics
> 
> 
> Specialness of ``None``
> ---
> 
> The ``None`` object denotes the lack of a value. For the purposes of these
> operators, the lack of a value indicates that the remainder of the 
> expression
> also lacks a value and should not be evaluated.
> 
> A rejected proposal was to treat any value that evaluates to false in a
> Boolean context as not having a value. However, the purpose of these 
> operators
> is to propagate the "lack of value" state, rather that the "false" state.
> 
> Some argue that this makes ``None`` special. We contend that ``None`` is
> already special, and that using it as both the test and the result of these
> operators does not change the existing semantics in any way.
> 
> See the `Rejected Ideas`_ section for discussion on the rejected approaches.
> 
> Grammar changes
> ---
> 
> The following rules of the Python grammar are updated to read::

[Python-ideas] PEP 505: None-aware operators

[Stephen J. Turnbull]

Chris Angelico writes later in thread:
 > On Thu, Jul 19, 2018 at 9:55 AM, Giampaolo Rodola'  
 > wrote:

 > Personally, I'm +0 on this. It'd be a few small wins here and there,
 > nothing huge, and I could easily live without it; but it's something
 > that I know some people will love.

I am 100% in sync with the reasoning, but -0 on the PEP (and only that
high because the advocates are so passionate).

To be honest, code transformations like this

 >  class BaseUploadObject(object):
 >  def find_content_type(self, filename):
 >  ctype, encoding = mimetypes.guess_type(filename)
 >  if ctype is None:
 >  return 'application/octet-stream'
 >  else:
 >  return ctype

to this

 >  class BaseUploadObject(object):
 >  def find_content_type(self, filename):
 >  ctype, encoding = mimetypes.guess_type(filename)
 >  return ctype ?? 'application/octet-stream'

make me cringe.  Exactly one of two things is true:

1.  mimetypes.guess_type guarantees that the only falsie it will ever
return is None, or

2.  it doesn't.

In case 1, "ctype or 'application/octet-stream'" ALSO does the right
thing.  In case 2, ONLY "ctype or 'application/octet-stream'" does the
right thing, as few callers of BaseUploadObject.find_content_type will
be prepared for "", (), [], or any variety of 0 as the return value.
This example is extreme in that by the nature of application/octet-
stream, any caller that can handle it at all will do a very reasonable
thing if find_content_type barfs up a falsie that is not None. 

In many cases, of course, it would be better to (eventually) raise an
exception when a falsie escapes the expression.  But in this
particular example, it's hard to imagine that is true: if the user's
expectation is violated, they'll complain, and then you can go debug.
In the meantime, the application doesn't crash and work gets done.

The prevalence of these cringe-worthy examples in advocates' posts are
why I'm basically - on the idea.

So I would like to see various examples of code where

1.  in the original code, treating a falsie that is not None the same
way that None is treated is a detectable bug; and

2a. letting such falsies escape to raise exceptions somewhere else is
a good idea (and how you could know that); or

2b. catching spurious falsies and handling them here is a better idea;
or

2c. some falsies that are not None are legitimate.

That's three kinds of examples.  I suspect the 2b examples are going
to be pretty "meh", because you're saving little reader cognition or
code.  More important are the 2a examples, because I suspect that most
examples will fall into the same category as find_content_type: "or"
is as good as "??", and that "??" is sometimes an accident waiting to
happen due to falsie escapes (eg, when receiving objects from somebody
else's code who might not understand the contract that the only falsie
they should ever provide is None).

Steve

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[INADA Naoki]

Honestly speaking, I don't want make Python syntax more complex.

But when comparing to accepted PEP 572, I think this PEP is useful
often enough.  And PEP 505 doesn't break border between
expression and statement unlike PEP 572.

Especially, ?? and ??= seems useful very often.
And `x ?? default` seems much more readable than
`x if x is not None else default` or `default if x is None else x`.

On the other hand, `?.` and `?[]` seems useful less often and more confusing.

When looking `spam?.egg`, people can expect `getattr(spam, 'egg', None)`
rather than `spam.egg if spam is not None else None`.

Since `?.` and `?[]` can't avoid AttributeError, IndexError and KeyError,
I think it's confusing and not useful enough compared with it's ugliness.

So my current position is +1 for `??` and `??=`, but -1 for others.

Regards,
-- 
INADA Naoki  
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Brice Parent]

Hi,
I think this breaks one of the most important aspect which makes Python 
more and more popular every year: its readability. Until now, you didn't 
need to know the language very well to have some understanding about 
what a script did. Some parts of it could be a bit complicated (like 
comprehensions, which we only allow in unit tests and small utilities 
where I work, exactly for this reason) or non-obvious (like the new := 
operator, but even if you don't get exactly what it does, it's not 
really cryptic either. It's still some-kind-of-assignment).
We've been teaching for years that there is virtually no cost in adding 
a few simple lines of code from times to times, and that it has to be 
done to improve readability, while this proposal's goal seems to be the 
opposite (make the code more compact at the cost of readability), which 
complicates the understanding of the whole language to save a few `if x 
is None:`. To me, it goes the opposite of most of the first lines of the 
zen of Python.


Where I work, we update most of our projects to use new Python versions 
almost immediately when a new one is out and the libs we use are 
compatible. This would probably change that; we would likely wait for a 
few years, the time for the feature to be used widely in other projects, 
and for us to have some real-life feedback to know what to do about it, 
like: are there some cases it really is justified? Does it make it 
harder/simpler to read/write python code for experts and non-experts? 
Should we add a rule to enforce its use, or to prevent it?


The biggest drawback of this, is that (if I understand it well), it may 
be done quite easily without any change to the language:


def first_set(*elements):  # please don't mind the name of the function, 
it's not the purpose here

    """ Will return the first element that is not None """
    for element in elements:
    if element is not None:
    return element

    raise AllNoneException()

first_set(3, 5)  # -> 3
first_set(None, 5)  # -> 5
first_set(None, None, 8, 10)  # -> 8
first_set(None, Car(model="sport")).buy()  # calling 
Car(model="sport").buy()

first_set(None, ["a", "b", "c"])[1]  # -> "b"
first_set(None, None)  # -> exception is raised

(note that such function could even accept a "rejected_values" kwarg, 
like `rejected_values=(None, [], "")`, just by replacing the `if` clause 
by `if element not in rejected_values:`)


I might have missed some implications, or even not understood the PEP at 
all, though!


- Brice




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[George Leslie-Waksman]

I am rather fond of the idea of null-coalescing, at the very least, for
mutable default values:

def foo(a=None):
a ??= []
...

but I worry about the code messes we will run into with some of the other
options.

Woe be unto anyone forced to understand the behavior of:

thing?.attr?[key]?.subattr ?? 127

What if we added the Elvis operator "?:" for null coalescing and left the
rest for future consideration


On Wed, Jul 18, 2018 at 10:49 PM Tim Peters  wrote:

> [Steve Dower ]
>
>> ...
>
> * The "``None``-aware attribute access" operator ``?.`` evaluates the
>> complete expression if the left hand side evaluates to a value that is
>> not
>> ``None``
>>
>
> And if the LHS does evaluate to `None` ...?  I'll assume the result is
> also `None` then.
>
>
>> ...
>
>
>>  From ``inspect.py``::
>>
>>  for base in object.__bases__:
>>  for name in getattr(base, "__abstractmethods__", ()):
>>  value = getattr(object, name, None)
>>  if getattr(value, "__isabstractmethod__", False):
>>  return True
>>
>> After updating to use the ``?.`` operator (and deliberately not
>> converting to use ``any()``)::
>>
>>  for base in object.__bases__:
>>  for name in base?.__abstractmethods__ ?? ():
>>  if object?.name?.__isabstractmethod__:
>>  return True
>>
>
> I got lost on the `for` here.  The part following `in`:
>
> for name in getattr(base, "__abstractmethods__", ()):
>
> looks in `base` (regardless of whether `base` is `None`) for an attribute
> named "_abstractmethods__"..  If such an attribute exists, the value of
> the attribute is returned (`None` or not).  Else an AttributeError is
> swallowed and `()` is returned.  It's hard to see how
>
>
>  for name in base?.__abstractmethods__ ?? ():
>
> does the same.  If `base` itself is `None`, I guess it returns `()`, or
> if  `base` has an "_abstractmethods__" attribute then the value of that
> attribute is returned - unless its value is None, in which case `()` is
> again returned.  But if `base` is not `None` and the attribute does not
> exist, doesn't this raise AttributeError?  The later "Exception-aware
> operators" section seemed to explicitly reject the idea that `?.` and `?[]`
> would suppress AttributeError and/or TypeError.
>
> In short, the original getattr() didn't care at all whether `base` was
> `None`, or whether the value of its "__abstractmethods__" attribute was
> `None`, but cared a whole lot about whether that attribute exists.  I just
> can't see how the updated code matches that in any of those respects.
>
> Ignoring that and pressing on, I suffer the same kind of confusions on the
> `if` part.  What am I missing?  For example, do these operators swallow
> exceptions after all?
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Multi-core reference count garbage collection

[Jonathan Fine]

Hi

Based on other people's work (including in particular talks by Larry
Hastings) and my own thinking, I've come up with a scheme for multi-core
reference count garbage collection. I think it works, and much or all of it
comes from others. It might even be in the literature. Of course, if it's
wrong then the debit goes to me.

I'll explain it in instalments. Please let me know what you think, as we go
along.

The basic ideas of reference counting garbage collection are:

1. Each allocated piece of memory is given an ID (which is often its
address in memory).
2. For each ID, the system keeps a count of how many references (to that
piece of memory).
3. The count starts at ONE.
4. Processes increment and decrement the count, to keep the count correct.
5. When the count reaches zero, the piece of memory is garbage collected.
6. The previous step might result in further reference count decrements.

The simplest possible garbage collection scheme is to do nothing (and let
the garbage build up). In other words, allocate on a stack, and never free
memory. This has very good performance, until memory is exhausted. It is
also conceptually useful. I'll call it the do-nothing garbage collection
scheme.

Suppose for definiteness we have a 6-core CPU, with 5 worker processes and
one garbage collection (GC) process. The worker processes will do as little
as possible, consistent with not running out of memory. To achieve this:

1. Each worker process with have two GC buffers, one for increment and the
other for decrement.
2. For increment, the process will append the ID to the increment buffer
(for the process). And similarly for decrement.
3. If the buffer to be appended to is full, the worker process will
(a) set a buffer-full flag
(b) pause until the buffer-full flag has been cleared
(c) and then do what it was previously blocked from doing

I call any such scheme BUFFERED multi-core reference count garbage
collection. The worker processes know nothing about how garbage collection
is managed. Instead, they pass over to the GC process sufficient
information to allow it to manage the garbage.

This is now a good place to pause. We've specified what it is that the
worker processes should do, regarding garbage collection. And we've passed
the problem on to the garbage collection process. If that process does
nothing, we have refactored the do-nothing garbage collection scheme.

Expect another instalment tomorrow.

with best regards

Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Pau Freixes]

Hi,

>
> Note that this everything you said here also exactly describes the
> programming model for the existing 'multiprocessing' module:
> "structured clone" is equivalent to how multiprocessing uses pickle to
> transfer arbitrary objects, or you can use multiprocessing.Array to
> get a shared view on raw "C"-style data.
>

That's good. If finally, CPython can provide a pattern plus an API
that has similitudes with the existing ones will make the adoption
less friction.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/