Re: [Python-ideas] Python-ideas Digest, Vol 131, Issue 106

[Ivan Pozdeev via Python-ideas]

On 31.10.2017 8:37, python-ideas-requ...@python.org wrote:
On Tue, Oct 31, 2017 at 3:50 PM, Ivan Pozdeev via Python-ideas 
 wrote:

On 30.10.2017 17:32, Guido van Rossum wrote:

This is a key example of a case where code speaks. Can you write an
implementation of how you would want single() to work in Python code?

On Mon, Oct 30, 2017 at 2:49 AM, Ivan Pozdeev via Python-ideas
mailto:python-ideas@python.org>> wrote:

 The initial post on the above link summarizes the suggested
 implementation pretty well.


|defsingle(i): try: ||v =i.next()
|||exceptStopIteration:raiseException('No values')|||try: ||i.next()
||exceptStopIteration: ||returnv||else: ||raiseException('Too many values')|
||printsingle(name forname in('bob','fred')ifname=='bob')||| |

||

raise WhitespaceDamagedException from None

ChrisA


Thunderbird jerked on me big time. It never did anything like this 
before! Switched off Digest mode, individual messages aren't so complicated.


def single(i):
    try:
    v =i.next()
    except StopIteration:
    raise ValueError('No items')
    try:
    i.next()
    except StopIteration:
    return v
    else:
    raise ValueError('More than one item')

print single(name for name in('bob','fred') if name=='bob')

--

Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add processor generation to wheel metadata

[Nathaniel Smith]

On Mon, Oct 30, 2017 at 5:45 AM, Ivan Pozdeev via Python-ideas
 wrote:
> Generally, packages are compiled for the same processor generation as the
> corresponding Python.
> But not always -- e.g. NumPy opted for SSE2 even for Py2 to work around some
> compiler bug
> (https://github.com/numpy/numpy/issues/6428).
> I was bitten by that at an old machine once and found out that there is no
> way for `pip' to have checked for that.
> Besides, performance-oriented packages like the one mentioned could probably
> benefit from newer instructions.

You should probably resend this to distutils-sig instead of
python-ideas -- that's where discussions about python packaging
happen. (Python-ideas is more for discussions about the language
itself.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Nathaniel Smith]

On Mon, Oct 30, 2017 at 10:25 AM, Alexander Belopolsky
 wrote:
> On Mon, Oct 30, 2017 at 11:44 AM, Nick Coghlan  wrote:
> ..
>> 3. We can't replicate it as readily in the regular REPL, since that runs
>> Python code directly in the current process, but even there I believe we
>> could potentially trigger a full process restart via execve (or the C++
>> style _execve on Windows)
>
> This exact problem is solved rather elegantly in Julia.  When you
> upgrade a package that is already loaded in the REPL, it prints a
> warning:
>
> "The following packages have been updated but were already imported:
> ... Restart Julia to use the updated versions."
>
> listing the affected packages.
>
> See 
> .

This seems like the obvious solution to me too. Pip knows exactly
which files it modified. The interpreter knows which packages have
been imported. Having the REPL provide a friendly interface that ran
pip and then compared the lists would need some coordination between
the projects but wouldn't be rocket science, and would be *much* more
new-user-friendly than the current system.

(Also, I'm kind of grossed out by the attitude that it's a good thing
to drive people away by giving a bad first impression. Sure the shell
is worth learning, but it can wait until you actually need it. If you
make people fail for opaque reasons on basic tasks then the lesson
they learn isn't "oh I need to learn the shell", it's "oh I must be
stupid / maybe girls really can't do programming / I should give up".)

If you want to support conda too then cool, conda can install a
site.py that provides a conda() builtin that uses the same machinery.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add single() to itertools

[Chris Angelico]

On Tue, Oct 31, 2017 at 3:50 PM, Ivan Pozdeev via Python-ideas
 wrote:
> On 30.10.2017 17:32, Guido van Rossum wrote:
>>
>> This is a key example of a case where code speaks. Can you write an
>> implementation of how you would want single() to work in Python code?
>>
>> On Mon, Oct 30, 2017 at 2:49 AM, Ivan Pozdeev via Python-ideas
>> mailto:python-ideas@python.org>> wrote:
>>
>> The initial post on the above link summarizes the suggested
>> implementation pretty well.
>>
> |defsingle(i): try: ||v =i.next()
> |||exceptStopIteration:raiseException('No values')|||try: ||i.next()
> ||exceptStopIteration: ||returnv||else: ||raiseException('Too many values')|
> ||printsingle(name forname in('bob','fred')ifname=='bob')||| |
>
> ||

raise WhitespaceDamagedException from None

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add single() to itertools

[Ivan Pozdeev via Python-ideas]

On 30.10.2017 17:32, Guido van Rossum wrote:
This is a key example of a case where code speaks. Can you write an 
implementation of how you would want single() to work in Python code?


On Mon, Oct 30, 2017 at 2:49 AM, Ivan Pozdeev via Python-ideas 
mailto:python-ideas@python.org>> wrote:


The initial post on the above link summarizes the suggested
implementation pretty well.


|defsingle(i): try: ||v =i.next() |||exceptStopIteration:raiseException('No 
values')|||try: ||i.next() ||exceptStopIteration: ||returnv||else: 
||raiseException('Too many values')|
||printsingle(name forname in('bob','fred')ifname=='bob')||| |
||

--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Chris Barker - NOAA Federal]

It's not 100% clear to me how my proposal below would work within a
> Jupyter Notebook, so that would also be an angle worth looking into.
>

I'm -1 on this as I view it as a tooling issue, not a language issue.


Agreed. And for the tool at hand, the notebook already controls the python
interpreter— it could have a “package installer” that you could run from
the notebook, but NOT with python code in the notebook.

If the Jupyter developers though it as a good idea.

I also wouldn't want to tie Python-the-language to pip-the-tool so tightly.


Exactly, I really wouldn’t want my students pip installing stuff from side
the REPL in the conda-based environment I’ve set up for them

-CHB
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add single() to itertools

[Steven D'Aprano]

On Tue, Oct 31, 2017 at 07:51:02AM +1100, Cameron Simpson wrote:

>  return the(nodes)
> 
> It's this kind of thing that expresses my intent better than the:
> 
>  node, = nodes
>  return node
> 
> idiom.

If the intent is to indicate that there is only one node, then 
"the(nodes)" fails completely. "The" can refer to plurals as easily as 
singular:

"Wash the dirty clothes."
(Later) "Why did you only wash one sock?"


The simplest implementation of this "single()" function I can think of 
would be:

def single(iterable):
result, = iterable
return result


That raises ValueError if iterable has too few or too many items, which 
I believe is the right exception to use. Conceptually, there's no 
indexing involved, so IndexError would be the wrong exception to use. 
We're expecting a compound value (an iterable) with exactly one item. If 
there's not exactly one item, that's a ValueError.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add single() to itertools

[Cameron Simpson]

On 30Oct2017 07:32, Guido van Rossum  wrote:

This is a key example of a case where code speaks. Can you write an
implementation of how you would want single() to work in Python code?


Myself, I'm not advocating for putting such a thing in itertools. However, I do 
have an equivalent utility function of my own that makes for more readable 
code, named "the".


It sees far less use than I'd imagined it would, but it does read nicely to my 
eye when used. I have a few select-something from data where there _can_ be 
multiple hits i.e. the data format/structure support multiple matching results, 
but the caller's use case requires just one hit or failure.


So I have some fuzzy-db-lookup functions which end with:

 return the(rows)

or include:

 row = the(rows)

and some HTML find-this-DOM-node code which ends with:

 return the(nodes)

It's this kind of thing that expresses my intent better than the:

 node, = nodes
 return node

idiom. And as remarked, you can embed the() in an expression.

I don't think it ranks in "belongs in the stdlib". I do keep it about in a 
module for ready use though. If nothing else, it raises IndexErrors with 
distinct text for 0 and >1 failure.


Cheers,
Cameron Simpson  (formerly c...@zip.com.au)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Brett Cannon]

On Mon, 30 Oct 2017 at 03:36 Erik Bray  wrote:

> On Mon, Oct 30, 2017 at 11:27 AM, Erik Bray  wrote:
> > On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters 
> wrote:
> >> Then those users have more fundamental problems.  There is a minimum
> level
> >> of computer knowledge needed to be successful in programming.
> Insulating
> >> users from the reality of the situation is not preparing them to be
> >> successful.  Pretending that there is no system command prompt, or
> shell, or
> >> whatever platform specific term applies, only hurts new programmers.
> Give
> >> users an error message they can google, and they will be better off in
> the
> >> long run than they would be if we just ran pip for them.
> >
> > While I completely agree with this in principle, I think you
> > overestimate the average beginner.  Many beginners I've taught or
> > helped, even if they can manage to get to the correct command prompt,
> > often don't even know how to run the correct Python.  They might often
> > have multiple Pythons installed on their system--maybe they have
> > Anaconda, maybe Python installed by homebrew, or a Python that came
> > with an IDE like Spyder.  If they're on OSX often running "python"
> > from the command prompt gives the system's crippled Python 2.6 and
> > they don't know the difference.
>
>
> I should add--another case that is becoming extremely common is
> beginners learning Python for the first time inside the
> Jupyter/IPython Notebook.  And in my experience it can be very
> difficult for beginners to understand the connection between what's
> happening in the notebook ("it's in the web-browser--what does that
> have to do with anything on my computer??") and the underlying Python
> interpreter, file system, etc.  Being able to pip install from within
> the Notebook would be a big win.  This is already possible since
> IPython allows running system commands and it is possible to run the
> pip executable from the notebook, then manually restart the Jupyter
> kernel.
>
> It's not 100% clear to me how my proposal below would work within a
> Jupyter Notebook, so that would also be an angle worth looking into.
>

I'm -1 on this as I view it as a tooling issue, not a language issue. If
you're teaching in an environment where you don't want to instruct on the
differences between the REPL and a command prompt, then that suggests to me
that the environment you're presenting students needs to be different
rather than asking Python to work around your teaching environment
(remember, any time you ask for a change in Python you're asking the change
to be applied to literally millions of developers).

I also wouldn't want to tie Python-the-language to pip-the-tool so tightly.
While we may make sure that pip is available for convenience, Python as a
language is not dependent on pip being installed in order to function.
Adding something that shells out to pip wouldn't suddenly change that
relationship between language and tool.

-Brett


>
> Best,
> Erik
>
>
> > One thing that has been a step in the right direction is moving more
> > documentation toward preferring running `python -m pip` over just
> > `pip`, since this often has a better guarantee of running `pip` in the
> > Python interpreter you intended.  But that still requires one to know
> > how to run the correct Python interpreter from the command-line (which
> > the newbie double-clicking on IDLE may not even have a concept of...).
> >
> > While I agree this is something that is important for beginners to
> > learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for
> > many newbies just to install one or two packages from pip, which they
> > often might need/want to do for whatever educational pursuit they're
> > following (heck, it's pretty common even just to want to install the
> > `requests` module, as I would never throw `urllib` at a beginner).
> >
> > So while I don't think anything proposed here will work technically, I
> > am in favor of an in-interpreter pip install functionality.  Perhaps
> > it could work something like this:
> >
> > a) Allow it *only* in interactive mode:  running `pip(...)` (or
> > whatever this looks like) outside of interactive mode raises a
> > `RuntimeError` with the appropriate documentation
> > b) When running `pip(...)` the user is supplied with an interactive
> > prompt explaining that since installing packages with `pip()` can
> > result in changes to the interpreter, it is necessary to restart the
> > interpreter after installation--give them an opportunity to cancel the
> > action in case they have any work they need to save.  If they proceed,
> > install the new package then restart the interpreter for them.  This
> > avoids any ambiguity as to states of loaded modules before/after pip
> > install.
> >
> >
> >
> >> From: Stephan Houben [mailto:stephan...@gmail.com]
> >> Sent: Sunday, October 29, 2017 3:43 PM
> >> To: Alex Walters 
> >> Cc: Python-Ideas 
> >> Subject: Re: [Python-ideas] install pip 

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Guido van Rossum]

So what's your list? What would you put in requirements.txt? I'm fine with
doing it in the form of requirements.txt -- I am worried that when push
comes to shove, we won't be able to agree on what list of package names
should go into that file.

On Mon, Oct 30, 2017 at 12:08 PM, Juancarlo Añez  wrote:

>
>
> On Mon, Oct 30, 2017 at 12:29 PM, Guido van Rossum 
> wrote:
>
>> What's your proposed process to arrive at the list of recommended
>> packages? And is it really just going to be a list of names, or is there
>> going to be some documentation (about the vetting, not about the contents
>> of the packages) for each name?
>>
>
> As I see it, the bootstrap would be a requirements.txt (requirements.pip)
> with the packages that more experienced developers prefer to use over
> stdlib, pinned to package versions known to work well with the CPython
> release.
>
> I think this is a great idea, specially if it's an easy opt-in.
>
> For example, sometimes I go into a programming niche for a few years, and
> I'd very much appreciate a curated list of packages when I move to another
> niche. A pay-forward kind of thing.
>
> The downside to the idea is that the list of packages will need a curator
> (which could be python.org).
>
>
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] IMAP4.__exit__ counterintuitive for with blocks

[Koos Zevenhoven]

On Mon, Oct 30, 2017 at 8:30 PM, Drew  wrote:

> Yeah, a flag for IMAP4 objects sounds like it'd solve
> backwards-compatibility problems. E.g.
>
> imaplib.IMAP4_SSL(..., commit=manual/auto)
>
> As for handling errors with automatic commits, we should probably just
> defer to whatever Python IO does to remain consistent.
>
>
​Yeah, at least if it makes sense here too.

BTW, it so happens that an old contextlib feature strikes again and allows
you to write this in plain English, with punctuation and all:

with imaplib.IMAP4_SSL(..) as i, closing(i):
...


This one does call .close() regardless of errors, though, and possibly the
kind of error may affect whether the commit succeeds or not. Not sure if
that matters.

-- Koos





> On Oct 30, 2017, at 1:33 PM, Guido van Rossum  wrote:
>>
>> But maybe if __exit__ is called with an exception it should roll back.
>>
>> In any case it looks like your proposal could break existing code (e.g.
>> code that uses `with` depending on its current behavior, using some other
>> logic to decide whether to commit or not) and that feels difficult to
>> overcome. Perhaps a new method or flag argument can be added to request
>> that the transactions be automatically committed?
>>
>> On Mon, Oct 30, 2017 at 10:20 AM, Drew  wrote:
>>
>>> IMAP4.close closes the selected inbox and commits changes such as
>>> deletions. It is not called on IMAP4.__exit__ (only logout is, which
>>> doesn't call close in its call stack) however, so:
>>>
>>> with imaplib.IMAP4_SSL(...) as i:
>>> ...
>>>
>>> would fail to commit those changes. close must be explicitly invoked
>>> i.e.
>>>
>>> with imaplib.IMAP4_SSL(...) as i:
>>> ...
>>> i.close()
>>>
>>> This is counterintuitive however, as the with statement is meant to
>>> automatically clean up. Another programmer might come along and delete
>>> i.close() because it seems unnecessary. Now changes aren't being committed
>>> and the programmer doesn't realize it's because of this weird Python
>>> idiosyncracy.
>>>
>>> Python IO such as open commits changes automatically, so I'm not sure
>>> why IMAP4 doesn't and only logs out.
>>>
>>> __ _
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>
>>
>> --
>> --Guido van Rossum ( python.org/~guido)
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Alex Walters]

> -Original Message-
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Erik Bray
> Sent: Monday, October 30, 2017 6:28 AM
> To: Python-Ideas 
> Subject: Re: [Python-ideas] install pip packages from Python prompt
> 
> On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters 
> wrote:
> > Then those users have more fundamental problems.  There is a minimum
> level
> > of computer knowledge needed to be successful in programming.
> Insulating
> > users from the reality of the situation is not preparing them to be
> > successful.  Pretending that there is no system command prompt, or shell,
> or
> > whatever platform specific term applies, only hurts new programmers.
> Give
> > users an error message they can google, and they will be better off in the
> > long run than they would be if we just ran pip for them.
> 
> While I completely agree with this in principle, I think you
> overestimate the average beginner.

Nope.  I totally get that they don’t know what a shell or command prompt is.  
THEY. NEED. TO. LEARN.  Hiding it is not a good idea for anyone.  If this is an 
insurmountable problem for the newbie, maybe they really shouldn’t be 
attempting to program.  This field is not for everyone.

>  Many beginners I've taught or
> helped, even if they can manage to get to the correct command prompt,
> often don't even know how to run the correct Python.  They might often
> have multiple Pythons installed on their system--maybe they have
> Anaconda, maybe Python installed by homebrew, or a Python that came
> with an IDE like Spyder.  If they're on OSX often running "python"
> from the command prompt gives the system's crippled Python 2.6 and
> they don't know the difference.
> 
> One thing that has been a step in the right direction is moving more
> documentation toward preferring running `python -m pip` over just
> `pip`, since this often has a better guarantee of running `pip` in the
> Python interpreter you intended.  But that still requires one to know
> how to run the correct Python interpreter from the command-line (which
> the newbie double-clicking on IDLE may not even have a concept of...).
> 
> While I agree this is something that is important for beginners to
> learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for
> many newbies just to install one or two packages from pip, which they
> often might need/want to do for whatever educational pursuit they're
> following (heck, it's pretty common even just to want to install the
> `requests` module, as I would never throw `urllib` at a beginner).
> 
> So while I don't think anything proposed here will work technically, I
> am in favor of an in-interpreter pip install functionality.  Perhaps
> it could work something like this:
> 
> a) Allow it *only* in interactive mode:  running `pip(...)` (or
> whatever this looks like) outside of interactive mode raises a
> `RuntimeError` with the appropriate documentation
> b) When running `pip(...)` the user is supplied with an interactive
> prompt explaining that since installing packages with `pip()` can
> result in changes to the interpreter, it is necessary to restart the
> interpreter after installation--give them an opportunity to cancel the
> action in case they have any work they need to save.  If they proceed,
> install the new package then restart the interpreter for them.  This
> avoids any ambiguity as to states of loaded modules before/after pip
> install.
> 
> 
> 
> > From: Stephan Houben [mailto:stephan...@gmail.com]
> > Sent: Sunday, October 29, 2017 3:43 PM
> > To: Alex Walters 
> > Cc: Python-Ideas 
> > Subject: Re: [Python-ideas] install pip packages from Python prompt
> >
> >
> >
> > Hi Alex,
> >
> >
> >
> > 2017-10-29 20:26 GMT+01:00 Alex Walters :
> >
> > return “Please run pip from your system command prompt”
> >
> >
> >
> >
> >
> > The target audience for my proposal are people who do not know
> >
> > which part of the sheep the "system command prompt" is.
> >
> > Stephan
> >
> >
> >
> >
> >
> > From: Python-ideas
> > [mailto:python-ideas-bounces+tritium-list=sdamon@python.org] On
> Behalf
> > Of Stephan Houben
> > Sent: Sunday, October 29, 2017 3:19 PM
> > To: Python-Ideas 
> > Subject: [Python-ideas] install pip packages from Python prompt
> >
> >
> >
> > Hi all,
> >
> > Here is in somewhat more detail my earlier proposal for
> >
> > having in the interactive Python interpreter a `pip` function to
> >
> > install packages from Pypi.
> >
> > Motivation: it appears to me that there is a category of newbies
> >
> > for which "open a shell and do `pip whatever`" is a bit too much.
> >
> > It would, in my opinion, simplify things a bit if they could just
> >
> > copy-and-paste some text into the Python interpreter and have
> >
> > some packages from pip installed.
> >
> > That would simplify instructions on how to install package xyz,
> >
> > without going into the vagaries of how to open a shell on various
> >
> > pla

Re: [Python-ideas] Thread.__init__ should call super()

[Ilya Kulakov]

Neil, thank you for doing much better job explaining the problem.

Generally, I'm cool with Python's standard library classes not calling super(), 
as many of them
are not designed for subclassing. But those which are should do that. E.g. take 
a look at more
recent asyncio's Protocol and Transport classes: they all properly call super().

One potential problem is that it will break existing code:

class X(Thread, SomethingElse):
def __init__(self):
Thread.__init__(self)
SomethingElse.__init__(self)

SomethingElse.__init__ will be called twice.
Is it a good reason for "old" classes to lag behind? I don't know.

Perhaps some mechanism (invisible to a user) can be designed to avoid that.
E.g. super() may leave a flag which should signal interpreter to "skip" all 
direct calls
of a function and warn about it (DeprecationWarning?).

Best Regards,
Ilya Kulakov

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Juancarlo Añez]

On Mon, Oct 30, 2017 at 12:29 PM, Guido van Rossum  wrote:

> What's your proposed process to arrive at the list of recommended
> packages? And is it really just going to be a list of names, or is there
> going to be some documentation (about the vetting, not about the contents
> of the packages) for each name?
>

As I see it, the bootstrap would be a requirements.txt (requirements.pip)
with the packages that more experienced developers prefer to use over
stdlib, pinned to package versions known to work well with the CPython
release.

I think this is a great idea, specially if it's an easy opt-in.

For example, sometimes I go into a programming niche for a few years, and
I'd very much appreciate a curated list of packages when I move to another
niche. A pay-forward kind of thing.

The downside to the idea is that the list of packages will need a curator
(which could be python.org).
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] IMAP4.__exit__ counterintuitive for with blocks

[Drew]

Yeah, a flag for IMAP4 objects sounds like it'd solve backwards-compatibility 
problems. E.g.

imaplib.IMAP4_SSL(..., commit=manual/auto)

As for handling errors with automatic commits, we should probably just defer to 
whatever Python IO does to remain consistent.

⁣

On Oct 30, 2017, 1:33 PM, at 1:33 PM, Guido van Rossum  wrote:
>But maybe if __exit__ is called with an exception it should roll back.
>
>In any case it looks like your proposal could break existing code (e.g.
>code that uses `with` depending on its current behavior, using some
>other
>logic to decide whether to commit or not) and that feels difficult to
>overcome. Perhaps a new method or flag argument can be added to request
>that the transactions be automatically committed?
>
>On Mon, Oct 30, 2017 at 10:20 AM, Drew  wrote:
>
>> IMAP4.close closes the selected inbox and commits changes such as
>> deletions. It is not called on IMAP4.__exit__ (only logout is, which
>> doesn't call close in its call stack) however, so:
>>
>> with imaplib.IMAP4_SSL(...) as i:
>> ...
>>
>> would fail to commit those changes. close must be explicitly invoked
>i.e.
>>
>> with imaplib.IMAP4_SSL(...) as i:
>> ...
>> i.close()
>>
>> This is counterintuitive however, as the with statement is meant to
>> automatically clean up. Another programmer might come along and
>delete
>> i.close() because it seems unnecessary. Now changes aren't being
>committed
>> and the programmer doesn't realize it's because of this weird Python
>> idiosyncracy.
>>
>> Python IO such as open commits changes automatically, so I'm not sure
>why
>> IMAP4 doesn't and only logs out.
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
>
>--
>--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Stephan Houben]

What about something like the following to simulate a "restart", portably.

def restart():
import sys
import os
import subprocess
if os.getenv("PYTHON_EXIT_ON_RESTART") == "1":
sys.exit(42)
else:
env = os.environ.copy()
env["PYTHON_EXIT_ON_RESTART"] = "1"
while True:
sp = subprocess.run([sys.executable], env=env)
if sp.returncode != 42:
sys.exit(sp.returncode)

Stephan

2017-10-30 17:33 GMT+01:00 Paul Moore :

> On 30 October 2017 at 16:22, Nick Coghlan  wrote:
> >> Also, on Windows, I believe that any emulation of execve either leaves
> >> the original process in memory, or has problems getting console
> >> inheritance right. It's been a long time since I worked at that level,
> >> and things may be better now, but getting a robust "restart this
> >> process" interface in Windows would need some care (that's one of the
> >> reasons the py launcher runs Python as a subprocess rather than doing
> >> any sort of exec equivalent).
> >
> > As long as the standard streams are passed along correctly, whatever the
> py
> > launcher does would presumably be adequate for a REPL restart as well,
> > assuming we decided to go down that path.
>
> The py launcher starts a subprocess for python.exe and waits on it. I
> wouldn't have thought that's going to work for installing mods in a
> REPL - imagine a long working session where I install 10 mods as I
> explore options for a particular problem (I don't know how likely that
> is in practice...) - there'd be a chain of 10+ Python processes, only
> the last of which is still useful. It's probably not a massive problem
> (I assume everything but the last process is paged out) but it's not
> exactly friendly.
>
> OTOH, if you lose the command history and the interpreter state after
> each install, you'll probably get fed up long before the number of
> processes is an issue...
>
> > It would also be reasonable to say that the regular REPL just issues a
> > warning that a restart might be needed, and it's only REPLs with a
> separate
> > client process that offer a way to restart the subprocess where code
> > actually executes.
>
> This feels awfully like the traditional Windows "your mouse has moved
> - please reboot to have your changes take effect" behaviour. I don't
> think we're going to impress many people emulating that :-(
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] IMAP4.__exit__ counterintuitive for with blocks

[Guido van Rossum]

But maybe if __exit__ is called with an exception it should roll back.

In any case it looks like your proposal could break existing code (e.g.
code that uses `with` depending on its current behavior, using some other
logic to decide whether to commit or not) and that feels difficult to
overcome. Perhaps a new method or flag argument can be added to request
that the transactions be automatically committed?

On Mon, Oct 30, 2017 at 10:20 AM, Drew  wrote:

> IMAP4.close closes the selected inbox and commits changes such as
> deletions. It is not called on IMAP4.__exit__ (only logout is, which
> doesn't call close in its call stack) however, so:
>
> with imaplib.IMAP4_SSL(...) as i:
> ...
>
> would fail to commit those changes. close must be explicitly invoked i.e.
>
> with imaplib.IMAP4_SSL(...) as i:
> ...
> i.close()
>
> This is counterintuitive however, as the with statement is meant to
> automatically clean up. Another programmer might come along and delete
> i.close() because it seems unnecessary. Now changes aren't being committed
> and the programmer doesn't realize it's because of this weird Python
> idiosyncracy.
>
> Python IO such as open commits changes automatically, so I'm not sure why
> IMAP4 doesn't and only logs out.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Alexander Belopolsky]

On Mon, Oct 30, 2017 at 11:44 AM, Nick Coghlan  wrote:
..
> 3. We can't replicate it as readily in the regular REPL, since that runs
> Python code directly in the current process, but even there I believe we
> could potentially trigger a full process restart via execve (or the C++
> style _execve on Windows)

This exact problem is solved rather elegantly in Julia.  When you
upgrade a package that is already loaded in the REPL, it prints a
warning:

"The following packages have been updated but were already imported:
... Restart Julia to use the updated versions."

listing the affected packages.

See 
.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] IMAP4.__exit__ counterintuitive for with blocks

[Drew]

IMAP4.close closes the selected inbox and commits changes such as deletions. It 
is not called on IMAP4.__exit__ (only logout is, which doesn't call close in 
its call stack) however, so:

with imaplib.IMAP4_SSL(...) as i:
    ...

would fail to commit those changes. close must be explicitly invoked i.e.

with imaplib.IMAP4_SSL(...) as i:
    ...
    i.close()

This is counterintuitive however, as the with statement is meant to 
automatically clean up. Another programmer might come along and delete 
i.close() because it seems unnecessary. Now changes aren't being committed and 
the programmer doesn't realize it's because of this weird Python idiosyncracy.

Python IO such as open commits changes automatically, so I'm not sure why IMAP4 
doesn't and only logs out. ___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Paul Moore]

On 30 October 2017 at 16:22, Nick Coghlan  wrote:
>> Also, on Windows, I believe that any emulation of execve either leaves
>> the original process in memory, or has problems getting console
>> inheritance right. It's been a long time since I worked at that level,
>> and things may be better now, but getting a robust "restart this
>> process" interface in Windows would need some care (that's one of the
>> reasons the py launcher runs Python as a subprocess rather than doing
>> any sort of exec equivalent).
>
> As long as the standard streams are passed along correctly, whatever the py
> launcher does would presumably be adequate for a REPL restart as well,
> assuming we decided to go down that path.

The py launcher starts a subprocess for python.exe and waits on it. I
wouldn't have thought that's going to work for installing mods in a
REPL - imagine a long working session where I install 10 mods as I
explore options for a particular problem (I don't know how likely that
is in practice...) - there'd be a chain of 10+ Python processes, only
the last of which is still useful. It's probably not a massive problem
(I assume everything but the last process is paged out) but it's not
exactly friendly.

OTOH, if you lose the command history and the interpreter state after
each install, you'll probably get fed up long before the number of
processes is an issue...

> It would also be reasonable to say that the regular REPL just issues a
> warning that a restart might be needed, and it's only REPLs with a separate
> client process that offer a way to restart the subprocess where code
> actually executes.

This feels awfully like the traditional Windows "your mouse has moved
- please reboot to have your changes take effect" behaviour. I don't
think we're going to impress many people emulating that :-(

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Guido van Rossum]

What's your proposed process to arrive at the list of recommended packages?
And is it really just going to be a list of names, or is there going to be
some documentation (about the vetting, not about the contents of the
packages) for each name?

On Mon, Oct 30, 2017 at 9:08 AM, Nick Coghlan  wrote:

> On 31 October 2017 at 00:28, Guido van Rossum  wrote:
>
>> I just feel that when you're talking about an org like PayPal they can
>> take care of themselves and don't need our help. They will likely have
>> packages they want installed everywhere that would never make in on your
>> list. So it feels this whole discussion is a distraction and a waste of
>> time (yours, too).
>>
>
> Just because companies are big doesn't mean they necessarily have anyone
> internally that's already up to speed on the specifics of recommended
> practices in a sprawling open source community like Python's. The genesis
> of this idea is that I think we can offer a more consistent initial
> experience for those folks than "Here's PyPI and Google, y'all have fun
> now" (and in so doing, help folks writing books and online tutorials to
> feel more comfortable with the idea of assuming that libraries like
> requests will be available in even the most restrictive institutional
> environments that still allow the use of Python).
>
> One specific situation this idea is designed to help with is the one where:
>
> - there's a centrally managed Standard Operating Environment that dictates
> what gets installed
> - they've approved the python.org installers
> - they *haven't* approved anything else yet
>
> Now, a lot of large orgs simply won't get into that situation in the first
> place, since their own supplier management rules will push them towards a
> commercial redistributor, in which case they'll get their chosen
> redistributor's preferred package set, which will then typically cover at
> least a few hundred of the most popular PyPI packages.
>
> But some of them will start from the narrower "standard library only"
> baseline, and I spent enough time back at Boeing arguing for libraries to
> be added to our approved component list to appreciate the benefits of
> transitive declarations of trust ("we trust supplier X, they unambigously
> state their trust in supplier Y, so that's an additional point in favour of
> our also trusting supplier Y") when it comes time to make your case to your
> supplier management organisation. Such declarations still aren'y always
> sufficient, but they definitely don't hurt, and they sometimes help.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Antoine Pitrou]

On Tue, 31 Oct 2017 02:22:50 +1000
Nick Coghlan  wrote:
> On 31 October 2017 at 02:06, Paul Moore  wrote:
> 
> > On 30 October 2017 at 15:53, Antoine Pitrou  wrote:  
> > > On Tue, 31 Oct 2017 01:44:10 +1000
> > > Nick Coghlan  wrote:  
> > >> (We'd want a real process restart, rather than emulating it by calling
> > >> Py_Initialize & Py_Finalize multiple times, as not every module properly
> > >> supports multiple initialise/finalise cycles within a single process,  
> > and  
> > >> module-specific quirks are exactly what we'd be trying to avoid by  
> > forcing  
> > >> an interpreter restart)  
> > >
> > > The main difference, though, is that a notebook will reload and
> > > replay all your session, while restarting the regular REPL will simply
> > > lose all current work.  I think that makes the idea much less
> > > appealing.  
> >  
> 
> Right, but if you want an installation to work reliably, you're going to
> lose that state anyway.

You're going to lose the concrete state, but not the sequence of prompt
commands and expressions which led to that state, and which a notebook
makes trivial to replay (it's a bit like statement-based replication on
a database). The regular REPL would lose both.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Nick Coghlan]

On 31 October 2017 at 02:06, Paul Moore  wrote:

> On 30 October 2017 at 15:53, Antoine Pitrou  wrote:
> > On Tue, 31 Oct 2017 01:44:10 +1000
> > Nick Coghlan  wrote:
> >> (We'd want a real process restart, rather than emulating it by calling
> >> Py_Initialize & Py_Finalize multiple times, as not every module properly
> >> supports multiple initialise/finalise cycles within a single process,
> and
> >> module-specific quirks are exactly what we'd be trying to avoid by
> forcing
> >> an interpreter restart)
> >
> > The main difference, though, is that a notebook will reload and
> > replay all your session, while restarting the regular REPL will simply
> > lose all current work.  I think that makes the idea much less
> > appealing.
>

Right, but if you want an installation to work reliably, you're going to
lose that state anyway. Erik's original comment included the suggestion to
"give them an opportunity to cancel the
action in case they have any work they need to save", and I think some kind
of warning's going to be necessary no matter how we handle the restart.


> Also, on Windows, I believe that any emulation of execve either leaves
> the original process in memory, or has problems getting console
> inheritance right. It's been a long time since I worked at that level,
> and things may be better now, but getting a robust "restart this
> process" interface in Windows would need some care (that's one of the
> reasons the py launcher runs Python as a subprocess rather than doing
> any sort of exec equivalent).
>

As long as the standard streams are passed along correctly, whatever the py
launcher does would presumably be adequate for a REPL restart as well,
assuming we decided to go down that path.

It would also be reasonable to say that the regular REPL just issues a
warning that a restart might be needed, and it's only REPLs with a separate
client process that offer a way to restart the subprocess where code
actually executes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Nick Coghlan]

On 31 October 2017 at 00:28, Guido van Rossum  wrote:

> I just feel that when you're talking about an org like PayPal they can
> take care of themselves and don't need our help. They will likely have
> packages they want installed everywhere that would never make in on your
> list. So it feels this whole discussion is a distraction and a waste of
> time (yours, too).
>

Just because companies are big doesn't mean they necessarily have anyone
internally that's already up to speed on the specifics of recommended
practices in a sprawling open source community like Python's. The genesis
of this idea is that I think we can offer a more consistent initial
experience for those folks than "Here's PyPI and Google, y'all have fun
now" (and in so doing, help folks writing books and online tutorials to
feel more comfortable with the idea of assuming that libraries like
requests will be available in even the most restrictive institutional
environments that still allow the use of Python).

One specific situation this idea is designed to help with is the one where:

- there's a centrally managed Standard Operating Environment that dictates
what gets installed
- they've approved the python.org installers
- they *haven't* approved anything else yet

Now, a lot of large orgs simply won't get into that situation in the first
place, since their own supplier management rules will push them towards a
commercial redistributor, in which case they'll get their chosen
redistributor's preferred package set, which will then typically cover at
least a few hundred of the most popular PyPI packages.

But some of them will start from the narrower "standard library only"
baseline, and I spent enough time back at Boeing arguing for libraries to
be added to our approved component list to appreciate the benefits of
transitive declarations of trust ("we trust supplier X, they unambigously
state their trust in supplier Y, so that's an additional point in favour of
our also trusting supplier Y") when it comes time to make your case to your
supplier management organisation. Such declarations still aren'y always
sufficient, but they definitely don't hurt, and they sometimes help.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Paul Moore]

On 30 October 2017 at 15:53, Antoine Pitrou  wrote:
> On Tue, 31 Oct 2017 01:44:10 +1000
> Nick Coghlan  wrote:
>>
>> A few specific notes here:
>>
>> 1. As you say, this sort of already works in notebooks, since instructors
>> can say to run "!pip install requests" and then restart the language kernel.
>> 2. We could probably replicate that style in IDLE, since that runs user
>> code in a subprocess, similar to the way Jupyter language kernels are
>> separate from the frontend client
>> 3. We can't replicate it as readily in the regular REPL, since that runs
>> Python code directly in the current process, but even there I believe we
>> could potentially trigger a full process restart via execve (or the C++
>> style _execve on Windows)
>>
>> (We'd want a real process restart, rather than emulating it by calling
>> Py_Initialize & Py_Finalize multiple times, as not every module properly
>> supports multiple initialise/finalise cycles within a single process, and
>> module-specific quirks are exactly what we'd be trying to avoid by forcing
>> an interpreter restart)
>
> The main difference, though, is that a notebook will reload and
> replay all your session, while restarting the regular REPL will simply
> lose all current work.  I think that makes the idea much less
> appealing.

Also, on Windows, I believe that any emulation of execve either leaves
the original process in memory, or has problems getting console
inheritance right. It's been a long time since I worked at that level,
and things may be better now, but getting a robust "restart this
process" interface in Windows would need some care (that's one of the
reasons the py launcher runs Python as a subprocess rather than doing
any sort of exec equivalent).

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Antoine Pitrou]

On Tue, 31 Oct 2017 01:44:10 +1000
Nick Coghlan  wrote:
> 
> A few specific notes here:
> 
> 1. As you say, this sort of already works in notebooks, since instructors
> can say to run "!pip install requests" and then restart the language kernel.
> 2. We could probably replicate that style in IDLE, since that runs user
> code in a subprocess, similar to the way Jupyter language kernels are
> separate from the frontend client
> 3. We can't replicate it as readily in the regular REPL, since that runs
> Python code directly in the current process, but even there I believe we
> could potentially trigger a full process restart via execve (or the C++
> style _execve on Windows)
> 
> (We'd want a real process restart, rather than emulating it by calling
> Py_Initialize & Py_Finalize multiple times, as not every module properly
> supports multiple initialise/finalise cycles within a single process, and
> module-specific quirks are exactly what we'd be trying to avoid by forcing
> an interpreter restart)

The main difference, though, is that a notebook will reload and
replay all your session, while restarting the regular REPL will simply
lose all current work.  I think that makes the idea much less
appealing.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Nick Coghlan]

On 30 October 2017 at 20:35, Erik Bray  wrote:

> I should add--another case that is becoming extremely common is
> beginners learning Python for the first time inside the
> Jupyter/IPython Notebook.  And in my experience it can be very
> difficult for beginners to understand the connection between what's
> happening in the notebook ("it's in the web-browser--what does that
> have to do with anything on my computer??") and the underlying Python
> interpreter, file system, etc.  Being able to pip install from within
> the Notebook would be a big win.  This is already possible since
> IPython allows running system commands and it is possible to run the
> pip executable from the notebook, then manually restart the Jupyter
> kernel.
>
> It's not 100% clear to me how my proposal below would work within a
> Jupyter Notebook, so that would also be an angle worth looking into.
>

A few specific notes here:

1. As you say, this sort of already works in notebooks, since instructors
can say to run "!pip install requests" and then restart the language kernel.
2. We could probably replicate that style in IDLE, since that runs user
code in a subprocess, similar to the way Jupyter language kernels are
separate from the frontend client
3. We can't replicate it as readily in the regular REPL, since that runs
Python code directly in the current process, but even there I believe we
could potentially trigger a full process restart via execve (or the C++
style _execve on Windows)

(We'd want a real process restart, rather than emulating it by calling
Py_Initialize & Py_Finalize multiple times, as not every module properly
supports multiple initialise/finalise cycles within a single process, and
module-specific quirks are exactly what we'd be trying to avoid by forcing
an interpreter restart)

So the main missing piece if we went down that path would be to offer a way
to say from within the interpreter itself "Restart the current interactive
session". One possible approach to that would be to define a
RestartInterpreter subclass of SystemExit, which the interpreter would
intercept at around the same point where it checks for the PYTHONINSPECT
flag, and then initiate a graceful process shutdown and restart, rather
than a normal exit. We'd probably want that capability to be off by default
and enable it explicitly from the CPython CLI though, as otherwise it could
have some really annoying side effects in runtime embedding use cases.

I'm sure there'd be some thorny edge cases that would arise in trying to
make this work in practice, but at first glance, the general idea sounds
potentially feasible to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add single() to itertools

[Guido van Rossum]

This is a key example of a case where code speaks. Can you write an
implementation of how you would want single() to work in Python code?

On Mon, Oct 30, 2017 at 2:49 AM, Ivan Pozdeev via Python-ideas <
python-ideas@python.org> wrote:

>
>
> On 30.10.2017 9:29, python-ideas-requ...@python.org wrote:
>
>> If I have understood your use-case, you have a function that returns a
>> list of results (or possibly an iterator, or a tuple, or some other
>> sequence):
>>
>>  print(search(haystack, needle))
>>  # prints ['bronze needle', 'gold needle', 'silver needle']
>>
>> There are times you expect there to be a single result, and if there are
>> multiple results, that is considered an error. Am I correct so far?
>>
> Correct.
>
>> If so, then sequence unpacking is your friend:
>>
>>  result, = search(haystack, needle)
>>
>> <...>
>>
>> I *think* this will solve your problem.
>>
>> If not, can you please explain what "single()" is supposed to do, why it
>> belongs in itertools, and show an example of how it will work.
>>
> That works. Too arcane in my book though (and others' too according to
> https://stackoverflow.com/a/473337/648265), and the error messages are
> cryptic in this use case.
> It also cannot be a part of an expression, unlike next().
>
> The initial post on the above link summarizes the suggested implementation
> pretty well.
>
> --
> Regards,
> Ivan
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Guido van Rossum]

I just feel that when you're talking about an org like PayPal they can take
care of themselves and don't need our help. They will likely have packages
they want installed everywhere that would never make in on your list. So it
feels this whole discussion is a distraction and a waste of time (yours,
too).

On Sun, Oct 29, 2017 at 10:24 PM, Nick Coghlan  wrote:

> On 30 October 2017 at 04:56, Guido van Rossum  wrote:
>
>> The two use cases you describe (scripters and teachers) leave me
>> luke-warm -- scripters live in the wild west and can just pip install
>> whatever (that's what it means to be scripting)
>>
>
> For personal scripting, we can install whatever, but "institutional
> scripting" isn't the same thing - there we're scripting predefined
> "Standard Operating Environments", like those Mahmoud Hashemi describes for
> PayPal at the start of https://www.paypal-engineering.com/2016/09/07/
> python-packaging-at-paypal/
>
> "Just use PyInstaller" isn't an adequate answer in large-scale
> environments because of the "zlib problem": you don't want to have to
> rebuild and redeploy the world to handle a security update in a low level
> frequently used component, you want to just update that component and have
> everything else pick it up dynamically.
>
> While "Just use conda" is excellent advice nowadays for any organisation
> contemplating defining their own bespoke Python SOE (hence Mahmoud's post),
> if that isn't being driven by the folks that already maintain the SOE (as
> happened in PayPal's case) convincing an org to add a handful of python-dev
> endorsed libraries to an established SOE is going to be easier than
> rebasing their entire Python SOE on conda.
>
>
>> and teachers tend to want a customized bundle anyway -- let the edu world
>> get together and create their own recommended bundle.
>>
>> As long as it's not going to be bundled, i.e. there's just going to be
>> some list of packages that we recommend to 3rd party repackagers, then I'm
>> fine with it. But they must remain clearly marked as 3rd party packages in
>> whatever docs we provide, and live in site-packages.
>>
>
> Yep, that was my intent, although it may not have been clear in my initial
> proposal. I've filed two separate RFEs in relation to that:
>
> * Documentation only: https://bugs.python.org/issue31898
> * Regression testing resource: https://bugs.python.org/issue31899
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add processor generation to wheel metadata

[Ivan Pozdeev via Python-ideas]

Generally, packages are compiled for the same processor generation as 
the corresponding Python.
But not always -- e.g. NumPy opted for SSE2 even for Py2 to work around 
some compiler bug

(https://github.com/numpy/numpy/issues/6428).
I was bitten by that at an old machine once and found out that there is 
no way for `pip' to have checked for that.
Besides, performance-oriented packages like the one mentioned could 
probably benefit from newer instructions.


Regarding identifiers:
gcc, cl and clang all have their private directories of generation 
identifiers:

https://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/i386-and-x86_002d64-Options.html
https://msdn.microsoft.com/en-us/library/7t5yh4fd.aspx
https://clang.llvm.org/doxygen/Driver_2ToolChains_2Arch_2X86_8cpp_source.html

Linux packages typically use gcc's ones. Clang generally follows in 
gcc's footsteps and accepts cl's IDs, too, as aliases.


So, using the IDs of whatever compiler is used to build the package 
(i.e. most likely the canonical compiler for CPython for that platform) 
looks like the simple&stupid(r) way - we can just take the value of the 
"march" argument.



The tricky part is mapping the system's processor to an ID when checking 
compatibility: the logic will have to keep a directory (that's the job 
of `wheel' package maintainers though, I guess).
I also guess that there are cases where there's no such thing as _the_ 
system's processor.


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Erik Bray]

On Mon, Oct 30, 2017 at 11:27 AM, Erik Bray  wrote:
> On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters  wrote:
>> Then those users have more fundamental problems.  There is a minimum level
>> of computer knowledge needed to be successful in programming.  Insulating
>> users from the reality of the situation is not preparing them to be
>> successful.  Pretending that there is no system command prompt, or shell, or
>> whatever platform specific term applies, only hurts new programmers.  Give
>> users an error message they can google, and they will be better off in the
>> long run than they would be if we just ran pip for them.
>
> While I completely agree with this in principle, I think you
> overestimate the average beginner.  Many beginners I've taught or
> helped, even if they can manage to get to the correct command prompt,
> often don't even know how to run the correct Python.  They might often
> have multiple Pythons installed on their system--maybe they have
> Anaconda, maybe Python installed by homebrew, or a Python that came
> with an IDE like Spyder.  If they're on OSX often running "python"
> from the command prompt gives the system's crippled Python 2.6 and
> they don't know the difference.


I should add--another case that is becoming extremely common is
beginners learning Python for the first time inside the
Jupyter/IPython Notebook.  And in my experience it can be very
difficult for beginners to understand the connection between what's
happening in the notebook ("it's in the web-browser--what does that
have to do with anything on my computer??") and the underlying Python
interpreter, file system, etc.  Being able to pip install from within
the Notebook would be a big win.  This is already possible since
IPython allows running system commands and it is possible to run the
pip executable from the notebook, then manually restart the Jupyter
kernel.

It's not 100% clear to me how my proposal below would work within a
Jupyter Notebook, so that would also be an angle worth looking into.

Best,
Erik


> One thing that has been a step in the right direction is moving more
> documentation toward preferring running `python -m pip` over just
> `pip`, since this often has a better guarantee of running `pip` in the
> Python interpreter you intended.  But that still requires one to know
> how to run the correct Python interpreter from the command-line (which
> the newbie double-clicking on IDLE may not even have a concept of...).
>
> While I agree this is something that is important for beginners to
> learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for
> many newbies just to install one or two packages from pip, which they
> often might need/want to do for whatever educational pursuit they're
> following (heck, it's pretty common even just to want to install the
> `requests` module, as I would never throw `urllib` at a beginner).
>
> So while I don't think anything proposed here will work technically, I
> am in favor of an in-interpreter pip install functionality.  Perhaps
> it could work something like this:
>
> a) Allow it *only* in interactive mode:  running `pip(...)` (or
> whatever this looks like) outside of interactive mode raises a
> `RuntimeError` with the appropriate documentation
> b) When running `pip(...)` the user is supplied with an interactive
> prompt explaining that since installing packages with `pip()` can
> result in changes to the interpreter, it is necessary to restart the
> interpreter after installation--give them an opportunity to cancel the
> action in case they have any work they need to save.  If they proceed,
> install the new package then restart the interpreter for them.  This
> avoids any ambiguity as to states of loaded modules before/after pip
> install.
>
>
>
>> From: Stephan Houben [mailto:stephan...@gmail.com]
>> Sent: Sunday, October 29, 2017 3:43 PM
>> To: Alex Walters 
>> Cc: Python-Ideas 
>> Subject: Re: [Python-ideas] install pip packages from Python prompt
>>
>>
>>
>> Hi Alex,
>>
>>
>>
>> 2017-10-29 20:26 GMT+01:00 Alex Walters :
>>
>> return “Please run pip from your system command prompt”
>>
>>
>>
>>
>>
>> The target audience for my proposal are people who do not know
>>
>> which part of the sheep the "system command prompt" is.
>>
>> Stephan
>>
>>
>>
>>
>>
>> From: Python-ideas
>> [mailto:python-ideas-bounces+tritium-list=sdamon@python.org] On Behalf
>> Of Stephan Houben
>> Sent: Sunday, October 29, 2017 3:19 PM
>> To: Python-Ideas 
>> Subject: [Python-ideas] install pip packages from Python prompt
>>
>>
>>
>> Hi all,
>>
>> Here is in somewhat more detail my earlier proposal for
>>
>> having in the interactive Python interpreter a `pip` function to
>>
>> install packages from Pypi.
>>
>> Motivation: it appears to me that there is a category of newbies
>>
>> for which "open a shell and do `pip whatever`" is a bit too much.
>>
>> It would, in my opinion, simplify things a bit if they could just
>>
>> copy-and-paste some text into the P

Re: [Python-ideas] install pip packages from Python prompt

[Erik Bray]

On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters  wrote:
> Then those users have more fundamental problems.  There is a minimum level
> of computer knowledge needed to be successful in programming.  Insulating
> users from the reality of the situation is not preparing them to be
> successful.  Pretending that there is no system command prompt, or shell, or
> whatever platform specific term applies, only hurts new programmers.  Give
> users an error message they can google, and they will be better off in the
> long run than they would be if we just ran pip for them.

While I completely agree with this in principle, I think you
overestimate the average beginner.  Many beginners I've taught or
helped, even if they can manage to get to the correct command prompt,
often don't even know how to run the correct Python.  They might often
have multiple Pythons installed on their system--maybe they have
Anaconda, maybe Python installed by homebrew, or a Python that came
with an IDE like Spyder.  If they're on OSX often running "python"
from the command prompt gives the system's crippled Python 2.6 and
they don't know the difference.

One thing that has been a step in the right direction is moving more
documentation toward preferring running `python -m pip` over just
`pip`, since this often has a better guarantee of running `pip` in the
Python interpreter you intended.  But that still requires one to know
how to run the correct Python interpreter from the command-line (which
the newbie double-clicking on IDLE may not even have a concept of...).

While I agree this is something that is important for beginners to
learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for
many newbies just to install one or two packages from pip, which they
often might need/want to do for whatever educational pursuit they're
following (heck, it's pretty common even just to want to install the
`requests` module, as I would never throw `urllib` at a beginner).

So while I don't think anything proposed here will work technically, I
am in favor of an in-interpreter pip install functionality.  Perhaps
it could work something like this:

a) Allow it *only* in interactive mode:  running `pip(...)` (or
whatever this looks like) outside of interactive mode raises a
`RuntimeError` with the appropriate documentation
b) When running `pip(...)` the user is supplied with an interactive
prompt explaining that since installing packages with `pip()` can
result in changes to the interpreter, it is necessary to restart the
interpreter after installation--give them an opportunity to cancel the
action in case they have any work they need to save.  If they proceed,
install the new package then restart the interpreter for them.  This
avoids any ambiguity as to states of loaded modules before/after pip
install.



> From: Stephan Houben [mailto:stephan...@gmail.com]
> Sent: Sunday, October 29, 2017 3:43 PM
> To: Alex Walters 
> Cc: Python-Ideas 
> Subject: Re: [Python-ideas] install pip packages from Python prompt
>
>
>
> Hi Alex,
>
>
>
> 2017-10-29 20:26 GMT+01:00 Alex Walters :
>
> return “Please run pip from your system command prompt”
>
>
>
>
>
> The target audience for my proposal are people who do not know
>
> which part of the sheep the "system command prompt" is.
>
> Stephan
>
>
>
>
>
> From: Python-ideas
> [mailto:python-ideas-bounces+tritium-list=sdamon@python.org] On Behalf
> Of Stephan Houben
> Sent: Sunday, October 29, 2017 3:19 PM
> To: Python-Ideas 
> Subject: [Python-ideas] install pip packages from Python prompt
>
>
>
> Hi all,
>
> Here is in somewhat more detail my earlier proposal for
>
> having in the interactive Python interpreter a `pip` function to
>
> install packages from Pypi.
>
> Motivation: it appears to me that there is a category of newbies
>
> for which "open a shell and do `pip whatever`" is a bit too much.
>
> It would, in my opinion, simplify things a bit if they could just
>
> copy-and-paste some text into the Python interpreter and have
>
> some packages from pip installed.
>
> That would simplify instructions on how to install package xyz,
>
> without going into the vagaries of how to open a shell on various
>
> platforms, and how to get to the right pip executable.
>
> I think this could be as simple as:
>
>   def pip(args):
>   import sys
>   import subprocess
>   subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>
>   print("Please re-start Python now to use installed or upgraded
> packages.")
>
> Note that I added the final message about restarting the interpreter
>
> as a low-tech solution to the problem of packages being already
>
> imported in the current Python session.
>
> I would imagine that the author of package xyz would then put on
>
> their webpage something like:
>
>   To use, enter in your Python interpreter:
>
>  pip("install xyz --user")
>
> As another example, consider prof. Baldwin from Woolamaloo university
>
> who teaches a course "Introductory

Re: [Python-ideas] Add single() to itertools

[Ivan Pozdeev via Python-ideas]

On 30.10.2017 9:29, python-ideas-requ...@python.org wrote:

If I have understood your use-case, you have a function that returns a
list of results (or possibly an iterator, or a tuple, or some other
sequence):

 print(search(haystack, needle))
 # prints ['bronze needle', 'gold needle', 'silver needle']

There are times you expect there to be a single result, and if there are
multiple results, that is considered an error. Am I correct so far?

Correct.

If so, then sequence unpacking is your friend:

 result, = search(haystack, needle)

<...>

I *think* this will solve your problem.

If not, can you please explain what "single()" is supposed to do, why it
belongs in itertools, and show an example of how it will work.
That works. Too arcane in my book though (and others' too according to 
https://stackoverflow.com/a/473337/648265), and the error messages are 
cryptic in this use case.

It also cannot be a part of an expression, unlike next().

The initial post on the above link summarizes the suggested 
implementation pretty well.


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add single() to itertools

[Stéfane Fermigier]

IIUC, this would be similar to "first" ( https://pypi.python.org/pypi/first/
) but would raise exception in case the iterable returns more than one (or
less than one) element.

Would also be similar to one() in SQLAlchemy queries (
http://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.one
).

Regards,

  S.

On Mon, Oct 30, 2017 at 5:51 AM, Steven D'Aprano 
wrote:

> On Mon, Oct 30, 2017 at 07:14:10AM +0300, Ivan Pozdeev via Python-ideas
> wrote:
>
> > The eponymous C#'s LINQ method, I found very useful in the following,
> > quite recurring use-case:
>
> If I have understood your use-case, you have a function that returns a
> list of results (or possibly an iterator, or a tuple, or some other
> sequence):
>
> print(search(haystack, needle))
> # prints ['bronze needle', 'gold needle', 'silver needle']
>
> There are times you expect there to be a single result, and if there are
> multiple results, that is considered an error. Am I correct so far?
>
> If so, then sequence unpacking is your friend:
>
> result, = search(haystack, needle)
>
> Note the comma after the variable name on the left-hand side of the
> assignment. That's a special case of Python's more general sequence
> unpacking:
>
> a, b, c = [100, 200, 300]
>
> assigns a = 100, b = 200, c == 300. Using a single item is valid:
>
> py> result, = [100]
> py> print(result)
> 100
>
>
> but if the right-hand side has more than one item, you get an exception:
>
> py> result, = [100, 200, 300]
> Traceback (most recent call last):
>   File "", line 1, in 
> ValueError: too many values to unpack (expected 1)
>
>
> I *think* this will solve your problem.
>
> If not, can you please explain what "single()" is supposed to do, why it
> belongs in itertools, and show an example of how it will work.
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Chairman, Free&OSS Group / Systematic Cluster -
http://www.gt-logiciel-libre.org/
Co-Chairman, National Council for Free & Open Source Software (CNLL) -
http://cnll.fr/
Founder & Organiser, PyData Paris - http://pydata.fr/
---
“You never change things by fighting the existing reality. To change
something, build a new model that makes the existing model obsolete.” — R.
Buckminster Fuller
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Composition over Inheritance

[Thomas Jollans]

On 29/10/17 19:25, Soni L. wrote:
> 
> 
> On 2017-10-29 02:57 PM, Brendan Barnwell wrote:
>> On 2017-10-29 04:44, Soni L. wrote:
>>> And this is how you miss the whole point of being able to dynamically
>>> add/remove arbitrary components on objects you didn't create, at
>>> runtime.
>>>
>>> Someone gave me this code and told me it explains what I'm trying to do:
>>> https://repl.it/NYCF/3
>>>
>>> class T:
>>>  pass
>>>
>>> class C:
>>>  pass
>>>
>>> c = C()
>>>
>>> #c.[T] = 1
>>> c.__dict__[T] = 1
>>
>> Again, can you please explain why you want to write c.[T]? What do
>> you intend that to *do*?  Your commented line seems to indicate you
>> want it to do what `c.__dict__[T]` does, but you can already do that
>> with `setattr(c, T, 1)`.  Or you can just give c an attribute that's a
>> dict, but has an easier-to-type name than __dict__, so you can do
>> `c.mydict[T]`.  What is the specific advantage of `c.[T]` over these
>> existing solutions?
>>
> 
> Hmm... Why can't we just allow empty identifiers, and set a default
> handler for empty identifiers that implements the proposed ECS?

It sounds to me like the general shape of what you're proposing is
already entirely possible, just without the idiosyncratic syntax. You
could write a library that adds support for your components using some
other syntax without any additional language support.

I don't know, something like this should be doable:

@componentlib.has_components
class Car:
# ...

class Engine(componentlib.Component): # might have to be a metaclass?
def stall(self, car):
raise UnpleasantNoise()
# ...

car = Car()
car.engine = Engine()
car.engine.stall()
# and so on.

If you prefer square brackets, you can implement it with __getitem__
syntax instead of __getattr__ syntax.

The point is: not only does your proposal not *need* additional language
support; I'm not at all convinced that it would benefit from additional
language support.

> 
> But the basic idea is to indicate something at the call site, namely
> that T is a contract and the object returned should respect that
> contract and any function calls should pass the original object as an
> argument. (I personally don't like how Python treats o.m() (has self)
> the same as o.f() (no self) syntax-wise, either.)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Composition over Inheritance

[Wes Turner]

On Monday, October 30, 2017, Wes Turner  wrote:

> ... But interfaces are clunky and traits are lightweight, and this isn't
> Go, so we can't just create a class as a namespace full of @staticmethods
> which accept the relevant object references.
>
> * __setattribute__ -> __getitem__, __setitem__
>
> On Monday, October 30, 2017, Wes Turner  > wrote:
>
>>
>>
>> On Sunday, October 29, 2017, Nick Coghlan  wrote:
>>
>>> On 29 October 2017 at 12:25, Brendan Barnwell 
>>> wrote:
>>>
 On 2017-10-28 19:13, Soni L. wrote:

> And to have all cars have engines, you'd do:
>
> class Car:
> def __init__(self, ???):
>   self[Engine] = GasEngine()
>
> car = Car()
> car[Engine].kickstart() # kickstart gets the car as second argument.
>
> And if you can't do that, then you can't yet do what I'm proposing, and
> thus the proposal makes sense, even if it still needs some refining...
>

 As near as I can tell you can indeed do that, although it's
 still not clear to me why you'd want to.  You can give Car a __getitem__
 that on-the-fly generates an Engine object that knows which Car it is
 attached to, and then you can make Engine.kickstart a descriptor that knows
 which Engine it is attached to, and from that can figure out which Car it
 is attached to.

>>>
>>> Right, I think a few different things are getting confused here related
>>> to how different folks use composition.
>>>
>>> For most data modeling use cases, the composition model you want is
>>> either a tree or an acyclic graph, where the subcomponents don't know
>>> anything about the whole that they're a part of. This gives you good
>>> component isolation, and avoids circular dependencies.
>>>
>>> However, for other cases, you *do* want the child object to be aware of
>>> the parent - XML etrees are a classic example of this, where we want to
>>> allow navigation back up the tree, so each node gains a reference to its
>>> parent node. This often takes the form of a combination of delegation
>>> (parent->child references) and dependency inversion (child->parent
>>> reference).
>>>
>>
>> This is Java-y and maybe not opcode optimizable, but maybe there's a case
>> for defining __setattribute__ so that square brackets denote Rust-like
>> traits:
>>
>> https://docs.spring.io/spring-python/1.2.x/sphinx/html/objec
>> ts-pythonconfig.html#object-definition-inheritance
>>
>>   @Object(parent="request")
>>   def request_dev(self, req=None):
>>
>>   > Observe that in the following example the child definitions must
>> define an optional ‘req’ argument; in runtime they will be passed its value
>> basing on what their parent object will return.
>>
>> It's testable, but confusing to Java programmers who aren't familiar with
>> why Guice forces the patterns that it does:
>>
>> https://docs.spring.io/spring-python/1.2.x/sphinx/html/objec
>> ts-more.html#testable-code
>>
>> https://github.com/google/guice/wiki/Motivation#dependency-injection
>>
>> > Like the factory, dependency injection is just a design pattern. The
>> core principle is to separate behaviour from dependency resolution. In our
>> example, the RealBillingService is not responsible for looking up the
>> TransactionLog and CreditCardProcessor. Instead, they're passed in as
>> constructor parameters:
>>
>> When these are constructor parameters, we don't need to monkeypatch attrs
>> in order to write tests; which, IIUC, is also partly why you'd want
>> traits/mixins with the proposed special Rust-like syntax:
>>
>> https://docs.pytest.org/en/latest/monkeypatch.html
>>
>> https://docs.pytest.org/en/latest/fixture.html#modularity-
>> using-fixtures-from-a-fixture-function (this is too magic(), too)
>>
>> But you want dynamic mixins that have an upward reference and Rust-like
>> syntax (and no factories).
>>
>>
>>> For the car/engine example, this relates to explicitly modeling the
>>> relationship whereby a car can have one or more engines
>>>
>>
>> class MultiEngine():
>> zope.interface.implements(IEngine)
>>
>> https://zopeinterface.readthedocs.io/en/latest/README.html#
>> declaring-implemented-interfaces
>>
>> But interfaces aren't yet justified because it's only a few lines and
>> those are just documentation or a too-complex adapter registry dict, anyway.
>>
>>
>>>  (but the engine may not currently be installed),
>>>
>>
>> So it should default to a MockEngine which also implements(IEngine) and
>> raises NotImplementedError
>>
>>
>>>
>>>  while an engine can be installed in at most one car at any given point
>>> in time.
>>>
>>
>> But the refcounts would be too difficult
>>
>> This:
>>
>>
>>>
>>> You don't even need the descriptor protocol for that though, you just
>>> need the subcomponent to accept the parent reference as a constructor
>>> parameter:
>>>
>>> class Car:
>>>   def __init__(self, engine_type):
>>> self.engine = engine_type(self)
>>>
>>> However, this form of explicit dependency