Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Nick Coghlan]

On 30 October 2017 at 04:56, Guido van Rossum  wrote:

> The two use cases you describe (scripters and teachers) leave me luke-warm
> -- scripters live in the wild west and can just pip install whatever
> (that's what it means to be scripting)
>

For personal scripting, we can install whatever, but "institutional
scripting" isn't the same thing - there we're scripting predefined
"Standard Operating Environments", like those Mahmoud Hashemi describes for
PayPal at the start of
https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/

"Just use PyInstaller" isn't an adequate answer in large-scale environments
because of the "zlib problem": you don't want to have to rebuild and
redeploy the world to handle a security update in a low level frequently
used component, you want to just update that component and have everything
else pick it up dynamically.

While "Just use conda" is excellent advice nowadays for any organisation
contemplating defining their own bespoke Python SOE (hence Mahmoud's post),
if that isn't being driven by the folks that already maintain the SOE (as
happened in PayPal's case) convincing an org to add a handful of python-dev
endorsed libraries to an established SOE is going to be easier than
rebasing their entire Python SOE on conda.


> and teachers tend to want a customized bundle anyway -- let the edu world
> get together and create their own recommended bundle.
>
> As long as it's not going to be bundled, i.e. there's just going to be
> some list of packages that we recommend to 3rd party repackagers, then I'm
> fine with it. But they must remain clearly marked as 3rd party packages in
> whatever docs we provide, and live in site-packages.
>

Yep, that was my intent, although it may not have been clear in my initial
proposal. I've filed two separate RFEs in relation to that:

* Documentation only: https://bugs.python.org/issue31898
* Regression testing resource: https://bugs.python.org/issue31899

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add single() to itertools

[Steven D'Aprano]

On Mon, Oct 30, 2017 at 07:14:10AM +0300, Ivan Pozdeev via Python-ideas wrote:

> The eponymous C#'s LINQ method, I found very useful in the following, 
> quite recurring use-case:

If I have understood your use-case, you have a function that returns a 
list of results (or possibly an iterator, or a tuple, or some other 
sequence):

print(search(haystack, needle))
# prints ['bronze needle', 'gold needle', 'silver needle']

There are times you expect there to be a single result, and if there are 
multiple results, that is considered an error. Am I correct so far?

If so, then sequence unpacking is your friend:

result, = search(haystack, needle)

Note the comma after the variable name on the left-hand side of the 
assignment. That's a special case of Python's more general sequence 
unpacking:

a, b, c = [100, 200, 300]

assigns a = 100, b = 200, c == 300. Using a single item is valid:

py> result, = [100]
py> print(result)
100


but if the right-hand side has more than one item, you get an exception:

py> result, = [100, 200, 300]
Traceback (most recent call last):
  File "", line 1, in 
ValueError: too many values to unpack (expected 1)


I *think* this will solve your problem.

If not, can you please explain what "single()" is supposed to do, why it 
belongs in itertools, and show an example of how it will work.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add single() to itertools

[Ivan Pozdeev via Python-ideas]

The eponymous C#'s LINQ method, I found very useful in the following, 
quite recurring use-case:


I need to get a specific element from a data structure that only 
supports search semantics (i.e. returns a sequence/iterator of results).
For that, I specify very precise search criteria, so only that one item 
is supposed to be found. But I'd rather verify that just in case.


"A data structure that only supports search semantics" is a recurring 
phenomenon due to this:


I make a special-purpose data structure (usually for some 
domain-specific data like task specification or data directory) using a 
combination of existing and/or new containers. Now, these types do not 
enforce all the integrity constraints of my data. And I don't wish to 
construct a special-purpose class, complete with validation procedures, 
and pass records into it one by one etc -- when I can just write an 
initializer, loading all the data at once in a nicely readable 
construct, and call it a day.


So, when querying this structure, I "know" that there should only be one 
item satisfying a certain criteria - if there's more, or less, something 
is wrong with the data.


https://stackoverflow.com/questions/46009985/get-contentcontrol-by-title-or-tag 
is the most recent occasion where I had this use case (that is not 
Python but the concept is language-agnostic).


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Composition over Inheritance

[Nick Coghlan]

On 29 October 2017 at 21:44, Soni L.  wrote:

> ORMs use this kind of descriptor based composition management extensively
> in order to reliably model database foreign key relationships in a way
> that's mostly transparent to users of the ORM classes.
>
>
> And this is how you miss the whole point of being able to dynamically
> add/remove arbitrary components on objects you didn't create, at runtime.
>

You can already do that by adding new properties to classes
post-definition, or by changing __class__ to refer to a different type, or
by wrapping objects in transparent proxy types the way wrapt does.

We *allow* that kind of thing, because it's sometimes beneficial in order
to get two libraries to play nicely together at runtime without having to
patch one or the other. However, it's a last resort option that you use
when you've exhausted the other more maintainable alternatives, not
something we actually want to encourage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Paul Moore]

On 29 October 2017 at 20:44, Alex Walters  wrote:
> Writing scripts for non-developers, in an unmanaged environment (IT cant
> push a python install to the system) on windows means running pyinstaller
> et. al., on your script, if it has dependencies or not.  Its not worth it to
> walk someone through a python install to run a script, let alone installing
> optional dependencies.

Let's just say "not in the environments I work in", and leave it at that.

> Not to sound too crass, but there are only so many edge cases
> that a third party platform developer can be expected to care about (enough
> to bend over backwards to support).

I never suggested otherwise. I just wanted to point out that
"scripting in Python" covers a wider range of use cases than "can use
pip install". I'm not asking python-dev to support those use cases
(heck, I'm part of python-dev and *I* don't want to bend over
backwards to support them), but I do ask that people be careful not to
dismiss a group of users who are commonly under-represented in open
source mailing lists and communities.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Alex Walters]

Writing scripts for non-developers, in an unmanaged environment (IT cant
push a python install to the system) on windows means running pyinstaller
et. al., on your script, if it has dependencies or not.  Its not worth it to
walk someone through a python install to run a script, let alone installing
optional dependencies.  For linux, you are not limited to the standard
library, but what the distros have packaged - its easy enough to tell a
typical non-developer linux user "apt install python python-twisted, then
run the script".

For deployment to restricted environments, optional dependencies suggested
by python-dev is unlikely to be installed either.  Not that this matters, or
would matter for years, because IME those environments have ancient versions
of python that the developer is restricted to anyways.

And environments with limited internet access ... Bandersnatch on a portable
hard drive.  Not to sound too crass, but there are only so many edge cases
that a third party platform developer can be expected to care about (enough
to bend over backwards to support).

Putting a curated list of "These are good packages that solve a lot of
problems that the stdlib doesn't or makes complicated" on python.org is a
great idea.  Building tooling to make those part of a default installation
is not.

> -Original Message-
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Paul Moore
> Sent: Sunday, October 29, 2017 4:26 PM
> To: Guido van Rossum 
> Cc: python-ideas@python.org
> Subject: Re: [Python-ideas] Defining an easily installable "Recommended
> baseline package set"
> 
> On 29 October 2017 at 18:56, Guido van Rossum  wrote:
> > The two use cases you describe (scripters and teachers) leave me luke-
> warm
> > -- scripters live in the wild west and can just pip install whatever
(that's
> > what it means to be scripting)
> 
> In my experience, "scripting" *does* include people for whom "being in
> a standard Python distribution" is a requirement.
> 
> Situations I have encountered:
> 
> 1. Servers where developers need to write administrative or monitoring
> scripts, but they don't control what's on that server. The Python
> installation that comes by default is all that's available.
> 2. Developers working in environments with limited internet access.
> For example, my employer has a Windows (NTLM) proxy that pip can't
> work with. Getting internet access for pip involves installing a
> separate application, and that's not always possible/desirable.
> 3. Developers writing scripts to be shared with non-developers. On
> Unix, this means "must work with the system Python", and on Windows
> "download and install Python from python.org" is typically all you can
> expect (although in that case "install Anaconda" is a possible
> alternative, although not one I've tried telling people to do myself).
> 
> Nick's proposal doesn't actually help for (1) or (2), as the problem
> there is that "pip install" won't work. And bundling a script with its
> (pure Python) dependencies, for example as a zipapp, is always a
> solution - although it's nowhere near as easy as simply copying a
> single-file script to the destination where it's to be run. So these
> situations don't actually matter in terms of the value of the
> proposals being discussed here. But I did want to dispute the idea
> that "scripters can just pip install whatever" is inherent to the idea
> of being a scripter - my experience is the opposite, that scripters
> are the people *least* able to simply pip install things.
> 
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Paul Moore]

On 29 October 2017 at 19:40, Stephan Houben  wrote:
> Hi Antoine,
>
> 2017-10-29 20:31 GMT+01:00 Antoine Rozo :
>>
>> Hi,
>>
>> What would be the difference with current pip module?
>> pip.main(['install', 'some_package'])
>
>
>
> My understanding is that direct use of the `pip` module is explicitly not
> recommended.

Not only not recommended, but explicitly not supported. And it won't
be available at all in pip 10.

Having said that, I'm -1 on this proposal. Installing new modules (or
upgrading existing ones) in a running Python process has all sorts of
subtle issues - the need to reload already-loaded modules, the fact
that failed imports may have resulted in different code paths being
taken ("except ImportError"), etc. Exiting the Python process to run
pi avoids all of these.

For someone who doesn't understand the difference between the Python
REPL and the command prompt, offering an interface that exposes them
to these sort of potential issues won't actually help them. Better to
teach them the basics of what a command line is, and what an
interactive Python prompt is, before exposing them to subtleties like
this.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Stephan Houben]

Hi Alex,

2017-10-29 20:26 GMT+01:00 Alex Walters :

> return “Please run pip from your system command prompt”
>
>
>

The target audience for my proposal are people who do not know
which part of the sheep the "system command prompt" is.

Stephan


>
>
>
>
> *From:* Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@
> python.org] *On Behalf Of *Stephan Houben
> *Sent:* Sunday, October 29, 2017 3:19 PM
> *To:* Python-Ideas 
> *Subject:* [Python-ideas] install pip packages from Python prompt
>
>
>
> Hi all,
>
> Here is in somewhat more detail my earlier proposal for
>
> having in the interactive Python interpreter a `pip` function to
>
> install packages from Pypi.
>
> Motivation: it appears to me that there is a category of newbies
>
> for which "open a shell and do `pip whatever`" is a bit too much.
>
> It would, in my opinion, simplify things a bit if they could just
>
> copy-and-paste some text into the Python interpreter and have
>
> some packages from pip installed.
>
> That would simplify instructions on how to install package xyz,
>
> without going into the vagaries of how to open a shell on various
>
> platforms, and how to get to the right pip executable.
>
> I think this could be as simple as:
>
>   def pip(args):
>   import sys
>   import subprocess
>   subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>
>   print("Please re-start Python now to use installed or upgraded
> packages.")
>
> Note that I added the final message about restarting the interpreter
>
> as a low-tech solution to the problem of packages being already
>
> imported in the current Python session.
>
> I would imagine that the author of package xyz would then put on
>
> their webpage something like:
>
>   To use, enter in your Python interpreter:
>
>  pip("install xyz --user")
>
> As another example, consider prof. Baldwin from Woolamaloo university
>
> who teaches a course "Introductory Python programming for Sheep Shavers".
>
> In his course material, he instructs his students to execute the
>
> following line in their Python interpreter.
>
>pip("install woolamaloo-sheepshavers-goodies --user")
>
> which will install a package which will in turn, as dependencies,
>
> pull in a number of packages which are relevant for sheep shaving but
>
> which have nevertheless irresponsibly been left outside the stdlib.
>
> Stephan
>
>
>
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Alex Walters]

If you are calling pip.main, you know what you are doing, and you know when you 
have to restart the interpreter or not.  An object that displays an error any 
time you try to use it, instructing a user of the proper way to use pip outside 
of the interpreter, is intended to catch the newbie mistake of trying to run 
pip from inside the repl, without encouraging potentially problematic behavior 
related to calling pip from inside the interpreter.  Don’t encourage the wrong 
way because it’s a common newbie mistake, give them better documentation and 
error messages.  Pip is a tool to be run outside of the repl, let’s keep it 
that way.

 

My method avoids the entire problem by not attempting to run pip, while still 
providing a meaningful and useful error message to new users.  If there is to 
be a solution other than SyntaxError or NameError (my solution only solves 
NameError), it should be to tell users how to use pip properly, not support 
improper use.

 

From: Python-ideas 
[mailto:python-ideas-bounces+tritium-list=sdamon@python.org] On Behalf Of 
Antoine Rozo
Sent: Sunday, October 29, 2017 3:32 PM
To: Python-Ideas 
Subject: Re: [Python-ideas] install pip packages from Python prompt

 

Hi,

 

What would be the difference with current pip module?

pip.main(['install', 'some_package'])

 

2017-10-29 20:26 GMT+01:00 Alex Walters  >:

I have a somewhat better, imo, implementation of a pip object to be loaded into 
the repl.

 

class pip:

def __call__(self, *a, **kw):

sys.stderr.write(str(self))

 

def __repr__(self):

return str(self)

 

def __str__(self):

return “Please run pip from your system command prompt”

 

 

 

From: Python-ideas [mailto:python-ideas-bounces+tritium-list 
 =sdamon@python.org 
 ] On Behalf Of Stephan Houben
Sent: Sunday, October 29, 2017 3:19 PM
To: Python-Ideas  >
Subject: [Python-ideas] install pip packages from Python prompt

 

Hi all,

Here is in somewhat more detail my earlier proposal for

having in the interactive Python interpreter a `pip` function to 

install packages from Pypi.

Motivation: it appears to me that there is a category of newbies

for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just

copy-and-paste some text into the Python interpreter and have

some packages from pip installed. 

That would simplify instructions on how to install package xyz,

without going into the vagaries of how to open a shell on various 

platforms, and how to get to the right pip executable.

I think this could be as simple as:

  def pip(args):
  import sys
  import subprocess
  subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

  print("Please re-start Python now to use installed or upgraded packages.")

Note that I added the final message about restarting the interpreter

as a low-tech solution to the problem of packages being already

imported in the current Python session.

I would imagine that the author of package xyz would then put on

their webpage something like:

  To use, enter in your Python interpreter:

 pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university

who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the

following line in their Python interpreter.

   pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,

pull in a number of packages which are relevant for sheep shaving but

which have nevertheless irresponsibly been left outside the stdlib.

Stephan

 

 


___
Python-ideas mailing list
Python-ideas@python.org  
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/





 

-- 

Antoine Rozo

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Stephan Houben]

Hi Antoine,

2017-10-29 20:31 GMT+01:00 Antoine Rozo :

> Hi,
>
> What would be the difference with current pip module?
> pip.main(['install', 'some_package'])
>


My understanding is that direct use of the `pip` module is explicitly not
recommended.

Stephan




>
> 2017-10-29 20:26 GMT+01:00 Alex Walters :
>
>> I have a somewhat better, imo, implementation of a pip object to be
>> loaded into the repl.
>>
>>
>>
>> class pip:
>>
>> def __call__(self, *a, **kw):
>>
>> sys.stderr.write(str(self))
>>
>>
>>
>> def __repr__(self):
>>
>> return str(self)
>>
>>
>>
>> def __str__(self):
>>
>> return “Please run pip from your system command prompt”
>>
>>
>>
>>
>>
>>
>>
>> *From:* Python-ideas [mailto:python-ideas-bounces+tritium-list=
>> sdamon@python.org] *On Behalf Of *Stephan Houben
>> *Sent:* Sunday, October 29, 2017 3:19 PM
>> *To:* Python-Ideas 
>> *Subject:* [Python-ideas] install pip packages from Python prompt
>>
>>
>>
>> Hi all,
>>
>> Here is in somewhat more detail my earlier proposal for
>>
>> having in the interactive Python interpreter a `pip` function to
>>
>> install packages from Pypi.
>>
>> Motivation: it appears to me that there is a category of newbies
>>
>> for which "open a shell and do `pip whatever`" is a bit too much.
>>
>> It would, in my opinion, simplify things a bit if they could just
>>
>> copy-and-paste some text into the Python interpreter and have
>>
>> some packages from pip installed.
>>
>> That would simplify instructions on how to install package xyz,
>>
>> without going into the vagaries of how to open a shell on various
>>
>> platforms, and how to get to the right pip executable.
>>
>> I think this could be as simple as:
>>
>>   def pip(args):
>>   import sys
>>   import subprocess
>>   subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>>
>>   print("Please re-start Python now to use installed or upgraded
>> packages.")
>>
>> Note that I added the final message about restarting the interpreter
>>
>> as a low-tech solution to the problem of packages being already
>>
>> imported in the current Python session.
>>
>> I would imagine that the author of package xyz would then put on
>>
>> their webpage something like:
>>
>>   To use, enter in your Python interpreter:
>>
>>  pip("install xyz --user")
>>
>> As another example, consider prof. Baldwin from Woolamaloo university
>>
>> who teaches a course "Introductory Python programming for Sheep Shavers".
>>
>> In his course material, he instructs his students to execute the
>>
>> following line in their Python interpreter.
>>
>>pip("install woolamaloo-sheepshavers-goodies --user")
>>
>> which will install a package which will in turn, as dependencies,
>>
>> pull in a number of packages which are relevant for sheep shaving but
>>
>> which have nevertheless irresponsibly been left outside the stdlib.
>>
>> Stephan
>>
>>
>>
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
>
> --
> Antoine Rozo
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Antoine Rozo]

Hi,

What would be the difference with current pip module?
pip.main(['install', 'some_package'])

2017-10-29 20:26 GMT+01:00 Alex Walters :

> I have a somewhat better, imo, implementation of a pip object to be loaded
> into the repl.
>
>
>
> class pip:
>
> def __call__(self, *a, **kw):
>
> sys.stderr.write(str(self))
>
>
>
> def __repr__(self):
>
> return str(self)
>
>
>
> def __str__(self):
>
> return “Please run pip from your system command prompt”
>
>
>
>
>
>
>
> *From:* Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@
> python.org] *On Behalf Of *Stephan Houben
> *Sent:* Sunday, October 29, 2017 3:19 PM
> *To:* Python-Ideas 
> *Subject:* [Python-ideas] install pip packages from Python prompt
>
>
>
> Hi all,
>
> Here is in somewhat more detail my earlier proposal for
>
> having in the interactive Python interpreter a `pip` function to
>
> install packages from Pypi.
>
> Motivation: it appears to me that there is a category of newbies
>
> for which "open a shell and do `pip whatever`" is a bit too much.
>
> It would, in my opinion, simplify things a bit if they could just
>
> copy-and-paste some text into the Python interpreter and have
>
> some packages from pip installed.
>
> That would simplify instructions on how to install package xyz,
>
> without going into the vagaries of how to open a shell on various
>
> platforms, and how to get to the right pip executable.
>
> I think this could be as simple as:
>
>   def pip(args):
>   import sys
>   import subprocess
>   subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>
>   print("Please re-start Python now to use installed or upgraded
> packages.")
>
> Note that I added the final message about restarting the interpreter
>
> as a low-tech solution to the problem of packages being already
>
> imported in the current Python session.
>
> I would imagine that the author of package xyz would then put on
>
> their webpage something like:
>
>   To use, enter in your Python interpreter:
>
>  pip("install xyz --user")
>
> As another example, consider prof. Baldwin from Woolamaloo university
>
> who teaches a course "Introductory Python programming for Sheep Shavers".
>
> In his course material, he instructs his students to execute the
>
> following line in their Python interpreter.
>
>pip("install woolamaloo-sheepshavers-goodies --user")
>
> which will install a package which will in turn, as dependencies,
>
> pull in a number of packages which are relevant for sheep shaving but
>
> which have nevertheless irresponsibly been left outside the stdlib.
>
> Stephan
>
>
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>


-- 
Antoine Rozo
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Alex Walters]

I have a somewhat better, imo, implementation of a pip object to be loaded into 
the repl.

 

class pip:

def __call__(self, *a, **kw):

sys.stderr.write(str(self))

 

def __repr__(self):

return str(self)

 

def __str__(self):

return “Please run pip from your system command prompt”

 

 

 

From: Python-ideas 
[mailto:python-ideas-bounces+tritium-list=sdamon@python.org] On Behalf Of 
Stephan Houben
Sent: Sunday, October 29, 2017 3:19 PM
To: Python-Ideas 
Subject: [Python-ideas] install pip packages from Python prompt

 

Hi all,

Here is in somewhat more detail my earlier proposal for

having in the interactive Python interpreter a `pip` function to 

install packages from Pypi.

Motivation: it appears to me that there is a category of newbies

for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just

copy-and-paste some text into the Python interpreter and have

some packages from pip installed. 

That would simplify instructions on how to install package xyz,

without going into the vagaries of how to open a shell on various 

platforms, and how to get to the right pip executable.

I think this could be as simple as:

  def pip(args):
  import sys
  import subprocess
  subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

  print("Please re-start Python now to use installed or upgraded packages.")

Note that I added the final message about restarting the interpreter

as a low-tech solution to the problem of packages being already

imported in the current Python session.

I would imagine that the author of package xyz would then put on

their webpage something like:

  To use, enter in your Python interpreter:

 pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university

who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the

following line in their Python interpreter.

   pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,

pull in a number of packages which are relevant for sheep shaving but

which have nevertheless irresponsibly been left outside the stdlib.

Stephan

 

 

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] install pip packages from Python prompt

[Stephan Houben]

Hi all,

Here is in somewhat more detail my earlier proposal for
having in the interactive Python interpreter a `pip` function to
install packages from Pypi.

Motivation: it appears to me that there is a category of newbies
for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just
copy-and-paste some text into the Python interpreter and have
some packages from pip installed.
That would simplify instructions on how to install package xyz,
without going into the vagaries of how to open a shell on various
platforms, and how to get to the right pip executable.

I think this could be as simple as:

  def pip(args):
  import sys
  import subprocess
  subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
  print("Please re-start Python now to use installed or upgraded
packages.")

Note that I added the final message about restarting the interpreter
as a low-tech solution to the problem of packages being already
imported in the current Python session.

I would imagine that the author of package xyz would then put on
their webpage something like:

  To use, enter in your Python interpreter:
 pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university
who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the
following line in their Python interpreter.

   pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,
pull in a number of packages which are relevant for sheep shaving but
which have nevertheless irresponsibly been left outside the stdlib.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Antoine Pitrou]

On Sun, 29 Oct 2017 11:44:52 +
Paul Moore  wrote:

> On 29 October 2017 at 09:51, Antoine Pitrou  wrote:
> > On Sun, 29 Oct 2017 17:54:22 +1000
> > Nick Coghlan  wrote:  
> >> This means that
> >> if educators aren't teaching them, or redistributors aren't providing them,
> >> then they're actively doing their users a disservice  
> >
> > Which redistributors do not provide the requests library, for example?
> > regex is probably not as popular (mostly because re is good enough for
> > most purposes), but it still appears to be available from Ubuntu and
> > Anaconda.  
> 
> I know it's not what you meant, but "the python.org installers" is the
> obvious answer here.

Well, I'm not sure what Nick meant by the word exactly, but "to
redistribute" means "to distribute again", so the word looked aimed
at third-party vendors of Python rather than python.org :-)

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Guido van Rossum]

The two use cases you describe (scripters and teachers) leave me luke-warm
-- scripters live in the wild west and can just pip install whatever
(that's what it means to be scripting) and teachers tend to want a
customized bundle anyway -- let the edu world get together and create their
own recommended bundle.

As long as it's not going to be bundled, i.e. there's just going to be some
list of packages that we recommend to 3rd party repackagers, then I'm fine
with it. But they must remain clearly marked as 3rd party packages in
whatever docs we provide, and live in site-packages.

I would really like to see what you'd add to the list besides requests -- I
really don't see why the teaching use case would need the regex module
(unless it's a class in regular expressions).

--Guido

On Sun, Oct 29, 2017 at 12:54 AM, Nick Coghlan  wrote:

> On 29 October 2017 at 15:16, Guido van Rossum  wrote:
>
>> Why? What's wrong with pip install?
>>
>
> At a technical level, this would just be a really thin wrapper around 'pip
> install' (even thinner than ensurepip in general, since these libraries
> *wouldn't* be bundled for offline installation, only listed by name).
>
>
>> Why complicate things? Your motivation is really weak here. "beneficial"?
>> "difficult cases"?
>>
>
> The main recurring problems with "pip install" are a lack of
> discoverability and a potential lack of availability (depending on the
> environment).
>
> This then causes a couple of key undesirable outcomes:
>
> - folks using Python as a teaching language have to choose between
> teaching with just the standard library APIs, requiring that learners
> restrict themselves to a particular preconfigured learning environment, or
> make a detour into package management tools in order to ensure learners
> have access to the APIs they actually want to use (this isn't hypothetical
> - I was a technical reviewer for a book that justified teaching XML-RPC
> over HTTPS+JSON on the basis that xmlrpc was in the standard library, and
> requests wasn't)
> - folks using Python purely as a scripting language (i.e without app level
> dependency management) may end up having to restrict themselves to the
> standard library API, even when there's a well-established frequently
> preferred alternative for what they're doing (e.g. requests for API
> management, regex for enhanced regular expressions)
>
> The underlying problem is that our reasons for omitting these particular
> libraries from the standard library relate mainly to publisher side
> concerns like the logistics of ongoing bug fixing and support, *not* end
> user concerns like software reliability or API usability. This means that
> if educators aren't teaching them, or redistributors aren't providing them,
> then they're actively doing their users a disservice (as opposed to other
> cases like web frameworks and similar, where there are multiple competing
> options, you're only going to want one of them in any given application,
> and the relevant trade-offs between the available options depend greatly on
> exactly what you're doing)
>
> Now, the Python-for-data-science community have taken a particular
> direction around handling this, and there's an additional library set
> beyond the standard library that's pretty much taken for granted in a data
> science context. While conda has been the focal point for those efforts
> more recently, it started a long time ago with initiatives like Python(x,
> y) and the Enthought Python Distribution.
>
> Similarly, initiatives like Raspberry Pi are able to assume a particular
> learning environment (Raspbian in the Pi's case), rather than coping with
> arbitrary starting points.
>
> Curated lists like the "awesome-python" one that Stephan linked don't
> really help that much with the discoverability problem, since they become
> just another thing for people to learn: How do they find out such lists
> exist in the first place? Given such a list, how do they determine if the
> recommendations it offers are actually relevant to their needs? Since
> assessing a published package API against your needs as a user is a skill
> that has to be learned like any other, it can be a lot easier to get
> started in a more prescriptive environment that says "This is what you have
> to work with for now, we'll explain more about your options for branching
> out later".
>
> The proposal in this thread thus stems from asking the question "Who is
> going to be best positioned to offer authoritative advice on which third
> party modules may be preferable to their standard library counterparts for
> end users of Python?" and answering it with "The standard library module
> maintainers that are already responsible for deciding whether or not to
> place appropriate See Also links in the module documentation".
>
> All the proposal does is to suggest taking those existing recommendations
> from the documentation and converting them into a more readibly 

Re: [Python-ideas] Composition over Inheritance

[Soni L.]

On 2017-10-29 02:57 PM, Brendan Barnwell wrote:

On 2017-10-29 04:44, Soni L. wrote:

And this is how you miss the whole point of being able to dynamically
add/remove arbitrary components on objects you didn't create, at 
runtime.


Someone gave me this code and told me it explains what I'm trying to do:
https://repl.it/NYCF/3

class T:
 pass

class C:
 pass

c = C()

#c.[T] = 1
c.__dict__[T] = 1


Again, can you please explain why you want to write c.[T]? What do 
you intend that to *do*?  Your commented line seems to indicate you 
want it to do what `c.__dict__[T]` does, but you can already do that 
with `setattr(c, T, 1)`.  Or you can just give c an attribute that's a 
dict, but has an easier-to-type name than __dict__, so you can do 
`c.mydict[T]`.  What is the specific advantage of `c.[T]` over these 
existing solutions?




Hmm... Why can't we just allow empty identifiers, and set a default 
handler for empty identifiers that implements the proposed ECS?


But the basic idea is to indicate something at the call site, namely 
that T is a contract and the object returned should respect that 
contract and any function calls should pass the original object as an 
argument. (I personally don't like how Python treats o.m() (has self) 
the same as o.f() (no self) syntax-wise, either.)

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Stefan Krah]

On Sun, Oct 29, 2017 at 10:10:21AM -0700, Brendan Barnwell wrote:
> Only now, since the libraries aren't in the stdlib, the Python devs
> won't really be able to do anything to fix that; all they could do
> is remove the offending package from the approved list.  In practice
> I think this is unlikely to happen, since the idea would be that
> Python devs would be judicious in awarding the seal of approval only
> to projects that are robust and not prone to breakage.  But it does
> change the nature of the approval from "we approve this *code* and
> we will fix it if it breaks" (which is how the existing stdlib
> works) to "we approve these *people* (the people working on requests
> or regex or whatever) and we will cease to do if they break their
> code".

Let's be realistic: If MRAB were to include regex in the stdlib, *he*
would be the one fixing things anyway.

And he'd get to fight feature requests and stylistic rewrites. :-)


Stefan Krah




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Brendan Barnwell]

On 2017-10-29 00:54, Nick Coghlan wrote:

The proposal in this thread thus stems from asking the question "Who is
going to be best positioned to offer authoritative advice on which third
party modules may be preferable to their standard library counterparts
for end users of Python?" and answering it with "The standard library
module maintainers that are already responsible for deciding whether or
not to place appropriate See Also links in the module documentation".

All the proposal does is to suggest taking those existing
recommendations from the documentation and converting them into a more
readibly executable form.


	So it sounds like you're just saying that there should be one more 
"awesome-python" style list, except it should be official, with the seal 
of approval of Python itself.  The details of exactly how that would 
displayed to users are not so important as that it be easily accessible 
from python.org and clearly shown to be endorsed by the Python developers.


	I think that is a good idea.  The current situation is quite confusing 
in many cases.  You see a lot of questions on StackOverflow where 
someone is tearing their hair out trying to use urllib to do heavy 
lifting and the answer is "use requests instead".  Likewise people 
trying to implement matrix multiplication who should probably be using 
numpy,  As for regex, to be honest, I have yet to find any way in which 
it is not completely superior to the existing re library, and I find it 
somewhat frustrating that the "publisher-side" concerns you mention 
continue to hold it back from replacing re.


	The only problem I see with this sort of "Python seal of approval" 
library list is that it carries some of the same risks as incorporating 
such libraries into the stdlib.  Not all, but some.  For one thing, if 
people go to python.org and see a thing that says "You may also want to 
use these libraries that bear the Python Seal of Approval. . .", and 
then they download them, and something goes wrong (e.g., there is some 
kind of version conflict with another package), they will likely blame 
python.org (at least partially).  Only now, since the libraries aren't 
in the stdlib, the Python devs won't really be able to do anything to 
fix that; all they could do is remove the offending package from the 
approved list.  In practice I think this is unlikely to happen, since 
the idea would be that Python devs would be judicious in awarding the 
seal of approval only to projects that are robust and not prone to 
breakage.  But it does change the nature of the approval from "we 
approve this *code* and we will fix it if it breaks" (which is how the 
existing stdlib works) to "we approve these *people* (the people working 
on requests or regex or whatever) and we will cease to do if they break 
their code".


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Composition over Inheritance

[Brendan Barnwell]

On 2017-10-29 04:44, Soni L. wrote:

And this is how you miss the whole point of being able to dynamically
add/remove arbitrary components on objects you didn't create, at runtime.

Someone gave me this code and told me it explains what I'm trying to do:
https://repl.it/NYCF/3

class T:
 pass

class C:
 pass

c = C()

#c.[T] = 1
c.__dict__[T] = 1


	Again, can you please explain why you want to write c.[T]?  What do you 
intend that to *do*?  Your commented line seems to indicate you want it 
to do what `c.__dict__[T]` does, but you can already do that with 
`setattr(c, T, 1)`.  Or you can just give c an attribute that's a dict, 
but has an easier-to-type name than __dict__, so you can do 
`c.mydict[T]`.  What is the specific advantage of `c.[T]` over these 
existing solutions?


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Koos Zevenhoven]

On Sun, Oct 29, 2017 at 1:44 PM, Paul Moore  wrote:

> On 29 October 2017 at 09:51, Antoine Pitrou  wrote:
> > On Sun, 29 Oct 2017 17:54:22 +1000
> > Nick Coghlan  wrote:
> >> All the proposal does is to suggest taking those existing
> recommendations
> >> from the documentation and converting them into a more readibly
> executable
> >> form.
> >
> > I'm curious what such a list looks like :-)
>
> I am also. I'd put requests on it immediately, but that's the only
> thing I consider obvious. regex is what triggered this, but I'm not
> sure it adds *that* much - it's a trade off between people who need
> the extra features and people confused over why we have two regex
> libraries available. After that, you're almost immediately into
> domain-specific answers, and it becomes tricky fast.
>

​If the list is otherwise not be long enough, maybe it will contain
something that is now in the stdlib?-)

-- Koos​


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Wes Turner]

On Sunday, October 29, 2017, Paul Moore  wrote:

> On 29 October 2017 at 10:40, Stephan Houben  > wrote:
> > Perhaps slightly off-topic, but I have sometimes wondered if
> > pip could not be made somewhat friendlier for the absolute newbie
> > and the classroom context.
> >
> > Some concrete proposals.
> >
> > 1. Add a function `pip` to the interactive interpreter
> >   (similar to how `help` is available).
> >
> >   def pip(args):
> >   import sys
> >   import subprocess
> >   subprocess.check_call([sys.executable, "-m", "pip"] +
> args.split())
> >
> >This allows people to install something using pip as long as they
> have a
> >Python prompt open, and avoids instructors to have to deal with
> > platform-specific
> >instructions for various shells. Also avoids confusion when multiple
> > Python interpreters
> >are available (it operates in the context of the current interpreter.)
>
> There are subtle issues around whether newly installed/upgraded
> packages are visible in a running Python interpreter.


If the package is already imported, reload() isn't sufficient; but
deepreload from IPython may be.

IPython also supports shell commands prefixed with '!':

! pip install -U requests regex IPython
! pip install -U --user psfblessedset1
%run pip install -U --user psfblessedset1
%run?

% autoreload?
# http://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html

This:


> It's possible
> that this would result in *more* confusion than the current situation.


Why isn't it upgrading over top of the preinstalled set?

At least with bash, the shell command history is logged to .bash_history by
default. IMO, it's easier to assume that environment-modifying commands are
logged in the shell log; and that running a ``%logstart -o logged-cmds.py``
won't change the versions of packages it builds upon.

Is it bad form to put ``! pip install -U requests `` at the top of every
jupyter notebook?
``! pip install requests==version `` is definitely more reproducible.
(seeAlso: binder, jupyter/docker-stacks)

I'll just include these here unnecessarily from the other curated package
set discussions:

https://github.com/jupyter/docker-stacks

https://github.com/Kaggle/docker-python/blob/master/Dockerfile

https://binderhub.readthedocs.io/en/latest/

I can see the appeal of something like this, but it's not as simple as
> it looks. If you want to discuss this further, I'd definitely suggest
> making it a thread of its own.


> Personally, as a pip maintainer, I'm -0
> on this (possibly even -1).


Same.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Composition over Inheritance

[Koos Zevenhoven]

(looks like you forgot to reply to the list)

On Sun, Oct 29, 2017 at 7:47 AM, Greg Ewing 
wrote:

> Koos Zevenhoven wrote:
>
>> It looks like you can just as easily drive the car without having the key
>>
>
> That's been found to be a problem in real life, too.
> More than one Python-programming car thief has been
> caught with this function in their personal library:
>
> def hotwire(car):
>car.engine.start()


​Yes, but my point was still about what the public API is. Starting the car
with the key should be easier than starting it without the key. Surely you
should be able to start the engine without the key if you look under the
hood and figure it out. And maybe you'd need a tool for that to be kept in
your garage for debugging, but not as a button attached to the outside of
the car or on the dashboard :-).

The problem becomes even worse if you instead make Car inherit from its
parts (including Engine), because the relationship of a car and and engine
is not well described by inheritance. A car is not a kind of engine, nor
should it pretend to be an implementation of an engine. A car *comprises*
(or contains) an engine.  And who knows, maybe it has two engines!

-- Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Paul Moore]

On 29 October 2017 at 10:40, Stephan Houben  wrote:
> Perhaps slightly off-topic, but I have sometimes wondered if
> pip could not be made somewhat friendlier for the absolute newbie
> and the classroom context.
>
> Some concrete proposals.
>
> 1. Add a function `pip` to the interactive interpreter
>   (similar to how `help` is available).
>
>   def pip(args):
>   import sys
>   import subprocess
>   subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>
>This allows people to install something using pip as long as they have a
>Python prompt open, and avoids instructors to have to deal with
> platform-specific
>instructions for various shells. Also avoids confusion when multiple
> Python interpreters
>are available (it operates in the context of the current interpreter.)

There are subtle issues around whether newly installed/upgraded
packages are visible in a running Python interpreter. It's possible
that this would result in *more* confusion than the current situation.
I can see the appeal of something like this, but it's not as simple as
it looks. If you want to discuss this further, I'd definitely suggest
making it a thread of its own. Personally, as a pip maintainer, I'm -0
on this (possibly even -1).

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Composition over Inheritance

[Soni L.]

On 2017-10-29 02:28 AM, Nick Coghlan wrote:
On 29 October 2017 at 12:25, Brendan Barnwell > wrote:


On 2017-10-28 19:13, Soni L. wrote:

And to have all cars have engines, you'd do:

class Car:
    def __init__(self, ???):
      self[Engine] = GasEngine()

car = Car()
car[Engine].kickstart() # kickstart gets the car as second
argument.

And if you can't do that, then you can't yet do what I'm
proposing, and
thus the proposal makes sense, even if it still needs some
refining...


        As near as I can tell you can indeed do that, although
it's still not clear to me why you'd want to. You can give Car a
__getitem__ that on-the-fly generates an Engine object that knows
which Car it is attached to, and then you can make
Engine.kickstart a descriptor that knows which Engine it is
attached to, and from that can figure out which Car it is attached to.


Right, I think a few different things are getting confused here 
related to how different folks use composition.


For most data modeling use cases, the composition model you want is 
either a tree or an acyclic graph, where the subcomponents don't know 
anything about the whole that they're a part of. This gives you good 
component isolation, and avoids circular dependencies.


However, for other cases, you *do* want the child object to be aware 
of the parent - XML etrees are a classic example of this, where we 
want to allow navigation back up the tree, so each node gains a 
reference to its parent node. This often takes the form of a 
combination of delegation (parent->child references) and dependency 
inversion (child->parent reference).


For the car/engine example, this relates to explicitly modeling the 
relationship whereby a car can have one or more engines (but the 
engine may not currently be installed), while an engine can be 
installed in at most one car at any given point in time.


You don't even need the descriptor protocol for that though, you just 
need the subcomponent to accept the parent reference as a constructor 
parameter:


    class Car:
      def __init__(self, engine_type):
    self.engine = engine_type(self)

However, this form of explicit dependency inversion wouldn't work as 
well if you want to be able to explicitly create an "uninstalled 
engine" instance, and then pass the engine in as a parameter to the 
class constructor:


    class Car:
      def __init__(self, engine):
    self.engine = engine # How would we ensure the engine is 
marked as installed here?


As it turns out, Python doesn't need new syntax for this either, as 
it's all already baked into the regular attribute access syntax, 
whereby descriptor methods get passed a reference not only to the 
descriptor, but *also* to the object being accessed: 
https://docs.python.org/3/howto/descriptor.html#descriptor-protocol


And then the property builtin lets you ignore the existence of the 
descriptor object entirely, and only care about the original object, 
allowing the above example to be written as:


    class Car:
      def __init__(self, engine):
    self.engine = engine # This implicitly marks the engine as 
installed


  @property
  def engine(self):
  return self._engine

  @engine.setter
  def engine(self, engine):
 if engine is not None:
         if self._engine is not None:
             raise RuntimeError("Car already has an engine installed")
         if engine._car is not None:
         raise RuntimeError("Engine is already installed in 
another car")

         engine._car = self
 self._engine = engine

car = Car(GasEngine())

ORMs use this kind of descriptor based composition management 
extensively in order to reliably model database foreign key 
relationships in a way that's mostly transparent to users of the ORM 
classes.


And this is how you miss the whole point of being able to dynamically 
add/remove arbitrary components on objects you didn't create, at runtime.


Someone gave me this code and told me it explains what I'm trying to do: 
https://repl.it/NYCF/3


class T:
    pass

class C:
    pass

c = C()

#c.[T] = 1
c.__dict__[T] = 1

I'd also like to add:

def someone_elses_lib_function(arbitrary_object):
  #arbitrary_object.[T] = object()
  arbitrary_object.__dict__[T] = object()

and

def another_ones_lib_function(arbitrary_object):
  #if arbitrary_object.[T]:
  if arbitrary_object.__dict__[T]:
    #arbitrary_object.[T].thing()
    arbitrary_object.__dict__[T].thing(arbitrary_object)



Cheers,
Nick.

--
Nick Coghlan   | ncogh...@gmail.com    | 
Brisbane, Australia



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: 

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Paul Moore]

On 29 October 2017 at 09:51, Antoine Pitrou  wrote:
> On Sun, 29 Oct 2017 17:54:22 +1000
> Nick Coghlan  wrote:
>> This means that
>> if educators aren't teaching them, or redistributors aren't providing them,
>> then they're actively doing their users a disservice
>
> Which redistributors do not provide the requests library, for example?
> regex is probably not as popular (mostly because re is good enough for
> most purposes), but it still appears to be available from Ubuntu and
> Anaconda.

I know it's not what you meant, but "the python.org installers" is the
obvious answer here. On Windows, if you say to someone "install
Python", they get the python.org distribution. Explicitly directing
them to Anaconda is an option, but that gives them a distinctly
different experience than "standard Python plus some best of breed
packages like requests" does.

>> All the proposal does is to suggest taking those existing recommendations
>> from the documentation and converting them into a more readibly executable
>> form.
>
> I'm curious what such a list looks like :-)

I am also. I'd put requests on it immediately, but that's the only
thing I consider obvious. regex is what triggered this, but I'm not
sure it adds *that* much - it's a trade off between people who need
the extra features and people confused over why we have two regex
libraries available. After that, you're almost immediately into
domain-specific answers, and it becomes tricky fast.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Paul Moore]

On 29 October 2017 at 07:54, Nick Coghlan  wrote:
> On 29 October 2017 at 15:16, Guido van Rossum  wrote:
>>
>> Why? What's wrong with pip install?
>
> At a technical level, this would just be a really thin wrapper around 'pip
> install' (even thinner than ensurepip in general, since these libraries
> *wouldn't* be bundled for offline installation, only listed by name).

I agree with Guido, this doesn't seem to add much as stated.

(From an earlier message of Nick's)
> At the same time, it would be beneficial to have a way to offer an even
> stronger recommendation to redistributors that we think full-featured
> general purpose Python scripting environments should offer the regex
> module as an opt-in alternative to the baseline re feature set, since that
> would also help with other currently difficult cases like the requests module.

This is key to me. The target area is *scripting*. Library and
application developers have their own ways of managing the dependency
handling problem, and they generally work fine. The people who don't
currently have a good solution are those who just use the system
install - i.e., people writing adhoc scripts, people writing code that
they want to share with other members of their organisation, via
"here's this file, just run it", and not as a full application. For
those people "not in a standard install" is a significantly higher
barrier to usage than elsewhere. "Here's a Python file, just install
Python and double click on the file" is a reasonable request. Running
pip may be beyond the capabilities of the recipient.

In my view, the key problems in this area are:

1. What users writing one-off scripts can expect to be available.
2. Corporate environments where "adding extra software" is a major hurdle.
3. Offline environments, where PyPI isn't available.

The solutions to these issues aren't so much around how we let people
know that they should do "pip install" (or "--install-recommended")
but rather around initial installs. To that end I'd suggest:

1. The python.org installers gain an "install recommended 3rd party
packages" option, that is true by default, which does "pip install
". This covers (1) and (2) above, as it makes
the recommended package set the norm, and ensures that they are
covered by an approval to "install Python".
2. For offline environments, we need to do a little more, but I'd
imagine having the installer look for wheels in the same directory as
the installer executable, and leave it to the user to put them there.
If wheels aren't available the installer should warn but continue.

For 3rd party distributions (Linux, Homebrew, Conda, ...) this gives a
clear message that Python users are entitled to expect these modules
to be available. Handling that expectation is down to those
distributions.

Things I haven't thought through:

1. System vs user site-packages. If we install (say) requests into the
system site-packages, the user could potentially need admin rights to
upgrade it. Or get into a situation where they have an older version
in site-packages and a newer one in user-site (which I don't believe
is a well-tested scenario),
2. Virtual environments. Should venv/virtualenv include the
recommended packages by default? Probably not, but that does make the
"in a virtual environment" experience different from the system Python
experience.
3. The suggestion above for offline environments isn't perfect, by any
means. Better solutions would be good here (but I don't think we want
to go down the bundling route like we did with pip...?).

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Stephan Houben]

Perhaps slightly off-topic, but I have sometimes wondered if
pip could not be made somewhat friendlier for the absolute newbie
and the classroom context.

Some concrete proposals.

1. Add a function `pip` to the interactive interpreter
  (similar to how `help` is available).

  def pip(args):
  import sys
  import subprocess
  subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

   This allows people to install something using pip as long as they have a
   Python prompt open, and avoids instructors to have to deal with
platform-specific
   instructions for various shells. Also avoids confusion when multiple
Python interpreters
   are available (it operates in the context of the current interpreter.)

2. Add to Pypi package webpages a line like:

To install, execute this line in your Python interpreter:
 pip("install my-package --user")

  Make this copyable with a button (like Github allows you to copy the repo
name).

3. Add the notion of a "channel" to which one can "subscribe".

   This is actually nothing new, just a Pypi package with no code and just
a bunch of
   dependencies. But it allows me to say: Just enter:

  pip("install stephans-awesome-stuff --user")

  in your Python and you get a bunch of useful stuff.

Stephan

2017-10-29 8:54 GMT+01:00 Nick Coghlan :

> On 29 October 2017 at 15:16, Guido van Rossum  wrote:
>
>> Why? What's wrong with pip install?
>>
>
> At a technical level, this would just be a really thin wrapper around 'pip
> install' (even thinner than ensurepip in general, since these libraries
> *wouldn't* be bundled for offline installation, only listed by name).
>
>
>> Why complicate things? Your motivation is really weak here. "beneficial"?
>> "difficult cases"?
>>
>
> The main recurring problems with "pip install" are a lack of
> discoverability and a potential lack of availability (depending on the
> environment).
>
> This then causes a couple of key undesirable outcomes:
>
> - folks using Python as a teaching language have to choose between
> teaching with just the standard library APIs, requiring that learners
> restrict themselves to a particular preconfigured learning environment, or
> make a detour into package management tools in order to ensure learners
> have access to the APIs they actually want to use (this isn't hypothetical
> - I was a technical reviewer for a book that justified teaching XML-RPC
> over HTTPS+JSON on the basis that xmlrpc was in the standard library, and
> requests wasn't)
> - folks using Python purely as a scripting language (i.e without app level
> dependency management) may end up having to restrict themselves to the
> standard library API, even when there's a well-established frequently
> preferred alternative for what they're doing (e.g. requests for API
> management, regex for enhanced regular expressions)
>
> The underlying problem is that our reasons for omitting these particular
> libraries from the standard library relate mainly to publisher side
> concerns like the logistics of ongoing bug fixing and support, *not* end
> user concerns like software reliability or API usability. This means that
> if educators aren't teaching them, or redistributors aren't providing them,
> then they're actively doing their users a disservice (as opposed to other
> cases like web frameworks and similar, where there are multiple competing
> options, you're only going to want one of them in any given application,
> and the relevant trade-offs between the available options depend greatly on
> exactly what you're doing)
>
> Now, the Python-for-data-science community have taken a particular
> direction around handling this, and there's an additional library set
> beyond the standard library that's pretty much taken for granted in a data
> science context. While conda has been the focal point for those efforts
> more recently, it started a long time ago with initiatives like Python(x,
> y) and the Enthought Python Distribution.
>
> Similarly, initiatives like Raspberry Pi are able to assume a particular
> learning environment (Raspbian in the Pi's case), rather than coping with
> arbitrary starting points.
>
> Curated lists like the "awesome-python" one that Stephan linked don't
> really help that much with the discoverability problem, since they become
> just another thing for people to learn: How do they find out such lists
> exist in the first place? Given such a list, how do they determine if the
> recommendations it offers are actually relevant to their needs? Since
> assessing a published package API against your needs as a user is a skill
> that has to be learned like any other, it can be a lot easier to get
> started in a more prescriptive environment that says "This is what you have
> to work with for now, we'll explain more about your options for branching
> out later".
>
> The proposal in this thread thus stems from asking the question "Who is
> going to be best positioned 

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Antoine Pitrou]

On Sun, 29 Oct 2017 17:54:22 +1000
Nick Coghlan  wrote:
> 
> The underlying problem is that our reasons for omitting these particular
> libraries from the standard library relate mainly to publisher side
> concerns like the logistics of ongoing bug fixing and support, *not* end
> user concerns like software reliability or API usability.

They're both really.  One important consequence of a library being in
the stdlib is to tie it to the stdlib's release cycle, QA
infrastructure and compatibility requirements -- which more or less
solves many dependency and/or version pinning headaches.

> This means that
> if educators aren't teaching them, or redistributors aren't providing them,
> then they're actively doing their users a disservice

Which redistributors do not provide the requests library, for example?
regex is probably not as popular (mostly because re is good enough for
most purposes), but it still appears to be available from Ubuntu and
Anaconda.

> All the proposal does is to suggest taking those existing recommendations
> from the documentation and converting them into a more readibly executable
> form.

I'm curious what such a list looks like :-)

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Nick Coghlan]

On 29 October 2017 at 15:16, Guido van Rossum  wrote:

> Why? What's wrong with pip install?
>

At a technical level, this would just be a really thin wrapper around 'pip
install' (even thinner than ensurepip in general, since these libraries
*wouldn't* be bundled for offline installation, only listed by name).


> Why complicate things? Your motivation is really weak here. "beneficial"?
> "difficult cases"?
>

The main recurring problems with "pip install" are a lack of
discoverability and a potential lack of availability (depending on the
environment).

This then causes a couple of key undesirable outcomes:

- folks using Python as a teaching language have to choose between teaching
with just the standard library APIs, requiring that learners restrict
themselves to a particular preconfigured learning environment, or make a
detour into package management tools in order to ensure learners have
access to the APIs they actually want to use (this isn't hypothetical - I
was a technical reviewer for a book that justified teaching XML-RPC over
HTTPS+JSON on the basis that xmlrpc was in the standard library, and
requests wasn't)
- folks using Python purely as a scripting language (i.e without app level
dependency management) may end up having to restrict themselves to the
standard library API, even when there's a well-established frequently
preferred alternative for what they're doing (e.g. requests for API
management, regex for enhanced regular expressions)

The underlying problem is that our reasons for omitting these particular
libraries from the standard library relate mainly to publisher side
concerns like the logistics of ongoing bug fixing and support, *not* end
user concerns like software reliability or API usability. This means that
if educators aren't teaching them, or redistributors aren't providing them,
then they're actively doing their users a disservice (as opposed to other
cases like web frameworks and similar, where there are multiple competing
options, you're only going to want one of them in any given application,
and the relevant trade-offs between the available options depend greatly on
exactly what you're doing)

Now, the Python-for-data-science community have taken a particular
direction around handling this, and there's an additional library set
beyond the standard library that's pretty much taken for granted in a data
science context. While conda has been the focal point for those efforts
more recently, it started a long time ago with initiatives like Python(x,
y) and the Enthought Python Distribution.

Similarly, initiatives like Raspberry Pi are able to assume a particular
learning environment (Raspbian in the Pi's case), rather than coping with
arbitrary starting points.

Curated lists like the "awesome-python" one that Stephan linked don't
really help that much with the discoverability problem, since they become
just another thing for people to learn: How do they find out such lists
exist in the first place? Given such a list, how do they determine if the
recommendations it offers are actually relevant to their needs? Since
assessing a published package API against your needs as a user is a skill
that has to be learned like any other, it can be a lot easier to get
started in a more prescriptive environment that says "This is what you have
to work with for now, we'll explain more about your options for branching
out later".

The proposal in this thread thus stems from asking the question "Who is
going to be best positioned to offer authoritative advice on which third
party modules may be preferable to their standard library counterparts for
end users of Python?" and answering it with "The standard library module
maintainers that are already responsible for deciding whether or not to
place appropriate See Also links in the module documentation".

All the proposal does is to suggest taking those existing recommendations
from the documentation and converting them into a more readibly executable
form.

I'm not particularly wedded to any particular approach to making the
recommendations available in a more machine-friendly form, though - it's
just the "offer something more machine friendly than scraping the docs for
recommendation links" aspect that I'm interested in. For example, we could
skip touching ensurepip or venv at all, and instead limit this to a
documentation proposal to collect these recommendations from the
documentation, and publish them within the `venv` module docs as a
"recommended-libraries.txt" file (using pip's requirements.txt format).
That would be sufficient to allow straightforward 3rd party automation,
without necessarily committing to providing such automation ourselves.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: 

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Stephan Houben]

There are already curated lists of Python packages, such as:

https://github.com/vinta/awesome-python

Even better, if you don't agree, you can just clone it and create your own
;-)

Stephan

2017-10-29 6:46 GMT+01:00 Alex Walters :

> At this point, I would punt to distutils-sig about curated packages, and
> pip tooling to support that, but they are bogged down as it stands with
> just getting warehouse up and running.  I don’t support putting specialized
> tooling in python itself to install curated packages, because that curation
> would be on the same release schedule as python itself.  Pip is an
> important special case, but it’s a special case to avoid further special
> cases.  If there was excess manpower on the packaging side, that’s where it
> should be developed.
>
>
>
> *From:* Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@
> python.org] *On Behalf Of *Guido van Rossum
> *Sent:* Sunday, October 29, 2017 1:17 AM
> *To:* Nick Coghlan 
> *Cc:* python-ideas@python.org
> *Subject:* Re: [Python-ideas] Defining an easily installable "Recommended
> baseline package set"
>
>
>
> Why? What's wrong with pip install? Why complicate things? Your motivation
> is really weak here. "beneficial"? "difficult cases"?
>
>
>
> On Sat, Oct 28, 2017 at 8:57 PM, Nick Coghlan  wrote:
>
> Over on python-dev, the question of recommending MRAB's "regex" module
> over the standard library's "re" module for more advanced regular
> expressions recently came up again.
>
> Because of various logistical issues and backwards compatibility risks,
> it's highly unlikely that we'll ever be able to swap out the current _sre
> based re module implementation in the standard library for an
> implementation based on the regex module.
>
> At the same time, it would be beneficial to have a way to offer an even
> stronger recommendation to redistributors that we think full-featured
> general purpose Python scripting environments should offer the regex module
> as an opt-in alternative to the baseline re feature set, since that would
> also help with other currently difficult cases like the requests module.
>
> What I'm thinking is that we could make some relatively simple additions
> to the `ensurepip` and `venv` modules to help with this:
>
>
>
> 1. Add a ensurepip.RECOMMENDED_PACKAGES mapping keyed by standard library
> module names containing dependency specifiers for recommended third party
> packages for particular tasks (e.g. "regex" as an enhanced alternative to
> "re", "requests" as an enhanced HTTPS-centric alternative to
> "urllib.request")
>
> 2. Add a new `install_recommended` boolean flag to ensurepip.bootstrap
>
> 3. Add a corresponding `--install-recommended flag to the `python -m
> ensurepip` CLI
>
> 4. Add a corresponding `--install-recommended flag to the `python -m venv`
> CLI (when combined with `--without-pip`, this would run pip directly from
> the bundled wheel file to do the installations)
>
> We'd also need either a new informational PEP or else a section in the
> developer guide to say that the contents of `ensurepip.RECOMMENDED_PACKAGES`
> are up to the individual module maintainers (hence keying the mapping by
> standard library module name, rather than having a single flat list for the
> entire standard library).
>
> For redistributors with weak dependency support, these reference
> interpreter level recommendations could become redistributor level
> recommendations. Redistributors without weak dependency support could still
> make a distinction between "default" installations (which would include
> them) and "minimal" installations (which would exclude them).
>
> Folks writing scripts and example code for independent distribution (i.e.
> no explicitly declared dependencies) could then choose between relying on
> just the standard library (as now), or on the standard library plus
> independently versioned recommended packages.
>
>
>
> Cheers,
>
> Nick.
>
>
> --
>
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
>
> --
>
> --Guido van Rossum (python.org/~guido )
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/