[Python-ideas] Re: Python, Be Bold!

[James Lu]

Ideally, we'd have a functional package manager: one that can delete
binaries when disk space is running low, and recompiles from sources when
the binary is needed again. It could store Python 2 as a series of diffs
against Python 3.

On Thu, Jan 2, 2020, 2:22 AM Abdur-Rahmaan Janhangeer 
wrote:

> if not self-updating, at least the ability to update
>
> Yours,
>
> Abdur-Rahmaan Janhangeer
> pythonmembers.club  | github
> 
> Mauritius
> --
> https://mail.python.org/mailman/listinfo/python-list
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZNLX7C52K7LTUDAX4MAMX7GGFCVGYXTF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python, Be Bold!

[Stephen J. Turnbull]

James Lu writes:

 > Ideally, we'd have a functional package manager: one that can
 > delete binaries when disk space is running low, and recompiles from
 > sources when the binary is needed again.

First, that's not what package managers do.  Package managers manage
dependencies, not disk space.

Second, my "where I get almost all my work done" Python 3.7
installation is only 571MB; I need about 20GB free to update XCode, so
disk space is not high on my reasons for screwing with a Python
installation.  Much more practical to search for old .isos and .dmgs
to delete.  If I really thought a few hundred MB would help, I'd just
delete the whole thing and hope nothing depended on it until I could
resolve the issue requiring more space than I have.

If you really want to save disk space measured in MB, I guess you'd
probably want LRU semantics, and possibly a blacklist of modules that
are only used interactively and at most once a day or so, so the time
to import doesn't really matter.  It would be easy enough to write a
script to do the cleaning (the stat module can get you the access
times), and Python will take care of the recompilation since it only
ever executes compiled code.

Although speaking practically, you might want to compile with -O and
delete all the sources if space is such a problem.  Teach help() to
call out to doc.python.org or/and pip the sources.

 > It could store Python 2 as a series of diffs against Python 3.

Even if you used a binary diff such as xdelta or bsdiff, I doubt this
would save space.  Feel free to try it and tell me I'm wrong, though.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6SRPMEWVNAK7DQE7BDTDL7KFKN6CLWMS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python, Be Bold!

[Chris Angelico]

On Sat, Jan 4, 2020 at 1:16 AM Stephen J. Turnbull
 wrote:
> If you really want to save disk space measured in MB, I guess you'd
> probably want LRU semantics, and possibly a blacklist of modules that
> are only used interactively and at most once a day or so, so the time
> to import doesn't really matter.  It would be easy enough to write a
> script to do the cleaning (the stat module can get you the access
> times), and Python will take care of the recompilation since it only
> ever executes compiled code.
>

Be aware that access times aren't always accurate, since they depend
on the system performing a write every time you request a read. That's
an inefficiency that can easily be disabled (perhaps selectively - the
"relatime" mount option in Linux means to update the access time only
if the file's been changed since it was last accessed), at the price
of breaking the stats of something like you propose.

But honestly, if you're worried about a MB here and a MB there, you
probably will do better to zip up the stdlib than to fiddle around
with deleting stale modules.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EK7MG22LF2ONRNUTDXNCD3XQEDMEMRBP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python, Be Bold!

[Andrew Barnert via Python-ideas]

Where’s the initial email you’re replying to here? I don’t have it in my inbox, 
and it isn’t on the Mailman archive either, and since you snipped it down to a 
single line I have no idea what that snippet was referring to.

Meanwhile:

> On Jan 2, 2020, at 18:14, James Lu  wrote:
> 
> 
> Ideally, we'd have a functional package manager: one that can delete binaries 
> when disk space is running low, and recompiles from sources when the binary 
> is needed again. It could store Python 2 as a series of diffs against Python 
> 3.

I’m not sure why you want to store Python 2 as diffs against Python 3, but that 
is exactly what a version control system does, not what a package manager does; 
it’s easy if you use the right tool. For example:

* Create an empty git repo in ~/python 
* Create a branch called 2.7
* Build Python 2.7 and install it to that directory, and also install any 
site-packages you want
* Commit
* Create a new branch off master called 3.8
* Build Python 3.8 and install it to that directory, and also install any 
site-packages you want
* Commit

Git (and its various plugins for diffing files) takes care of what’s a diff 
against what and so on; you just have to know there are at least two useful 
leaves on the graph of diffs and they can be accessed with the names 2.7 and 
3.8.

Now you have to checkout the appropriate branch every time you want to run 
python or python2, or activate a non-isolated virtual environments built from 
either, etc. Which seems like a huge pain, but it seems like you’re asking for 
that pain.

Would this actually save any space? I don’t know. I think 2.7 and 3.8 have so 
little binary code in common (bad enough for just the executable—things like 
stdlib .pyc files end up with different names in different locations that git 
probably isn’t smart enough to link up in the first place) that you might well 
waste more on repo overhead than you’d gain in deduplicating. (If you wanted 
3.8.0 and 3.8.1 and 3.8.2, on the other hand, that would probably save a lot 
more, but how often do you want an earlier bug fix build of the same minor 
version?)

But it’s simple enough to do that rather than guessing you can just try it.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PQVDFDVH7NEQZK7YRAXTGBZ6NWDRCNSZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python, Be Bold!

[James Lu]

I use macOS, and using Python is very confusing.

- Apple's bundled Python 2.7.
- Anaconda (Python scientific stack package manager) Python and conda.
- Homebrew (3rd party package manager for macOS) Python and pip.
I also believe that there is a PSF Python installer, but I am not sure.


Python, Be Bold captures the spirit of it should not be a shame to have the
interpreter/VM installed on end-users machines. It also facilitates the
work of other python devs. You installed it because of one Python program,
other programs benefit from it. It also proposes enhancements to the VM to
better facilitate that.

One big reason the web is a popular platform today among the general public

 is because it offers strong sandboxing,

and privelges are granted per-site, opt out by default.
Another reason the web is popular is because it loads quickly.
I suggest python apps feature sandboxing with a way to opt-in to special
permissions.
Perhaps, like the web, it could have a uniform distribution mechanism. I
suggest DNS.

I am suggesting Python compete with the web by implementing strong language
sandboxing.
I imagine a browser that can load regular web and "PyWeb."

A web browser is both a kernel and a VM for the web. The kernel interfaces
with the underlying OS: Linux, windows, MacOS.
I do not see "PyWeb" as a kernel, but I do see it as a VM. PyWeb would
merely provide a secure gatekeeper to the underlying operating system.
Like the Mac App Store, PyWeb could give each app its own sandboxed file
system.
This would also help introduce young people to Python. Like how the
DevTools console has taught many kids JavaScript.
I imagine being able to run Tensorflow or Calibre inside a Python "browser."

>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4EKYH5T3MZINO2Y7MPH7MTCADEMRE45T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python, Be Bold!

[Chris Angelico]

On Sun, Jan 5, 2020 at 7:19 PM James Lu  wrote:
>
> I use macOS, and using Python is very confusing.
>
> - Apple's bundled Python 2.7.
> - Anaconda (Python scientific stack package manager) Python and conda.
> - Homebrew (3rd party package manager for macOS) Python and pip.
> I also believe that there is a PSF Python installer, but I am not sure.

Use Homebrew. It's a good package manager for the Mac and it gets you
away from Apple's ancient version of Python.

> Python, Be Bold captures the spirit of it should not be a shame to have the
> interpreter/VM installed on end-users machines. It also facilitates the
> work of other python devs. You installed it because of one Python program,
> other programs benefit from it. It also proposes enhancements to the VM to
> better facilitate that.

I still don't understand this concept of it "not being a shame" to
have Python installed. How is that different from the way it now is?
You install Python. Now Python is installed. How does the VM need to
be enhanced to change this? Other than *not* bundling Python with your
application?

> One big reason the web is a popular platform today among the general public
>  is because it offers strong sandboxing,
> and privelges are granted per-site, opt out by default.
> Another reason the web is popular is because it loads quickly.
> I suggest python apps feature sandboxing with a way to opt-in to special 
> permissions.
> Perhaps, like the web, it could have a uniform distribution mechanism. I 
> suggest DNS.

Eh?

> I am suggesting Python compete with the web by implementing strong language 
> sandboxing.
> I imagine a browser that can load regular web and "PyWeb."
>
> A web browser is both a kernel and a VM for the web. The kernel interfaces 
> with the underlying OS: Linux, windows, MacOS.
> I do not see "PyWeb" as a kernel, but I do see it as a VM. PyWeb would merely 
> provide a secure gatekeeper to the underlying operating system.
> Like the Mac App Store, PyWeb could give each app its own sandboxed file 
> system.
> This would also help introduce young people to Python. Like how the DevTools 
> console has taught many kids JavaScript.
> I imagine being able to run Tensorflow or Calibre inside a Python "browser."

You can already run Python code in a web environment. There are a
number of web sites that allow this, including pythontutor.com (which
also does detailed analysis and visualization, so it's not JUST
Python-in-the-web), and if you want to, you can compile PyPy to Asm.js
and run the entire interpreter right there in the browser. But what's
the point? How does that make it easier to do anything? You just have
to live within the restrictions of either a browser tab, or the things
you can do remotely.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EXOVTEYY4TOBH4MGJ7C6ZOOEOI2TASQ7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python, Be Bold!

[Andrew Barnert via Python-ideas]

On Jan 5, 2020, at 00:17, James Lu  wrote:
> 
> 
> I use macOS, and using Python is very confusing.
> 
> - Apple's bundled Python 2.7.

Apple has made a mess of things, but they’ve actually fixed that mess in 
10.15—they now give you 3.7 and 2.7, and neither one is broken or weird.

That being said, I don’t think anyone trusts them to not to change their minds 
again and leave everything without updates for half a decade to the point where 
their pre-installed Python is more of a nuisance than helpful. So everyone is 
still using Homebrew or Anaconda, and you probably should as well.

I think there’s more reason than ever for developers to learn virtual 
environments early and never have to think about all the different Pythons they 
have installed except whole setting up a new env.

But you’re apparently concerned about end users, not devs here. So:

> Python, Be Bold captures the spirit of it should not be a shame to have the
> interpreter/VM installed on end-users machines. It also facilitates the
> work of other python devs. You installed it because of one Python program,
> other programs benefit from it. It also proposes enhancements to the VM to
> better facilitate that.

There’s already a better solution for end users today. You don’t install Python 
to run Spam, you just `brew install spam`, and if Spam requires Python 3.7 and 
you don’t have it, Homebrew installs it for you. And if you just `brew install 
eggs` and Eggs also requires 3.7 and you’ve already got it from installing 
Spam, it uses the same 3.7. You don’t even have to know you have Python 
installed, much less know which versions you have. Of course this means someone 
has to build a Homebrew recipe—and RPM and DEB and Portage and Choco—for every 
application, but people have already been doing that for decades, and it works.

For users who do want to know they have Python installed, there are a lot of 
“power user” apps that they can install the same way they do libraries: `pip 
install cheese`. This also already works today.

I suppose it would be even better if the whole world used one single package 
manager that worked the same on every platform, and pip/setuptools could just 
be a trivial way to hook into that rather than an entirely separate system. But 
I don’t see any way to solve that from inside the Python ecosystem. The closest 
you can get is probably Anaconda—which already exists and already works fine.

> One big reason the web is a popular platform today among the general public
> 
>  is because it offers strong sandboxing,

The web was popular long before it offered strong sandboxing. That sandboxing 
had to be wedged in after the fact, because the web had become so popular that 
millions of people who didn’t know how to deal with security issues were 
visiting dynamic websites with IE4 and Netscape and getting hacked, tracked, 
scammed, and spammed.

> and privelges are granted per-site, opt out by default.
> Another reason the web is popular is because it loads quickly.
> I suggest python apps feature sandboxing with a way to opt-in to special 
> permissions.

How? In CPython, unlike the browser, you have direct access to the filesystem, 
and the sockets layer. You even have hooks to load any dylib/so/dll and call 
functions out of it. So it can only be sandboxed the same way C can be 
sandboxed. Which means only the OS can do it.

There have been attempts to build a useful environment out of Python by 
removing all of those features, and then adding in other, more restricted ways 
of doing the things you really need to do (access sandboxed local storage, talk 
to web services, accept connections as a web service, etc.).

The most successful is probably Google App Engine. But GAE Python doesn’t feel 
the same as Python. It can’t do a lot of the things you can do with a normal 
Python install. And with version 2.0, Google scrapped all of that and went back 
to building things around a normal Python instead.

If you wanted to do the same thing for end users instead of web services, you’d 
need to provide some way to run a GUI, to accept local files by some mechanism 
like drag&drop, etc. At which point you’re just designing a web browser. You 
might as well just write a Python interpreter that runs in the browser, and 
then add hooks to talk to the DOM and do WebSockets and make AJAX requests and 
so on the same way JS does. And there are already multiple projects to do that.

> Perhaps, like the web, it could have a uniform distribution mechanism. I 
> suggest DNS.

Why not just use URIs the same way browser JS and most of the existing browser 
Python projects do? What benefit do you get from building a custom package 
manager when every app has to be sandboxed and therefore can’t share packages 
with any other app?

> I am suggesting Python compete with the web by implementing strong language 
> sandboxing. 
> I imagine a browser that can load regular web and "PyWeb." 
> 
> A web browser is both a kernel and a VM fo

[Python-ideas] Re: Python, Be Bold!

[Ronald Oussoren via Python-ideas]

> On 5 Jan 2020, at 22:41, Andrew Barnert via Python-ideas 
>  wrote:
> 
> On Jan 5, 2020, at 00:17, James Lu  wrote:
>> 
>> 
>> I use macOS, and using Python is very confusing.
>> 
>> - Apple's bundled Python 2.7.
> 
> Apple has made a mess of things, but they’ve actually fixed that mess in 
> 10.15—they now give you 3.7 and 2.7, and neither one is broken or weird.

Do they?  The only python3 I’ve found is part of Xcode with a stub executable 
in /usr/bin (just like with clang).  I wouldn’t rely on the bundled 
installation of Python for anything important.

> 
> That being said, I don’t think anyone trusts them to not to change their 
> minds again and leave everything without updates for half a decade to the 
> point where their pre-installed Python is more of a nuisance than helpful. So 
> everyone is still using Homebrew or Anaconda, and you probably should as well.

Or the python installer on python.org :-)

Ronald
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DGEJQSZIUENGMTIDBBBQ3KXURZHVM6NG/
Code of Conduct: http://python.org/psf/codeofconduct/