[issue34977] Release Windows Store app containing Python

[Alexey Izbyshev]

Change by Alexey Izbyshev :


--
pull_requests: +16379
pull_request: https://github.com/python/cpython/pull/5812

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[miss-islington]

miss-islington  added the comment:


New changeset 9bb306d586e3f1a48db40bd9519412de4fff3ee8 by Miss Islington (bot) 
in branch '3.7':
bpo-34977: Remove unused preprocessor definition (GH-11092)
https://github.com/python/cpython/commit/9bb306d586e3f1a48db40bd9519412de4fff3ee8


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[miss-islington]

Change by miss-islington :


--
pull_requests: +10325

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:


New changeset d5a6a389d492c5e3d7933bafbd5252fd86ac4d49 by Steve Dower in branch 
'master':
bpo-34977: Remove unused preprocessor definition (GH-11092)
https://github.com/python/cpython/commit/d5a6a389d492c5e3d7933bafbd5252fd86ac4d49


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

Luckily, just noticed that I introduced a bug as part of splitting up the 
original PR into two.

It wasn't going to have an impact until the package was properly installed from 
the store, but without the proper directory resolution in there (since the 
variable I removed is never defined now), in some scenarios it wouldn't be 
possible to launch Python from a process outside of the app package, such as a 
virtual environment.

Testing in a proper app container is difficult, but I'll see if I can come up 
with a way to do it. It might require a custom buildbot, so I'll see whether 
there's a way I can do that (historically we've not been able to wire internal 
work machines - including Azure hosted ones - into other CI systems, but I'll 
try again).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +10324

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:


New changeset 0e4ad88ff8956d9289ab0a1314636ac15b374459 by Steve Dower in branch 
'3.7':
bpo-34977: Add Windows App Store package (GH-11027)
https://github.com/python/cpython/commit/0e4ad88ff8956d9289ab0a1314636ac15b374459


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:


New changeset 0cd6391fd890368ea1743dac50c366b42f2fd126 by Steve Dower in branch 
'master':
bpo-34977: Add Windows App Store package (GH-11027)
https://github.com/python/cpython/commit/0cd6391fd890368ea1743dac50c366b42f2fd126


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

Thanks, Paul. Appreciate it :)

Doing a final call before I hit merge in an hour or so. Right now:
* full release build works
* full test pass on all CI + custom buildbots work
* prior test package made it to the store
* docs clearly show it may change again

Any last pushback?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

Paul (and anyone else) - the below link should go directly to just the commit 
with the docs update. I did a slight rearrangement of the install docs to make 
the options clearer, and wrote up *just* enough info on nuget to help people 
use it right (I hope). Similarly for the Store package.

Any comments appreciated. Still aiming to get this in for 3.7.2rc1 today.

https://github.com/python/cpython/pull/11027/commits/7d7ddbc41bd30690de8c6227cebc41c4e0f3cf88

I've also kicked off a custom buildbot run against the 3.7 version of the 
change. The version in master ran earlier and was clean. Plus I'm running test 
builds on the release machine, so it should be ready to go.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Jeremy Kloth]

Jeremy Kloth  added the comment:

See also bpo-35450: venv module doesn't create a copy of python binary by 
default

--
nosy: +jkloth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:


New changeset b264c609853eae9dbb45c6dbee11e84ae3927e88 by Steve Dower in branch 
'3.7':
[3.7] bpo-34977: Use venv redirector instead of original python.exe on Windows 
(GH-11029)
https://github.com/python/cpython/commit/b264c609853eae9dbb45c6dbee11e84ae3927e88


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:


New changeset 1c3de541e64f75046b20cdd27bada1557e550bcd by Steve Dower in branch 
'master':
bpo-34977: Use venv redirector instead of original python.exe on Windows 
(GH-11029)
https://github.com/python/cpython/commit/1c3de541e64f75046b20cdd27bada1557e550bcd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

Okay, I'll merge the first part now so I can rebase the main one and 
*hopefully* someone will be willing to review it now it's smaller. Otherwise, I 
think we're best to go with buildbots and all the testing I've already done.

This is my sole job today, so I'll keep an eye on the buildbots. I'll also get 
the docs updated as Paul suggested for the main PRs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:

Sorry, I don't have the bandwidth to provide a proper review.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

I'm just waiting on Victor to sign off on PR 11029 (or someone else to do it) 
so I can merge it and rebase PR 11027.

Hopefully then someone will review it today so that we don't have to hold up 
3.7.2rc1 any longer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

> In which case, let's have a placeholder note in the docs, saying that
> the Windows Store installer exists, but it's experimental and may have
> unexpected interactions with other tools that expect a standard Python
> install. Users trying the Windows Store installer should be prepared
> for such issues, and we'd appreciate reports so that we can address
> them. Add that to the release announcement as well for extra
> visibility.

Sure, but first I need PR 11029 (and 11030 the backport) approved, so that I 
can rebase and work on redoing the other half of the change. Victor seems to 
have disappeared for the weekend, but I made all his requested changes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Paul Moore]

Paul Moore  added the comment:

> Yeah, and these are all essentially my fault (though I documented the 
> embeddable package relatively well, modulo a few uses I hadn't imagined). 
> Maybe my holiday writing project can be some better docs for choosing how to 
> install Python, especially since if/when this merges we'll finally have a 
> good answer for "installing from a build without the installer" (using the 
> PC/layout script, which *sigh* also needs a big fat manual).

Sorry about that - I wasn't trying to make you feel guilty! And yes,
the embeddable package docs are very good. The problem with that one
is mostly that people don't *read* the docs (but having them to point
to makes responding to pip issues much easier, so you have my
gratitude there :-)

> I feel like right now there's enough here that isn't quite locked down and I 
> don't want to guarantee it. For example, in a future release of Windows we 
> may be able to enable it to write to the actual %APPDATA% folder instead of 
> an app-private one - should we document the current paths and then "break" it 
> in the future? Or should we document that using the "Reset app" features will 
> clean up everything fully now, knowing that if we change the %APPDATA% 
> location later then _that_ will break?
>
> So for now, I want an unguaranteed store package that can be used but may 
> change over time (and I'll update the text on the page to reflect that). Now 
> is not the time to lock in all aspects of how it works, exactly as you say. 
> But please hold me accountable for adding specific docs for the first 3.8 
> release - hopefully it's clear by then (if not earlier) what behavior we're 
> happy with.

In which case, let's have a placeholder note in the docs, saying that
the Windows Store installer exists, but it's experimental and may have
unexpected interactions with other tools that expect a standard Python
install. Users trying the Windows Store installer should be prepared
for such issues, and we'd appreciate reports so that we can address
them. Add that to the release announcement as well for extra
visibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

> In general, I don't think we've done that great a job of publicising
> the various install options on Windows - we still get pip users trying
> to install packages into the embedded distribution, which they are
> using as if it were a standalone interpreter, and as far as I can tell
> essentially no-one knows about the nuget packages for Python. Let's
> not repeat the mistakes we've made previously with this new
> distribution option.

Yeah, and these are all essentially my fault (though I documented the 
embeddable package relatively well, modulo a few uses I hadn't imagined). Maybe 
my holiday writing project can be some better docs for choosing how to install 
Python, especially since if/when this merges we'll finally have a good answer 
for "installing from a build without the installer" (using the PC/layout 
script, which *sigh* also needs a big fat manual).

I feel like right now there's enough here that isn't quite locked down and I 
don't want to guarantee it. For example, in a future release of Windows we may 
be able to enable it to write to the actual %APPDATA% folder instead of an 
app-private one - should we document the current paths and then "break" it in 
the future? Or should we document that using the "Reset app" features will 
clean up everything fully now, knowing that if we change the %APPDATA% location 
later then _that_ will break?

So for now, I want an unguaranteed store package that can be used but may 
change over time (and I'll update the text on the page to reflect that). Now is 
not the time to lock in all aspects of how it works, exactly as you say. But 
please hold me accountable for adding specific docs for the first 3.8 release - 
hopefully it's clear by then (if not earlier) what behavior we're happy with.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Paul Moore]

Paul Moore  added the comment:

Thanks for the clarification.

> In all the cases I've tested, regular Python install shows up first. If it's 
> not installed, the Store app shows up much of the time. This feels like the 
> right design (with the caveat about "much of the time" requiring changes 
> outside of our direct control), but without actually giving it to people in 
> all their messy configurations there is literally no way to be sure.

Agreed, it does sound like it's probably the correct approach. But I'd
strongly recommend that as much of the behaviour as possible gets
documented as part of this change. There are a *lot* of projects out
there that write code for Windows based on a relatively superficial
understanding of all of the complexities that come into play here -
most registry scanning code is based on guesswork and reverse
engineering of how things worked before PEP 514 (I should know, I
wrote rather a lot of it!!!) and plays fast and loose with even things
like 32-bit/64-bit registry overlaying (it's late and I'm tired, sorry
for not using the accurate term). Settig projects like virtualenv,
tox, nox, pew, pipenv etc up for new bug reports associated with store
installs, without any sort of guidance as to what's going on in the
official docs, seems unfriendly at best. (I'm thinking of a section in
"Python Setup and Usage" covering the store installer - that in
addition to the changes in
https://docs.python.org/3.7/using/windows.html#installation-steps
needed to mention that there's now five installers available).

> A regular install does not create "python3" and "python3.7" shortcuts on 
> PATH, so if you use those you'll prefer the store app. But we've never used 
> those, so there shouldn't be any change to existing behaviour. And I've never 
> really been opposed to adding them to the Python installer, I just haven't 
> done the work (and neither has anyone else) - I'd still be opposed to 
> modifying PATH from the traditional installer though, as all of those 
> problems remain.

The "pythonup" project (https://github.com/uranusjr/pythonup-windows)
creates versioned copies of python - python3.7.exe. Again, it's a
non-standard project, so not technically our problem, but I'd be
interested in how it would interact with the Store python.

I do appreciate that we need to get the store installer out there in
order to test things like this, and that initially at least there
aren't going to be a huge number of people using the store installer
over the traditional one (and likely vanishingly small numbers of
people using both in parallel). But conversely, there's a class of
problems that will be reported to other projects rather than to us,
and I think we should be trying to support those other projects in
dealing with this change, not simply leaving it up to them to find out
about it (I doubt they'd even know to ask "did you use the store
install or the traditional one" when diagnosing a problem).

In general, I don't think we've done that great a job of publicising
the various install options on Windows - we still get pip users trying
to install packages into the embedded distribution, which they are
using as if it were a standalone interpreter, and as far as I can tell
essentially no-one knows about the nuget packages for Python. Let's
not repeat the mistakes we've made previously with this new
distribution option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Ethan Smith]

Change by Ethan Smith :


--
nosy: +Ethan Smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

> How will the Windows Store install work alongside the PSF one?

Oh, and since I didn't mention it in my direct answer, the Store install goes 
to a totally different location (per machine, no elevation required), as do 
packages installed using pip (automatically per user). Nothing is shared 
between the two installs at all.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

> The first thing I'll note is that I'm not aware of any documentation
> on the Windows Store installer, and how it works.

Right now, correct.

> How will the Windows Store install work alongside the PSF one?

It will go into PATH under the same names as on other platforms (python, 
python3, python3.7, and same for pip and idle). The Windows Store makes it 
possible to do this reliably, so it's just on by default (and configurable 
through OS-provided UI for selecting the app to respond to a certain name).

If you have already installed it traditionally and enabled the PATH 
modification, that will take precedence. So you'll get the MSI installed 
Python. If you've modified PATH by hand, it depends on where you put it.

> If I have PSF Python 3.7.1 and Store Python
> 3.7.2 simultaneously installed, which will py.exe -3.7 select? Can I
> influence it to choose the other one? Same question for 3rd party
> tools like "virtualenv -p 3.7"

Right now, py.exe will not detect it. This is an unavoidable limitation right 
now (specifically, enumeration of classic app compatibility keys does not work, 
though direct access does), but it may become avoidable in the future (and 
being able to say "Python needs it" is actually going to be useful for removing 
the limitation).

As usual, our recommended way to influence choice of Python is to specify it. 
"virtualenv -p 3.7" will find a traditionally installed Python first, but if 
it's not there and the store one is it *might* find the store one (it depends 
whether it requests the specific registry key or tries to enumerate all of them 
- again, I've got people looking into enabling enumeration, but we're talking 
about adding another feature to Windows specifically for Python, and those take 
time).

> Basically, I'm not clear on how having the Store install and the PSF
> install simultaneously present will work - very few tools in my
> experience really support PEP 514 (not even the `py` launcher) - and
> frankly, if we're going to break tool assumptions that there's only
> one official "3.7" present at any time, we're on very shaky ground
> claiming that PEP 514 warned them if we haven't even updated our own
> supplied tool! (I know that there's lots of ways people can have
> multiple versions right now - PSF plus Anaconda, for example - but
> it's fine to say "you're mixing distributions, you're on your own" in
> that case. If the PSF is publishing two versions of 3.7, which can be
> simultaneously installed, we can't really use that argument.

In all the cases I've tested, regular Python install shows up first. If it's 
not installed, the Store app shows up much of the time. This feels like the 
right design (with the caveat about "much of the time" requiring changes 
outside of our direct control), but without actually giving it to people in all 
their messy configurations there is literally no way to be sure.

A regular install does not create "python3" and "python3.7" shortcuts on PATH, 
so if you use those you'll prefer the store app. But we've never used those, so 
there shouldn't be any change to existing behaviour. And I've never really been 
opposed to adding them to the Python installer, I just haven't done the work 
(and neither has anyone else) - I'd still be opposed to modifying PATH from the 
traditional installer though, as all of those problems remain.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Paul Moore]

Paul Moore  added the comment:

> > Steve, assuming that I can access the Store, should I be able to install 
> > both PSF 3.7.2 and Windows Store 3.7.2 simultaneously?  So I could test 
> > both IDLEs that beginners might install?
>
> Yes, absolutely. (To be clear, both are PSF approved and under the PSF 
> name/copyright/etc., so I've been calling them "traditional" and "Store" 
> installers when I talk about them.)

I'd like to expand on this question a little. The first thing I'll
note is that I'm not aware of any documentation on the Windows Store
installer, and how it works. It's possible there's a docs patch in
here somewhere, but to be honest, there are 8 PRs linked to this issue
and I don't have the time to check them all right now.

How will the Windows Store install work alongside the PSF one? Will it
work the same, in that neither will go on PATH unless the user
explicitly requests it? If I have PSF Python 3.7.1 and Store Python
3.7.2 simultaneously installed, which will py.exe -3.7 select? Can I
influence it to choose the other one? Same question for 3rd party
tools like "virtualenv -p 3.7" - I know it's not up to us to deal with
virtualenv issues, but conversely I do think we have a responsibility
to not change assumptions that 3rd party tools rely on. In terms of
PEP 514, will the Store installer use a different tag, or some other
way of having registry entries independent of the PSF installer?

Basically, I'm not clear on how having the Store install and the PSF
install simultaneously present will work - very few tools in my
experience really support PEP 514 (not even the `py` launcher) - and
frankly, if we're going to break tool assumptions that there's only
one official "3.7" present at any time, we're on very shaky ground
claiming that PEP 514 warned them if we haven't even updated our own
supplied tool! (I know that there's lots of ways people can have
multiple versions right now - PSF plus Anaconda, for example - but
it's fine to say "you're mixing distributions, you're on your own" in
that case. If the PSF is publishing two versions of 3.7, which can be
simultaneously installed, we can't really use that argument.

Paul

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

Thanks Terry, great questions.

> I want this to be and remain an added alternative rather than a replacement 
> of the current installer directly available on python.org.  Can you assure us 
> that the latter are not going away?

Absolutely. The current installer is not going anywhere, and there's no point 
considering the possibility until we've completely dropped support for Windows 
10 1803 and earlier (which was only released this year, so dropping support is 
a long way out yet).

> Installers are always experimental until tested by multiple users on multiple 
> machines.  Even then, installation specific issues can arise. ...  To have a 
> Windows Store Python ready for 3.8, we need to start now.

Thanks for the vote of support for this point.

> Steve, assuming that I can access the Store, should I be able to install both 
> PSF 3.7.2 and Windows Store 3.7.2 simultaneously?  So I could test both IDLEs 
> that beginners might install?

Yes, absolutely. (To be clear, both are PSF approved and under the PSF 
name/copyright/etc., so I've been calling them "traditional" and "Store" 
installers when I talk about them.)

Also, I've been testing IDLE regularly, since one of the advantages of being in 
the Store is that we can safely put idle[3[.7]].exe on PATH. It definitely 
works :)

I've also set up tests that run from an "installed" layout, rather than the 
source tree, and those are all passing as well (with one of the trivial test 
fixes in the PR).

> I am curious what 'restricted' versus 'non restricted' mean to Windows Store, 
> and why Python can now go there.

In general, all app stores disallow their apps from doing things that may break 
or corrupt the machines they are installed on. Apps are self-contained and can 
only operate outside of their boundaries through carefully designed 
permissions/capabilities (which often prompt the user before continuing).

Recently, Microsoft added support for "full trust" apps, which can bypass many 
of these restrictions. The aim is to allow traditional applications to swap 
their MSIs for being installed through the store, which can then ensure 
trustworthiness of at least the installers. (For example, a Store app is 
*never* going to install something into System32 or modify operating system 
files, whether it's full trust or not.) Full trust allows us to do things like 
launching non-app store processes and creating raw sockets through legacy APIs, 
so we can very easily move Python into a full trust package rather than having 
to rewrite it using approved APIs (a project that's been attempted a few times, 
but always loses low-level functionality that is considered a core part of 
Python - hence my long-term desire to make the low-level parts optional and 
have higher-level abstractions be the guaranteed parts).

> While most PRs should be complete in themselves, reasons such as ease of 
> review may suggest splitting one conceptual Pr into pieces.  They just have 
> to be reviewed together, or with expectation of a needed followup (and a 
> revert if it does not appear).  I have done this to get readable diffs for 
> major refactorings that moved and edited multiple chunks of code in a large 
> file.

I agree that it's valuable and I've done it internally plenty of times. 
Unfortunately, no matter how much people repeatedly say that a PR for CPython 
is part of a chain, there always seems to be pushback from people who ignore 
that and won't approve a critical prerequisite because there's "no need".

There's also the point that many of the changes were not known to be needed 
before the others were made. The pre-squash history is very interleaved between 
the various changes, and extracting them into separate PRs that totally stand 
alone is nearly as much work as solving the issues in the first place. I made 
the call this time to keep them all together and direct reviewers' attention 
towards the relevant parts. It turns out that didn't attract reviewers, so I'll 
come up with a new strategy next time I want to get people's attention on a 
change (maybe Buzzfeed style headlines? :) )

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Various questions and comments:

0. Steve: For more than one reason, I want this to be and remain an added 
alternative rather than a replacement of the current installer directly 
available on python.org.  Can you assure us that the latter are not going away? 
 I would have the same concern about adding something to the Apple/Mac store.

1. Ned: Since enhancements are not normally backported, it would be nice to 
have your approval, in whatever form it took, recorded here and not just in 
private email.  I give a reason below.

2. Installers are always experimental until tested by multiple users on 
multiple machines.  Even then, installation specific issues can arise.  As I 
remember, the current installer, when new in 3.5, needed a shakedown period.  
One installer-specific issue was the initial working directory.  To have a 
Windows Store Python ready for 3.8, we need to start now.

There was once an issue where one of the installers failed to include one 
idlelib file. I believe it was help.html. I since added idle_test/test_help.py, 
which would now fail with something like "No such file or directory: 
'C:\\Programs\\Python37\\Lib\\idlelib\\help.html'".)
Repository testing does not test the actual binaries that users install, in 
their production context.

Steve, assuming that I can access the Store, should I be able to install both 
PSF 3.7.2 and Windows Store 3.7.2 simultaneously?  So I could test both IDLEs 
that beginners might install?

3. Anyone: Though it is not essential that I know, I am curious what 
'restricted' versus 'non restricted' mean to Windows Store, and why Python can 
now go there.  My search only came up with age restrictions, which python does 
not need (in my opinion).

4. While most PRs should be complete in themselves, reasons such as ease of 
review may suggest splitting one conceptual Pr into pieces.  They just have to 
be reviewed together, or with expectation of a needed followup (and a revert if 
it does not appear).  I have done this to get readable diffs for major 
refactorings that moved and edited multiple chunks of code in a large file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +10266

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +10265

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

>> We haven't done RC yet - why is pushing this now so bad?
> I reverted your change because it broke all Windows buildbots, and we have a 
> policy to revert a change if buildbots turn red and no fix can be found in 2 
> hours

I don't dispute that policy, but you said it shouldn't go in between RC and a 
release, which it isn't.

> nitpicking: Your change modify venv and two tests, it's not purely 
> Windows-only :-)

More nitpicking: the venv changes only affect Windows. Which makes it purely 
Windows and tests, exactly as I said. Adding smiles doesn't save you from 
having to read :)

> Sorry, I don't know Windows App Store, I don't understand the restriction 
> "there has to be a release of this or it remains untestable". Would you mind 
> to elaborate?

Sure. You can start by reading this thread, but the summary is that App Store 
apps (regardless of platform) generally have to go through an approval process 
before they can be installed. Without going through this process, they can't be 
tested fully in context. I believe we should make a true release and put that 
through the process, rather than me just doing one out of my branch.

I provided instructions for developer testing above, which covers about 80% of 
what needs to be checked. The rest can only be tested by real use of a proper 
release.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +10264

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +10263

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:

Since the first attempt broke all Windows buildbot workers, it would be nice to 
use a custom build before merging a new attempt :-) See instructions in the 
devguide:
https://devguide.python.org/buildbots/?highlight=custom#custom-builders

Note: Pablo used buildbot-custom to test a revert of the App Store change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:

> We haven't done RC yet - why is pushing this now so bad?

I reverted your change because it broke all Windows buildbots, and we have a 
policy to revert a change if buildbots turn red and no fix can be found in 2 
hours:
https://pythondev.readthedocs.io/ci.html#revert-on-fail


> Most of the change is purely additive and does not modify the code base, the 
> rest is Windows-only or tests.

nitpicking: Your change modify venv and two tests, it's not purely Windows-only 
:-) By the way, as I wrote in my email, I would prefer to see the venv+tests 
change pushed in separated change (and explain why you do these changes).


> I'll add checks for outdated build machines, but there has to be a release of 
> this or it remains untestable, so it'll either be from my branch or from the 
> release branch.

Sorry, I don't know Windows App Store, I don't understand the restriction 
"there has to be a release of this or it remains untestable". Would you mind to 
elaborate?

Do you mean that buildbots must upgrade to VS 2017? Can Python 3.6 and 3.7 be 
compiled without any issue on VS 2017? Maybe send an email to buildbot-status 
if you want to see more buildbots running VS 2017 rather than VS 2015.

--

There was a small bug in your PR, in PC/launcher.c, fixed by commit 
4c49da0cb7434c676d70b9ccf38aca82ac0d64a9 (bpo-35436):

+if (executable == NULL) {
+error(RC_NO_MEMORY, L"A memory allocation failed");
+}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

We haven't done RC yet - why is pushing this now so bad? Most of the change is 
purely additive and does not modify the code base, the rest is Windows-only or 
tests.

I'll add checks for outdated build machines, but there has to be a release of 
this or it remains untestable, so it'll either be from my branch or from the 
release branch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:

> Revert "[3.7] bpo-34977: Add Windows App Store package (GH-10245)" (GH-11021)

That should repair buildbots, so I remove "release blocker" priority.

--
priority: release blocker -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 783b794a5e6ea3bbbaba45a18b9e03ac322b3bd4 by Victor Stinner in 
branch '3.7':
Revert "[3.7] bpo-34977: Add Windows App Store package (GH-10245)" (GH-11021)
https://github.com/python/cpython/commit/783b794a5e6ea3bbbaba45a18b9e03ac322b3bd4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:

> I suggest to revert this change from Python 3.7: it broke all Windows 
> buildbots, see bpo-35437.

I reverted the change in master and I will merge PR 11021 once tests pass to 
revert in 3.7 as well.


> It doesn't seem like a good idea to push such change between a release 
> candidate and a final version :-(

I started a discussion on python-dev:
https://mail.python.org/pipermail/python-dev/2018-December/155932.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:


New changeset cb0b78a070ea3b704416e74f64046178ae0dff3e by Victor Stinner in 
branch 'master':
Revert "bpo-34977: Add Windows App Store package (GH-10245)" (GH-11019)
https://github.com/python/cpython/commit/cb0b78a070ea3b704416e74f64046178ae0dff3e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +10258

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

STINNER Victor  added the comment:

> Making the release experimental as part of the next 3.7 update was approved 
> by Ned (over email), so I merged the build.

I suggest to revert this change from Python 3.7: it broke all Windows 
buildbots, see bpo-35437. It doesn't seem like a good idea to push such change 
between a release candidate and a final version :-(

--
nosy: +ned.deily, vstinner
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +10255
stage: commit review -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

Making the release experimental as part of the next 3.7 update was approved by 
Ned (over email), so I merged the build. As soon as we snap for the RC I'll 
kick off an update and make the store page public, and hopefully can promote it 
enough to get eyes on it.

Unfortunately, I discovered as part of a test submission that the minimum 
Windows version matters, and it's a version that hasn't been rolled out fully 
yet because of some bugs, so there may not be that many people who can use it 
this first time. But that will improve over time, and I'm sure I can find 
enough people to flush out issues before the next release (anyone on the 
Windows Insiders program should be fine).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:


New changeset 253209149389e6793a052034e1f2d97691086f18 by Steve Dower in branch 
'3.7':
[3.7] bpo-34977: Add Windows App Store package (GH-10245)
https://github.com/python/cpython/commit/253209149389e6793a052034e1f2d97691086f18


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:


New changeset 468a15aaf9206448a744fc5eab3fc21f51966aad by Steve Dower in branch 
'master':
bpo-34977: Add Windows App Store package (GH-10245)
https://github.com/python/cpython/commit/468a15aaf9206448a744fc5eab3fc21f51966aad


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

Updated testing instructions - this one should work with just sideloading 
permissions I think.

iwr 
"https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34760/artifacts?artifactName=appx=5.0-preview.5&%24format=zip;
 -outfile python37.zip
Expand-Archive .\python37.zip .
Add-AppxPackage appx\python.msix

To remove:
Remove-AppxPackage (Get-AppxPackage *Python.3.7)
(delete the downloaded files)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
Removed message: https://bugs.python.org/msg330453

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

For my own reference, some issues I found installing this package on a clean 
machine:

* requires Developer Mode (until I get the sccd file signed - just sideloading 
is not sufficient)
* requires vcruntime140.dll to be included in the package (workaround is to 
install the redistributable from 
https://visualstudio.microsoft.com/downloads/#title-39324)
* venv needs to treat .pdb files as binary

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

For my own reference, some issues I found installing this package on a clean 
machine:

* requires Developer Mode (until I get the sccd file signed - just sideloading 
is not sufficient)
* requires vcruntime140.dll to be included in the package (workaround is to 
install the redistributable from 
https://visualstudio.microsoft.com/downloads/#title-39324)
* venv needs to treat .pdb files as binary

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

If anyone would like to try this out early, here's some instructions for a 
build I just made.

Enable sideloading apps on your machine (see 
https://docs.microsoft.com/en-us/windows/uwp/get-started/enable-your-device-for-development
 - may already be done, sideload or developer mode should do)

In a regular (non-admin) PowerShell prompt in a temporary directory, run these 
commands:

iwr 
"https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34540/artifacts?artifactName=appx=5.0-preview.5&%24format=zip;
 -outfile python37.zip
Expand-Archive .\python37.zip .
Add-AppxPackage appx\python.msix

(The first may take a while as it downloads approx. 80MB. I'm intending to 
leave the test suite out of the final package, which should bring it down 
closer to 40MB.)

Now play with it:
$> python.exe
$> python3.exe
$> python3.7.exe
$> pip.exe
$> pip3.exe
$> pip3.7.exe
$> idle.exe
$> idle3.exe
$> idle3.7.exe
(Or press Win+S and search for new entries)

To remove it:

Remove-AppxPackage (Get-AppxPackage *Python.3.7)
(delete the downloaded files)

Right now, the py.exe launcher can't find it, nor can any tools that rely on 
enumerating the registry - directly requesting the reg keys will return the 
right values though. VS Code seems to pick it up just fine though, I guess 
because it's on PATH.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +9932

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

For people watching the bug, this is ready for review.

Most of the change is adding a "layout" script (roughly the equivalent of "make 
install" for Windows, but with more flexibility to generate different 
structures, precompile stuff, etc.)

About the only CPython change is to venv. I added a new launcher variation that 
looks for pyvenv.cfg and runs the python.exe listed in that, so now we don't 
have to copy any DLLs into the venv at all (it's needed because the old one 
would try to LoadLibrary DLLs inside the store package, which is not allowed, 
but it also fixes my #1 complaint about venv which is that upgrading the base 
Python breaks all venvs).

There are also a few tweaks to the startup process, including extending the 
existing __PYVENV_LAUNCHER__ env variable from macOS to Windows as well, though 
it's handled in a different location. And a couple of minor tweaks to tests - 
issues discovered now that it's pretty easy to generate a "real" layout to run 
tests in.

I can break this up into a couple of PRs if that's preferred, though it doesn't 
really gain anything. If you're not interested in the new layout script, stop 
reading the PR after you finish launcher.c :)

--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +9559
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

(And I'll deal with that 1% at the PyPA sprints next weekend. We are due for a 
new pip insertion anyway, so I guess that'll come for 3.7.2.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

Steve Dower  added the comment:

I'm 99% sure that with issue34725 I'll be able to make "pip --user" the default 
when using this install, as well as making the commands globally available 
(i.e. "python[3[.8]]", "pip[3[.8]]", "idle[3[.8]]" will work everywhere).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

+1 for reliable installation. There were some cases (issue33220, issue30944) 
where the installer was detected as a Virus by some vendors which would be 
reduced since this is installed from the App store as a verified app. I hope 
this will also reduce the scenarios where users need to resort to bug tracker 
with log files when there are installation failures [1] providing a more smooth 
one click install.

There is a long standing open issue [2] to make `pip install --user` as a 
default which I hope gets resolved and there is an issue with using `pip 
install --user` inside a virtualenv which seems to block the earlier issue [3]

Thanks a lot for this!

[1] 
https://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40sort=-activity&%40filter=status&%40action=searchid=file%3Acontent&%40search_text=windows+msi+install=search=-1%2C1%2C2%2C3

[2] https://github.com/pypa/pip/issues/1668

[3] https://github.com/pypa/pip/issues/5702

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

New submission from Steve Dower :

The Windows Store can now install unrestricted apps, which means we can provide 
the Python interpreter in there.

Advantages:
* far more reliable installation
* can have specific executables on PATH
* automatic updates
* other apps can act as extensions (with permission), so things like Mu could 
also become store apps in the future

Disadvantages:
* only --user installs for pip will work
* harder to find/modify installed files

I think for a (very significant) subset of our users, this will be a much 
better experience than downloading our current installer. It also has the 
advantage of making "Python" appear in searches in the start menu, which will 
link directly to a one-click "Install" button.

The biggest issue is likely to be pip not installing with --user by default, as 
well as subsequent issues with apps that require users to navigate to their 
%AppData% directories manually (since these will be redirected to different 
locations).

But until we get a package available for testing, it will be hard to figure out 
what needs fixing. Given this is just distribution and not a new platform, I 
plan to enable it for Python 3.7 (probably 3.7.2, but it may be possible to 
bundle 3.7.1 as-is for testing).

--
assignee: steve.dower
components: Windows
messages: 327679
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Release Windows Store app containing Python
type: enhancement
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com