On Sun, Jul 14, 2013 at 2:10 PM, Noah Kantrowitz <n...@coderanger.net> wrote: > > On Jul 14, 2013, at 10:43 AM, Noah Kantrowitz wrote: > >> >> On Jul 14, 2013, at 10:39 AM, Noah Kantrowitz wrote: >> >>> >>> On Jul 14, 2013, at 10:31 AM, Ian Cordasco wrote: >>> >>>> On Sun, Jul 14, 2013 at 1:12 PM, Noah Kantrowitz <n...@coderanger.net> >>>> wrote: >>>>> >>>>> On Jul 14, 2013, at 9:45 AM, Steve Dower wrote: >>>>> >>>>>> From: Paul Moore >>>>>>> On 13 July 2013 10:05, Paul Moore <p.f.mo...@gmail.com> wrote: >>>>>>> How robust is the process of upgrading pip using itself? Specifically on >>>>>>> Windows, where these things typically seem less reliable. >>>>>>> >>>>>>> OK, I just did some tests. On Windows, "pip install -U pip" FAILS. The >>>>>>> reason >>>>>>> for the failure is simple enough to explain - the pip.exe wrapper is >>>>>>> held open >>>>>>> by the OS while it's in use, so that the upgrade cannot replace it. >>>>>>> >>>>>>> The result is a failed upgrade and a partially installed new version of >>>>>>> pip. In >>>>>>> practice, the exe stubs are probably added fairly late in the install >>>>>>> (at least >>>>>>> when installing from sdist, with a wheel that depends on the order of >>>>>>> the files >>>>>>> in the wheel), so it's probably only a little bit broken, but "a little >>>>>>> bit >>>>>>> broken" is still broken :-( >>>>>>> >>>>>>> On the other hand, "python -m pip install -U pip" works fine because it >>>>>>> avoids >>>>>>> the exe wrappers. >>>>>>> >>>>>>> There's a lot of scope for user confusion and frustration in all this. >>>>>>> For >>>>>>> standalone pip I've tended to recommend "don't do that" - manually >>>>>>> uninstall and >>>>>>> reinstall pip, or recreate your virtualenv. It's not nice, but it's >>>>>>> effective. >>>>>>> That sort of advice isn't going to be realistic for a pip bundled with >>>>>>> CPython. >>>>>>> >>>>>>> Does anyone have any suggestions? >>>>>> >>>>>> Unless I misunderstand how the exe wrappers work (they're all the same >>>>>> code that looks for a .py file by the same name?) it may be easiest to >>>>>> somehow mark them as non-vital, such that failing to update them does >>>>>> not fail the installer. Maybe detect that it can't be overwritten, >>>>>> compare the contents/hash with the new one, and only fail if it's >>>>>> changed (with an instruction to use 'python -m...')? >>>>>> >>>>>> Spawning a separate process to do the install is probably no good, since >>>>>> you'd have to kill the original one which is going to break command line >>>>>> output. >>>>>> >>>>>> MoveFileEx (with its copy-on-reboot flag) is off the table, since it >>>>>> requires elevation and a reboot. But I think that's the only supported >>>>>> API for doing a deferred copy. >>>>>> >>>>>> If Windows was opening .exes with FILE_SHARE_DELETE then it would be >>>>>> possible to delete the exe and create a new one by the same name, but I >>>>>> doubt that will work and in any case could not be assumed to never >>>>>> change. >>>>>> >>>>>> So unless the exe wrapper is changing with each version, I think the >>>>>> best way of handling this is to not force them to be replaced when they >>>>>> have not changed. >>>>> >>>>> The usual way to do this is just move the existing executable to >>>>> pip.exe.deleteme or something, and then write out the new one. Then on >>>>> every startup (or maybe some level of special case for just pip >>>>> upgrades?) try to unlink *.deleteme. Not the simplest system ever, but it >>>>> gets the job done. >>>> >>>> I accidentally only emailed Paul earlier, but why can't we upgrade the >>>> pip module with the exe and then replace the process (using something >>>> in the os.exec* family) with `python -m pip update-exe` which could >>>> then succeed since the OS isn't holding onto the exe file? I could be >>>> missing something entirely obvious since I haven't developed >>>> (directly) on or for Windows in at least 5 years. >>> >>> Unfortunately windows doesn't actually offer the equivalent of a POSIX >>> exec(). The various functions in os don't actually replace the current >>> process, they just create a new one and terminate the old one. This means >>> the controlling terminal would see the pip process as ended, so it makes >>> showing output difficult at best. >> >> Check that, maybe I'm wrong, does anyone know if the P_OVERLAY flag unlocks >> the original binary? /me drags out a windows VM … > > Ignore my ignoring, with os.execl command flow does return back to the > controlling terminal process (the new process continues in the background) > and with os.spawnl(os.P_OVERLAY, 'python-2') I just get a segfault on 3.3. > Yay for not completely misremembering, boo for this being so complicated.
I expected I was wrong, but I appreciate you looking at it to be certain. _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
