[Python-Dev] PEP 446: Open issues/questions

[Victor Stinner]

Hi,

I have a few more questions on the PEP 446:

(A) How should we support support where os.set_inheritable() is not
supported? Can we announce that os.set_inheritable() is always
available or not? Does such platform exist?

(B) Should subprocess make the file descriptors of pass_fds
inheritable? If yes, should it be done before or after the fork? If it
is done after the fork and before exec, it only affects the child
process, at least on Linux (the file descriptor is still
non-inheritable in the parent process).

(C) Should we handle standard streams (0: stdin, 1: stdout, 2: stderr)
differently? For example, os.dup2(fd, 0) should make the file
descriptor 0 (stdin) inheritable or non-inheritable? On Windows,
os.set_inheritable(fd, False) fails (error 87, invalid argument) on
standard streams (0, 1, 2) and copies of the standard streams (created
by os.dup()).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding Python scripts to PATHEXT on Windows

[Steve Dower]

From: Paul Moore 
> I'm a bit confused by your comment about "people who prefer to associate 
> scripts
> with an editor", though. This is only done in precisely those situations when
> the installer associates .py scripts with py.exe (the launcher). So if you
> switch that off, you don't get PATHEXT either.

That's optional enough for me - I always disable that (because I typically have 
4-10 different versions installed at once).

> And if you change the association after the fact, you're presumably just as 
> capable
> of changing PATHEXT.

Not if the association is changed by another installer (presumably with the 
user's explicit permission). It would be very easy for people to go a long time 
without noticing this until some program does ShellExecute(...) on a name that 
happens to be a Python script and Visual Studio pops open...

>> Env var changes typically require a reboot to take full effect, which I've 
>> also seen cause confusion.
> Really? If I add python.exe to PATH in the installer do I need to reboot? (I
> never do this, so I don't know - it's a genuine question).

Env vars are inherited by child processes, so until you create a process 
without inheriting them, you won't see the new value. Depending on the shortcut 
used, cmd.exe is likely to inherit the variables from explorer.exe, which won't 
see the new ones until you log off/on again. Certainly any running prompts will 
need to be closed and restarted after installing.

That said, on my Win8 machine, cmd.exe seems to be okay after changing the 
variables. However, PowerShell did not see the change immediately, so cmd.exe 
may be getting special treatment (or just ignoring the default environment).

>> Also, it should include .pyw, .pyz and whatever was decided instead
>> of .pywz (.pwz, I think?).
>
> It's basically a command line thing - I don't see why you'd want to launch
> a GUI script from the command line in the same way.

That's fair enough, though it may be seen as inconsistent. TBH I've never 
actually seen anything in the wild other than .py.

My suspicion is that the .exe wrappers would remain for tools like pip anyway, 
rather than breaking people who choose to deselect that option. Of course, it's 
easy enough to replace them.

Another issue to consider is that the modification to PATHEXT can't be undone 
when Python is uninstalled, unless each installation adds another ".PY" and 
each uninstall removes only one (so my PATHEXT would look like 
...;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY). I haven't checked, but this could 
seriously hurt performance of anything that uses PATHEXT to search PATH for 
anything ending in this extension. I hope that cmd.exe and PowerShell check for 
duplicate extensions, but I wouldn't be betting any money that they do.


Steve


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding Python scripts to PATHEXT on Windows

[Paul Moore]

On 27 July 2013 21:14, Steve Dower  wrote:

>  Any chance of this being made optional when installing? It provides no
> benefit for people who prefer to associate scripts with an editor and may
> be a source of confusion/complaints.
>

Personally, I don't know how to do this so someone else would have to - but
if someone does, I won't particularly object. I'd prefer it to be on by
default, though (just like associating py.exe with scripts is) because
otherwise it's not reasonable to assume that a .py script will work as a
command in the same way that a bat file does - which is really the point of
wanting this in the first place.

I'm a bit confused by your comment about "people who prefer to associate
scripts with an editor", though. This is only done in precisely those
situations when the installer associates .py scripts with py.exe (the
launcher). So if you switch that off, you don't get PATHEXT either. And if
you change the association after the fact, you're presumably just as
capable of changing PATHEXT.

The key point here is that I want the *default* behaviour of Python when
installed using the standard msi to be that if I put a script foo.py on
PATH somewhere, then typing "foo" at the command line will run that script.
For that to be the case, PATHEXT needs to be set, and I view that as a
(minor) omission which was made when the association with py.exe was
implemented.

(Which reminds me - it would be nice if the standalone installer for the py
launcher included setting PATHEXT as well).


> Env var changes typically require a reboot to take full effect, which I've
> also seen cause confusion.
>

Really? If I add python.exe to PATH in the installer do I need to reboot?
(I never do this, so I don't know - it's a genuine question).

Also, it should include .pyw, .pyz and whatever was decided instead of
> .pywz (.pwz, I think?).
>

It's basically a command line thing - I don't see why you'd want to launch
a GUI script from the command line in the same way. But again, if someone
wants to do it, I don't really mind (in this case I know what to do, so if
there's consensus that it's needed, I can add it to the patch myself).

The PEP for pyz and pwz hasn't been accepted yet. If it does, then yes pyz
should be treated the same way (and we can have the same debate about pwz
as we do about pyw :-))

Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding Python scripts to PATHEXT on Windows

[Steve Dower]

Any chance of this being made optional when installing? It provides no benefit 
for people who prefer to associate scripts with an editor and may be a source 
of confusion/complaints.

Env var changes typically require a reboot to take full effect, which I've also 
seen cause confusion.

Also, it should include .pyw, .pyz and whatever was decided instead of .pywz 
(.pwz, I think?).

Steve

(Apologies for the top post.)
Sent from my Windows Phone

From: Paul Moore
Sent: ‎7/‎27/‎2013 1:13
To: Martin v. Löwis
Cc: Python Dev
Subject: Re: [Python-Dev] Adding Python scripts to PATHEXT on Windows

On 23 July 2013 17:33, Paul Moore 
mailto:p.f.mo...@gmail.com>> wrote:
On 23 July 2013 17:11, "Martin v. Löwis" 
mailto:mar...@v.loewis.de>> wrote:
Am 15.07.13 10:26, schrieb Paul Moore:
> Does anyone have any objections to this? I could try to write a patch,
> but I know next to nothing about building MSIs, so if any of the
> installer experts could help that would be fantastic.

It's fine with me. I could write the patch, but will likely forget - so
please remind me with every alpha that misses this. Also, contributions
are welcome.

Thanks.

I think I have the basic idea of what's needed, so I'll write an initial patch. 
If you can check it that'd be great.

OK, see http://bugs.python.org/issue18569

As I mention in the patch, I have yet to successfully manage to build an MSI 
installer (I tried a while ago, and the instructions didn't work - I'm not 100% 
sure why and I'm afraid I don't have the time right now to try again) so the 
patch will definitely need checking by someone who can build the MSI.

When I get some time (probably not for a few weeks) I'll try the "how to build 
the installer" instructions again and see if I can work out what went wrong and 
either fix the issue or report back.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Victor Stinner]

2013/7/27 Guido van Rossum :
> P.S. perhaps more important than a PEP rewrite is a working patch to see how
> realistic this is. Could you make the alpha 1 release?

I already ran the whole Python test suite with non-inheritable file
descriptors when I developed the PEP 433: it just works. So I'm
confident :-)

I "just" had to fix the cgi module, and some tests. For example,
test_socket checks the exact type of sockets, whereas SOCK_CLOEXEC
flag is present in sockobj.type for non-inheritable sockets created
with this flag.

I implemented the *new* PEP 446 (not written yet :-)) in a new repository:
http://hg.python.org/features/pep-446

I had to invert the value of cloexec (inheritable value is just the opposite).

The implementation works but it is not completed:

* The doc should be reviewed
* test_swap_fds() of test_subprocess fails
* The implementation should be tested on Windows, FreeBSD and Solaris
* I have to check if _Py_try_set_inheritable() can/must be replaced
with _Py_set_inheritable()

The implementation can be seen as a patch and reviewed using the
following new issue:
http://bugs.python.org/issue18571

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Guido van Rossum]

P.S. perhaps more important than a PEP rewrite is a working patch to see
how realistic this is. Could you make the alpha 1 release?

On Saturday, July 27, 2013, Guido van Rossum wrote:

>
>
> On Saturday, July 27, 2013, Victor Stinner wrote:
>
>> 2013/7/27 Guido van Rossum :
>> > Do we even need a new PEP, or should we just do it? Or can we adapt
>> > Victor's PEP 446?
>>
>> I can rewrite the PEP 446 to:
>>
>> * make all file descriptors and handles non-inheritable
>> * remove the cloexec parameter
>> * remove everything about non-blocking sockets (O_NONBLOCK), it should
>> be discussed in a new PEP (it's no more related to O_CLOEXEC /
>> HANDLE_INHERIT_FLAG)
>
>
> Sounds good.
>
>>
>> Should I rename os.set_cloexec(fd, cloexec) to os.set_inheritable(fd,
>> inheritable), and os.get_cloexec(fd) to os.get_inheritable(fd)?
>
>
> Yes.
>
>>
>> Or do you prefer a simple os.make_inheritable(fd) with no inheritable
>> parameter? I prefer an explicit parameter, so it's also possible to
>> force again non-inheritable, which also makes sense if the file
>> descriptor was not created by Python.
>
>
> Agreed.
>
>>
>> Victor
>>
>
>
> --
> --Guido van Rossum (on iPad)
>


-- 
--Guido van Rossum (on iPad)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Guido van Rossum]

On Saturday, July 27, 2013, Victor Stinner wrote:

> 2013/7/27 Guido van Rossum >:
> > Do we even need a new PEP, or should we just do it? Or can we adapt
> > Victor's PEP 446?
>
> I can rewrite the PEP 446 to:
>
> * make all file descriptors and handles non-inheritable
> * remove the cloexec parameter
> * remove everything about non-blocking sockets (O_NONBLOCK), it should
> be discussed in a new PEP (it's no more related to O_CLOEXEC /
> HANDLE_INHERIT_FLAG)


Sounds good.

>
> Should I rename os.set_cloexec(fd, cloexec) to os.set_inheritable(fd,
> inheritable), and os.get_cloexec(fd) to os.get_inheritable(fd)?


Yes.

>
> Or do you prefer a simple os.make_inheritable(fd) with no inheritable
> parameter? I prefer an explicit parameter, so it's also possible to
> force again non-inheritable, which also makes sense if the file
> descriptor was not created by Python.


Agreed.

>
> Victor
>


-- 
--Guido van Rossum (on iPad)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

[Victor Stinner]

2013/7/27 Guido van Rossum :
> Do we even need a new PEP, or should we just do it? Or can we adapt
> Victor's PEP 446?

I can rewrite the PEP 446 to:

* make all file descriptors and handles non-inheritable
* remove the cloexec parameter
* remove everything about non-blocking sockets (O_NONBLOCK), it should
be discussed in a new PEP (it's no more related to O_CLOEXEC /
HANDLE_INHERIT_FLAG)

Should I rename os.set_cloexec(fd, cloexec) to os.set_inheritable(fd,
inheritable), and os.get_cloexec(fd) to os.get_inheritable(fd)?

Or do you prefer a simple os.make_inheritable(fd) with no inheritable
parameter? I prefer an explicit parameter, so it's also possible to
force again non-inheritable, which also makes sense if the file
descriptor was not created by Python.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adding Python scripts to PATHEXT on Windows

[Paul Moore]

On 23 July 2013 17:33, Paul Moore  wrote:

> On 23 July 2013 17:11, "Martin v. Löwis"  wrote:
>
>> Am 15.07.13 10:26, schrieb Paul Moore:
>> > Does anyone have any objections to this? I could try to write a patch,
>> > but I know next to nothing about building MSIs, so if any of the
>> > installer experts could help that would be fantastic.
>>
>> It's fine with me. I could write the patch, but will likely forget - so
>> please remind me with every alpha that misses this. Also, contributions
>> are welcome.
>>
>
> Thanks.
>
> I think I have the basic idea of what's needed, so I'll write an initial
> patch. If you can check it that'd be great.
>

OK, see http://bugs.python.org/issue18569

As I mention in the patch, I have yet to successfully manage to build an
MSI installer (I tried a while ago, and the instructions didn't work - I'm
not 100% sure why and I'm afraid I don't have the time right now to try
again) so the patch will definitely need checking by someone who can build
the MSI.

When I get some time (probably not for a few weeks) I'll try the "how to
build the installer" instructions again and see if I can work out what went
wrong and either fix the issue or report back.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com