[issue40467] subprocess: replacement shell on windows with executable="..." arg

[Eryk Sun]

Change by Eryk Sun :


--
versions: +Python 3.10, Python 3.11 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40467] subprocess: replacement shell on windows with executable="..." arg

[Eryk Sun]

Eryk Sun  added the comment:

> it might be nice if it's possible to give some sort of useful 
> warning/error when this happens -- perhaps say that specifying 
> both shell=True and executable="..." isn't supported on Windows?

The `shell` parameter is documented as follows for Windows:

On Windows with shell=True, the COMSPEC environment variable 
specifies the default shell. The only time you need to specify 
shell=True on Windows is when the command you wish to execute is 
built into the shell (e.g. dir or copy). You do not need
shell=True to run a batch file or console-based executable.

It wouldn't hurt to clarify that the COMSPEC shell has to support `/c`. This is 
required by CreateProcessW(), which uses COMSPEC to run BAT and CMD files.

The discussion about using `executable` with `shell` could be extended for 
Windows. But this would also require new behavior. For example:

Original:

if shell:
startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
startupinfo.wShowWindow = _winapi.SW_HIDE
comspec = os.environ.get("COMSPEC", "cmd.exe")
args = '{} /c "{}"'.format (comspec, args)

Proposed:

if shell:
startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
startupinfo.wShowWindow = _winapi.SW_HIDE
if executable is not None:
cmd = executable
else:
cmd = os.path.normpath(os.environ.get("COMSPEC", "cmd.exe"))
if "\\" in cmd:
executable = cmd
args = '"{}" /c "{}"'.format(cmd, args)

> if comspec.endswith('sh.exe') or comspec.endswith('sh'):
> args = '{} -c "{}"'.format (comspec, args) 

sh is not a valid COMSPEC shell. To use sh automatically, subprocess would have 
to support and prefer the SHELL [1] environment variable in Windows -- and in 
POSIX for that matter.

---
[1] 
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40467] subprocess: replacement shell on windows with executable="..." arg

[Joe Cool]

Joe Cool  added the comment:

Proposed solution:

if comspec.endswith('sh.exe') or comspec.endswith('sh'):# 
issue 40467
args = '{} -c "{}"'.format (comspec, args)  # 
issue 40467
else:   # 
issue 40467
args = '{} /c "{}"'.format (comspec, args)

--
nosy: +snoopyjc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40467] subprocess: replacement shell on windows with executable="..." arg

[Anish Athalye]

New submission from Anish Athalye :

On Windows, using subprocess.call() and specifying both shell=True and the
executable='...' keyword arguments produces an undesirable result when the
specified shell is a POSIX-like shell rather than the standard cmd.exe.

I think the documentation is unclear on the semantics of Popen() when both
shell=True and executable= are specified on Windows. It does say the following
about POSIX systems:

> If shell=True, on POSIX the executable argument specifies a replacement shell
> for the default /bin/sh.

But the documentation doesn't say anything about Windows in this scenario, so
I'm not sure if this is a bug, or if it's undefined behavior.

Concretely, here's an example program that fails due to this:

import subprocess
bash = 'C:\\Program Files\\Git\\usr\\bin\\bash.exe'
subprocess.call('f() { echo test; }; f', shell=True, executable=bash)

It prints out this mysterious-looking error:

/c: /c: Is a directory

Tracing this into subprocess.py, it looks like it's because the executable bash
(as specified) is being called with the argv that's approximately ['cmd.exe',
'/c', 'f() { echo test; }; f'] (and the program being launched is indeed bash).

Bash doesn't expect a '/c' argument, it wants a '-c' there.

The problematic code in subprocess.py is here:
https://github.com/python/cpython/blob/1def7754b7a41fe57efafaf5eff24cfa15353444/Lib/subprocess.py#L1407
If the '/c' is replaced with a '-c', the example program above works (bash
doesn't seem to care that it's called with an argv[0] that doesn't make sense,
though presumably that should be fixed too).

I'm not sure how this could be fixed. It's unclear when '/c' should be used, as
opposed to '-c'. Doing it based on the contents of the executable= argument or
the SHELL environment variable or COMSPEC might be fragile? I couldn't find
much about this online, but I did find one project (in Ruby) that seems to have
run into a similar problem. See
https://github.com/kimmobrunfeldt/chokidar-cli/issues/15 and
https://github.com/kimmobrunfeldt/chokidar-cli/pull/16.

At the very least, even if this isn't fixed / can't be fixed, it might be nice
if it's possible to give some sort of useful warning/error when this happens --
perhaps say that specifying both shell=True and executable="..." isn't
supported on Windows?

I ran into this issue while while debugging an issue in a project of mine. In
case the additional context is useful, here is the discussion:
https://github.com/anishathalye/dotbot/issues/219

--
components: Library (Lib), Windows
messages: 367844
nosy: anishathalye, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: subprocess: replacement shell on windows with executable="..." arg
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com