[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Martin Panter]

Changes by Martin Panter :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> subprocess.Popen(cwd) documentation

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

Jovik added the comment:

I appreciate your suggestion regarding cygwin, but in the new code base we want 
to avoid this dependency. Thanks for your time on this issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

Jovik added the comment:

Isn't Python's crossplatform functionality a key feature? A quick fix would be 
to retry the call by adding cwd to arg[0] in case of FileNotFoundError.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Eric V. Smith]

Eric V. Smith added the comment:

We don't always provide fully cross-platform functionality (see the os module 
for many examples), but we might be able to do better here. It might be some 
functionality we can add, it might be a documentation issue.

Note, for example, the subprocess.STARTUPINFO and subprocess.Popen 
creationflags argument. These expose Windows-only functionality.

I'm opposed to trying again with cwd added. There's a long history of security 
problems doing exactly this. It's why '.' is not in the PATH by default on Unix.

On my list of things to do is trace through exactly which scenarios work and 
don't work on Windows.

If you really want a more cross-platform solution, Cygwin python might work for 
you.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

Jovik added the comment:

As requested I did extra tests with extension. Same result as before:
 proc = subprocess.Popen(plink.exe, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace)
 proc = subprocess.Popen(.\plink.exe, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace)
 proc = subprocess.Popen(.\\plink.exe, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace)
 proc = subprocess.Popen(.\plink.exe, stdout=subprocess.PIPE, 
 cwd=c:/python33/workspace)
 proc = subprocess.Popen(plink.exe, stdout=subprocess.PIPE, 
 cwd=c:/python33/workspace)

Traceback (most recent call last):
  File C:\Python33\lib\subprocess.py, line 1104, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 1, in module
  File C:\Python33\lib\subprocess.py, line 819, in __init__
restore_signals, start_new_session)
  File C:\Python33\lib\subprocess.py, line 1110, in _execute_child
raise WindowsError(*e.args)
FileNotFoundError: [WinError 2] The system cannot find the file specified

I believe it's a wider issue, since my colleagues insisted on using shell=True 
on Windows by default (I didn't understand why, until now)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Eric V. Smith]

Eric V. Smith added the comment:

Where is plink.exe? If it's not in cwd (c:\python33\workspace), note that the 
documentation for cwd says:
If cwd is not None, the function changes the working directory to cwd before 
executing the child. In particular, the function looks for executable (or for 
the first item in args) relative to cwd if the executable path is a relative 
path.

Although confusingly, the 2.7 documentation says:
If cwd is not None, the child’s current directory will be changed to cwd 
before it is executed. Note that this directory is not considered when 
searching the executable, so you can’t specify the program’s path relative to 
cwd.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

Jovik added the comment:

plink.exe is located in c:\python33\workspace. I'm aware of the docstring 
difference between Python 2 and 3, thus submitted the bug for Python 3 only.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Eric V. Smith]

Eric V. Smith added the comment:

I think the 2.7 documentation is correct: the current directory when the call 
is made, and not cwd, is included in the search path. I'd suggest specifying 
args as [c:\\python33\\workspace\\plink.exe].

I think I may have mislead you earlier on the search path. The Windows call is:
CreateProcess(lpApplicationName, lpCommandLine, other stuff, 
lpCurrentDirectory, other stuff)

The first parameter to Popen (args) becomes lpCommandLine. The executable 
parameter to Popen, which you're not setting, becomes lpApplicationName. So, 
you're calling CreateProcess(NULL, plink.exe, ..., 
lpCurrentDirectory=c:\\python33\\workspace).

In this case, .exe would be added if missing. But, the search path rules seem 
to not include the directory pointed to by lpCurrentDirectory (aka cwd).

So I think this would work:
subprocess.Popen([c:\\python33\\workspace\\plink.exe], 
stdout=subprocess.PIPE, cwd=c:\\python33\\workspace)
or
subprocess.Popen([plink], executable=c:\\python33\\workspace\\plink.exe, 
stdout=subprocess.PIPE, cwd=c:\\python33\\workspace)

Unfortunately, I don't have a copy of 3.x on my Windows machine to test with.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

Jovik added the comment:

Why this feature works on Posix systems, but not Windows? If my memory is 
correct, it didn't work anywhere when I used Python 2.7 (according with docs).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Eric V. Smith]

Eric V. Smith added the comment:

The underlying APIs are very different. It's debatable how much of a shim we 
should provide to make all platforms look alike.

I think first we should understand what it currently takes to make something 
work in both environments. Then we can talk about how or if we can make them 
look more similar.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

Jovik added the comment:

I'm quite aware of the os.sep issues between the systems, but I checked both 
out of curiosity. Here are latest results:

All of the following commands raises the same exception:
 proc = subprocess.Popen(plink, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace)
 proc = subprocess.Popen(plink, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace\\)
 proc = subprocess.Popen(.\plink, stdout=subprocess.PIPE, 
 cwd=c:\python33\workspace)
 proc = subprocess.Popen(.\\plink, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace)
 proc = subprocess.Popen(plink, stdout=subprocess.PIPE, 
 cwd=c:/python33/workspace)

Traceback (most recent call last):
  File C:\Python33\lib\subprocess.py, line 1104, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 1, in module
  File C:\Python33\lib\subprocess.py, line 819, in __init__
restore_signals, start_new_session)
  File C:\Python33\lib\subprocess.py, line 1110, in _execute_child
raise WindowsError(*e.args)
FileNotFoundError: [WinError 2] The system cannot find the file specified

But, when I set shell=True, then everything works just fine:
 proc = subprocess.Popen(plink, stdout=subprocess.PIPE, 
 cwd=c:\python33\workspace, shell=True)
 proc = subprocess.Popen(.\plink, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace, shell=True)
 proc = subprocess.Popen(.\\plink, stdout=subprocess.PIPE, 
 cwd=c:\\python33\\workspace, shell=True)
 proc = subprocess.Popen(plink, stdout=subprocess.PIPE, 
 cwd=c:/python33/workspace, shell=True)
I can get plink's output afterwards with proc.communicate()

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Eric V. Smith]

Eric V. Smith added the comment:

In the first example, you switch from ./app to app.exe when using 
shell=True.

What happens to any of your examples if you add .exe without shell=True?

Popen eventually calls CreateProcess on Windows. From: 
http://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx, 
describing the lpApplicationName  parameter: This parameter must include the 
file name extension; no default extension is assumed.

Running the shell though, you don't need the extension.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Eric V. Smith]

Eric V. Smith added the comment:

Assuming this is the problem, we should at least document this. It does make 
cross-platform coding difficult.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Kathleen Weaver]

Changes by Kathleen Weaver kathl...@kweaver.org:


--
nosy: +kathweaver

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

New submission from Jovik:

This works on Linux as expected:
import subprocess
proc = subprocess.Popen([./app], stdout=subprocess.PIPE, cwd=workspace)

but on Windows I get:
FileNotFoundError: [WinError 2] The system cannot find the file specified

To successfully execute it on Windows I need to set shell=True first:
proc = subprocess.Popen([app.exe], stdout=subprocess.PIPE, cwd=workspace, 
shell=True)

which is odd since by default it should use cwd when searching for binary:
If cwd is not None, the function changes the working directory to cwd before 
executing the child. In particular, the function looks for executable (or for 
the first item in args) relative to cwd if the executable path is a relative 
path.
from http://docs.python.org/3.3/library/subprocess.html

--
assignee: docs@python
components: Documentation
messages: 213570
nosy: Jovik, docs@python
priority: normal
severity: normal
status: open
title: Different behaviour on Posix and Windows when using 
subprocess.Popen(..., cwd=path)
type: behavior
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[R. David Murray]

R. David Murray added the comment:

Your cwd is relative.  What happens if you make it absolute?  (What I'm 
thinking is that the non-shell starting cwd may be different on windows than it 
is on unix...but I don't know windows very well, so this may be irrelevant...)

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Eric V. Smith]

Changes by Eric V. Smith e...@trueblade.com:


--
nosy: +eric.smith

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Jovik]

Jovik added the comment:

I did a test with cwd being set to full path, but the result was the same 
(still had to use shell=True to execute a command). Let me know if I can 
provide any more details.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20927] Different behaviour on Posix and Windows when using subprocess.Popen(..., cwd=path)

[Terry J. Reedy]

Terry J. Reedy added the comment:

I am sure that using / instead of \, which is to say, not using os.sep, is the 
problem as / is *not* allowed in Windows command names even though the 
substitution works for paths given as options. In a Windows console,
 python # works
 .\python # works
 ./python # see /python as an option for the . program, which does not exist.

I suspect that shell=True cause subprocess to execute shell ./app (howver 
'shell' is spelled on the system), with whatever other options and quotation 
are needed to make ./app work as an option passed to shell instead of a command 
to be executed directly.

I also suspect that passing .\\app might work, which would mean that you 
should use .%sapp % os.sep to get a cross-platform string. If so, please close 
this issue.

--
nosy: +terry.reedy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20927
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com