[issue30420] [doc] subprocess module: Clarify kwarg handing for convenience APIs

[Alex Waygood]

Change by Alex Waygood :


--
title: Clarify kwarg handing for subprocess convenience APIs -> [doc] 
subprocess module: Clarify kwarg handing for convenience APIs

___
Python tracker 
<https://bugs.python.org/issue30420>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7438] Allow to use a part of subprocess module during building Python

[Irit Katriel]

Irit Katriel  added the comment:

Closed after 12 years of inactivity. The code has moved on quite a bit since 
then.

--
nosy: +iritkatriel
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41151] Support for new Windows pseudoterminals in the subprocess module

[Chad Smith]

Change by Chad Smith :


--
nosy: +cs01

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41151] Support for new Windows pseudoterminals in the subprocess module

[Eryk Sun]

Eryk Sun  added the comment:

> However, the API is a bit weird. Unlike Unix, when you create a 
> Windows pty, there's no way to directly get access to the "slave" 
> handle. Instead, you first call CreatePseudoConsole to get a 
> special "HPCON" object, which is similar to a Unix pty master. 
> And then you can have to use a special CreateProcess incantation 
> to spawn a child process that's attached to our pty.

As implemented in Windows 8+ (the previous implementation is very different), 
if a process is attached to a console session, the ConsoleHandle value in the 
PEB ProcessParameters is a file handle for "\Device\ConDrv\Connect", which is a 
connection to a console session that's hosted by conhost.exe (or 
openconsole.exe if using Windows Terminal). The connection handle is used 
implicitly with NtDeviceIoControlFile to implement API functions such as 
GetConsoleWindow, GetConsoleTitle, GetConsoleCP, GetConsoleAliases, 
GetConsoleProcessList, GenerateConsoleCtrlEvent, and so on. Console I/O, on the 
other hand, uses dedicated files on "\Device\ConDrv". The initial files used 
for standard I/O are generic "Input" and "Output", which work with any console 
connection. There are other ConDrv files that, when opened, attach to a 
particular console session and are only valid when the process is connected to 
that session, including "Console" (CON), "CurrentIn" (CONIN$), "CurrentOut" 
(CONOUT$), and "Scr
 eenBuffer" (CreateConsoleScreenBuffer). Simple ReadFile and WriteFile use NT 
NtReadFile and NtWriteFile system calls. Console-specific I/O functions such as 
GetConsoleMode, ReadConsoleW, and WriteConsoleW use NtDeviceIoControlFile.

If a console connection is inherited, initially the ConsoleHandle value is a 
handle for "\Device\ConDrv\Reference", which references the console session. 
(If a console session isn't inherited, the ConsoleHandle value may be one of 
the values reserved for the creation flags CREATE_NEW_CONSOLE, 
CREATE_NO_WINDOW, and DETACHED_PROCESS.) The base API opens the "\Connect" 
connection handle relative to the reference handle. The NtCreateFile call in 
this case uses the reserved EaBuffer (extended attributes) argument to send the 
console session host the information it needs about the connecting client 
process. Once connected, depending on the state, the base API may close 
inherited console handles (i.e. handles with the NT device type 
FILE_DEVICE_CONSOLE) that are set in StandardInput, StandardOutput, and 
StandardError, in which case new generic "Input" and "Output" handles are 
opened relative to the connection handle.

The pseudoconsole API apparently uses PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE to 
explicitly pass the "\Device\ConDrv\Reference" handle to client processes, 
instead of inheriting the reference handle of the current process, if the 
current process even has a console. An HPCON appears to be just a pointer to an 
array of three handles. One of the handles is the "Reference" handle that 
client processes need. The other two handles are for the API -- one for the 
console-session host process (conhost.exe) and one for the signal pipe. You can 
see the handle value for the other end of the signal pipe passed on the 
conhost.exe command line as the "--signal" parameter.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41151] Support for new Windows pseudoterminals in the subprocess module

[Gregory P. Smith]

Change by Gregory P. Smith :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41151] Support for new Windows pseudoterminals in the subprocess module

[Nathaniel Smith]

New submission from Nathaniel Smith :

So Windows finally has pty support: 
https://devblogs.microsoft.com/commandline/windows-command-line-introducing-the-windows-pseudo-console-conpty/

However, the API is a bit weird. Unlike Unix, when you create a Windows pty, 
there's no way to directly get access to the "slave" handle. Instead, you first 
call CreatePseudoConsole to get a special "HPCON" object, which is similar to a 
Unix pty master. And then you can have to use a special CreateProcess 
incantation to spawn a child process that's attached to our pty.

Specifically, what you have to do is set a special entry in the 
"lpAttributeList", with type "PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE".

Details: 
https://docs.microsoft.com/en-us/windows/console/creating-a-pseudoconsole-session

Unfortunately, the subprocess module does not provide any way to pass arbitrary 
attributes through lpAttributeList, which means that it's currently impossible 
to use pty support on Windows without reimplementing the whole subprocess 
module.

It would be nice if the subprocess module could somehow support this.

Off the top of my head, I can think of three possible APIs:

Option 1: full support for Windows ptys: this would require wrapping 
CreatePseudoConsole, providing some object to represent a Windows pty, etc. 
This is fairly complex (especially since CreatePseudoConsole requires you to 
pass in some pipe handles, and the user might want to create those themselves).

Option 2: minimal support for Windows ptys: add another supported field to the 
subprocess module's lpAttributeList wrapper, that lets the user pass in an 
"HPCON" cast to a Python integer, and stuffs it into the attribute list. This 
would require users to do all the work to actually *get* the HPCON object, but 
at least it would make ptys possible to use.

Option 3: generic support for unrecognized lpAttributeList entries: add a field 
to the subprocess module's lpAttributeList wrapper that lets you add arbitrary 
entries, specified by type number + arbitrary pointer/chunk of bytes. (Similar 
to how Python's ioctl or setsockopt wrappers work.) Annoyingly, it doesn't seem 
to be enough to just pass in a buffer object, because for pseudoconsole 
support, you actually have to pass in an opaque "HPCON" object directly. (This 
is kind of weird, and might even be a bug, see: 
https://github.com/microsoft/terminal/issues/6705)

--
messages: 372526
nosy: giampaolo.rodola, gregory.p.smith, njs
priority: normal
severity: normal
status: open
title: Support for new Windows pseudoterminals in the subprocess module
type: enhancement
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue41151>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Where is getstatusoutput() in subprocess module?

[MRAB]

On 2019-05-03 20:41, Grant Edwards wrote:

I'm trying to update a python2 app to make it python3 compatible. It
uses commands.getstatusoutput(), which according to

https://docs.python.org/2/library/commands.html#commands.getstatusoutput

... getstatusoutput() and getoutput() have been moved to the subprocess module.

Where are they?


Here:

https://docs.python.org/3/library/subprocess.html#subprocess.getstatusoutput
--
https://mail.python.org/mailman/listinfo/python-list

Re: Where is getstatusoutput() in subprocess module?

[Grant Edwards]

On 2019-05-03, Grant Edwards  wrote:
> I'm trying to update a python2 app to make it python3 compatible. It
> uses commands.getstatusoutput(), which according to
>
> https://docs.python.org/2/library/commands.html#commands.getstatusoutput
>
> ... getstatusoutput() and getoutput() have been moved to the subprocess 
> module.
>
> Where are they?

Doh, never mind.  They were only moved in Python3.  I didn't realize
that the "In Python 3.x" qualifier from the beginning of the previous
sentence applied to the entire paragraph.

-- 
Grant Edwards   grant.b.edwardsYow! I am deeply CONCERNED
  at   and I want something GOOD
  gmail.comfor BREAKFAST!

-- 
https://mail.python.org/mailman/listinfo/python-list

Where is getstatusoutput() in subprocess module?

[Grant Edwards]

I'm trying to update a python2 app to make it python3 compatible. It
uses commands.getstatusoutput(), which according to

https://docs.python.org/2/library/commands.html#commands.getstatusoutput

... getstatusoutput() and getoutput() have been moved to the subprocess module.

Where are they?

-- 
Grant Edwards   grant.b.edwardsYow! How many retured
  at   bricklayers from FLORIDA
  gmail.comare out purchasing PENCIL
   SHARPENERS right NOW??

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35486] subprocess module import hooks breaks back compatibility

[Nick Coghlan]

Nick Coghlan  added the comment:

Aye, we can :)

--
assignee:  -> docs@python
components: +Documentation -Library (Lib)
nosy: +docs@python
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Ned Deily]

Ned Deily  added the comment:

I've merged the doc changes for 3.6, thanks.  Can we close this now?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Ned Deily]

Ned Deily  added the comment:


New changeset 1edb3dc6ff70db88a7e89586578e58a86ee0e75e by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
bpo-35486: Note Py3.6 import system API requirement change (GH-11540) (GH-11588)
https://github.com/python/cpython/commit/1edb3dc6ff70db88a7e89586578e58a86ee0e75e


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

miss-islington  added the comment:


New changeset 422db3777874f4f31fc8f4e718f440a2abc59347 by Miss Islington (bot) 
in branch '3.7':
bpo-35486: Note Py3.6 import system API requirement change (GH-11540)
https://github.com/python/cpython/commit/422db3777874f4f31fc8f4e718f440a2abc59347


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

Change by miss-islington :


--
pull_requests: +11279, 11280, 11281, 11282, 11283

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

Change by miss-islington :


--
pull_requests: +11278, 11279, 11280

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

Change by miss-islington :


--
pull_requests: +11279, 11280, 11281, 11283

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

Change by miss-islington :


--
pull_requests: +11279, 11280, 11283

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

miss-islington  added the comment:


New changeset cee29b46a19116261b083dc803217aa754c7df40 by Miss Islington (bot) 
(Nick Coghlan) in branch 'master':
bpo-35486: Note Py3.6 import system API requirement change (GH-11540)
https://github.com/python/cpython/commit/cee29b46a19116261b083dc803217aa754c7df40


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

Change by miss-islington :


--
pull_requests: +11278, 11279

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[miss-islington]

Change by miss-islington :


--
pull_requests: +11278

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Nick Coghlan]

Nick Coghlan  added the comment:

dw: we routinely impose new requirements on folks modifying runtime internals 
in new feature releases, so the only aspect we missed for this changing is to 
explicitly call it out in the Porting section of the Python 3.6 What's New 
document as a potential forward compatibility risk for import system 
replacement authors that don't update their plugins accordingly.

For folks implementing PEP 302 finders and loaders, no change is required, as 
they indicate failure to find a module by returning None, not by raising 
ImportError.

It's only folks emulating the full import system by replacing `__import__` that 
should need to worry about this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Nick Coghlan]

Change by Nick Coghlan :


--
keywords: +patch, patch
pull_requests: +11147, 11148
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Nick Coghlan]

Change by Nick Coghlan :


--
keywords: +patch, patch, patch
pull_requests: +11147, 11148, 11149
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Nick Coghlan]

Change by Nick Coghlan :


--
keywords: +patch
pull_requests: +11147
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[David Wilson]

David Wilson  added the comment:

Hi Nick,

The purpose of ModuleNotFoundError is clear and unrelated to the problem 
documented here. The problem is that due to the introduction of 
ModuleNotFoundError, ImportError's semantics have been changed within a minor 
release in a breaking, backwards-incompatible manner.

As of Python 3.5, ImportError meant both "ImportError" and 
"ModuleNotFoundError". As of 3.6, the senses are distinct, and thus code 
written against ImportError as it existed in 3.5 no longer works correctly as 
of 3.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Nick Coghlan]

Nick Coghlan  added the comment:

Note that the above distinction is also the rationale for introducing the new 
subtype: so that it's easy to tell the difference between "that module was not 
found at all" (ModuleNotFoundError) and "the module was found, but attempting 
to actually load it failed" (other cases of ImportError)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Nick Coghlan]

Nick Coghlan  added the comment:

The two errors mean different things: ModuleNotFoundError means literally that 
the module could not be found at all (i.e. no import hook offered to try to 
load it)

A plain ImportError then means that the module was located, but attempting to 
actually load it failed.

find_spec()/find_loader()/find_module() implementations on import plugins 
shouldn't be raising exceptions for modules they don't offer, and hence 
shouldn't be needing to raise ModuleNotFoundError directly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Brett Cannon]

Brett Cannon  added the comment:

RE: "PEP-302 and PEP-451 are the definitive specifications for how 
sys.meta_path is supposed to work"

That's actually not true. In the case of import the language reference is 
considered the reference for import: 
https://docs.python.org/3/reference/import.html (and true for PEPs in general 
once they are implemented).

--
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module import hooks breaks back compatibility

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +brett.cannon, eric.snow, ncoghlan
stage:  -> test needed
title: subprocess module breaks backwards compatibility with import hooks -> 
subprocess module import hooks breaks back compatibility
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue35486>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module breaks backwards compatibility with import hooks

[David Wilson]

David Wilson  added the comment:

Having considered this for a few hours, it seems the following is clear:

- 3.6 introduces ModuleImportError
- 3.7 begins using it within importlib
- 3.8 first instance of stdlib code catching it

- PEP-302 and PEP-451 are the definitive specifications for how sys.meta_path 
is supposed to work, and neither makes mention of ModuleNotFoundError. It seems 
at least this should have been flagged during review of the original change, 
but apparently the name of the exception was more important.

- The recent work done to move most of the import machinery on to sys.meta_path 
has exposed a set of import hooks that don't quite comply to the documented 
import hook interface

- The newly advertised ModuleNotFoundError appearing in stack traces for 
everyone means that more people will continue to write new cases of "except 
ModuleNotFoundError:", which while following best practice (catch most specific 
relevant exception), at present it amounts to relying on an implementation 
detail of the default importer. GitHub search reveals this to be an accurate 
reading: https://github.com/search?q=%22except+ModuleNotFoundError%22=Code

Therefore we are in a position where:

- Natural developer habit will cause much more protocol-violating code to exist 
over time, there is no option to stop this process
- New import hooks written against the existing documentation will break in the 
face of developer habit
- Existing import hooks were broken in Python 3.6 and this is not documented 
anywhere
- Python's own import machinery contravenes its specification

Options:

- Updating PEP-302 to mention introduction of the new exception type
- Updating ModuleNotFoundError/ImportError documentation to somehow capture the 
compatibility issue

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module breaks backwards compatibility with import hooks

[David Wilson]

Change by David Wilson :


--
title: subprocess module breaks backwards compatibility with older import hooks 
-> subprocess module breaks backwards compatibility with import hooks

___
Python tracker 
<https://bugs.python.org/issue35486>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module breaks backwards compatibility with older import hooks

[David Wilson]

David Wilson  added the comment:

Corrected GitHub link for the commit: 
https://github.com/python/cpython/commit/880d42a3b24

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35486] subprocess module breaks backwards compatibility with older import hooks

[David Wilson]

New submission from David Wilson :

The subprocess package since 880d42a3b24 / September 2018 has begun using this 
idiom:

try:
import _foo
except ModuleNotFoundError:
bar

However, ModuleNotFoundError may not be thrown by older import hook 
implementations, since that subclass was only introduced in Python 3.6 -- and 
so the test above will fail. PEP-302 continues to document ImportError as the 
appropriate exception that should be raised.

https://mitogen.readthedocs.io/en/stable/ is one such import hook that lazily 
loads packages over the network when they aren't available locally. Current 
Python subprocess master breaks with Mitogen because when it discovers the 
master cannot satisfy the import, it throws ImportError.

The new exception subtype was introduced in https://bugs.python.org/issue15767 
, however very little in the way of rationale was included, and so it's unclear 
to me what need the new subtype is addressing, whether this is a problem with 
the subprocess module or the subtype as a whole, or indeed whether any of this 
should be considered a bug.

It seems clear that some kind of regression is in the process of occurring 
during a minor release, and it also seems clear the new subtype will 
potentially spawn a whole variety of similar new regressions.

I will be updating Mitogen to throw the new subtype if it is available, but I 
felt it was important to report the regression to see what others think.

--
components: Library (Lib)
messages: 331774
nosy: dw
priority: normal
severity: normal
status: open
title: subprocess module breaks backwards compatibility with older import hooks
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue35486>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Chris Angelico]

On Mon, Nov 12, 2018 at 11:20 PM Anssi Saari  wrote:
>
> Chris Angelico  writes:
>
> > On Fri, Nov 9, 2018 at 11:11 PM Anssi Saari  wrote:
> >>
> >> Chris Angelico  writes:
> >>
> >> > No helper needed. Safe against command injection. Uses the known
> >> > format of the command's output; if you want other information as well
> >> > as the type, you could get that too.
> >>
> >> Can someone let me in on this secret helper module? Doesn't seem to
> >> match the helper module in PyPI at least.
> >>
> >
> > What helper?
>
> Helper module used in the original post. I assumed you know what it is
> since you declared it's not needed.

Oh! I see what you mean. I wasn't referring to an entire module, just
a helper *function*. Look at the OP's code at this function:

def execute_cmd_output_string(self, cmd, enable_shell=False):

This is a function whose sole purpose is to make it easier to write
other functions. That's what's often referred to as a "helper
function". (It's also dangerously written, as it turns other failures
into AssertionError.) By simplifying the code and removing the need
for this helper, we can make it more consistent with the rest of the
Python ecosystem, less buggy [1], and probably easier to read.

ChrisA

[1] It's generally true that a programmer's bugs-per-lines-of-code
metric is fairly stable, so having less code usually means less bugs.
Not certain by any means, but consider: non-code cannot have bugs in
it, and code usually does. It's a good rule of thumb.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Anssi Saari]

Chris Angelico  writes:

> On Fri, Nov 9, 2018 at 11:11 PM Anssi Saari  wrote:
>>
>> Chris Angelico  writes:
>>
>> > No helper needed. Safe against command injection. Uses the known
>> > format of the command's output; if you want other information as well
>> > as the type, you could get that too.
>>
>> Can someone let me in on this secret helper module? Doesn't seem to
>> match the helper module in PyPI at least.
>>
>
> What helper? 

Helper module used in the original post. I assumed you know what it is
since you declared it's not needed.

> I said you don't need one. Just use subprocess directly.

This part was clear. 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Cousin Stanley]

srinivasan wrote:

> Even after changing as per the below
> "blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
> or:
> 'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
> or:
> "blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"
> 
> Still my output is:
> */dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"*
> 
> My expected output should be only:
> *vfat*
> 
> Could you guys please do the needful?

 
  I tried a simiar command line from a rock64 sbc shell
  and found that  -f2  instead of  -f3  returned only  vfat 
   

-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Chris Angelico]

On Fri, Nov 9, 2018 at 11:11 PM Anssi Saari  wrote:
>
> Chris Angelico  writes:
>
> > No helper needed. Safe against command injection. Uses the known
> > format of the command's output; if you want other information as well
> > as the type, you could get that too.
>
> Can someone let me in on this secret helper module? Doesn't seem to
> match the helper module in PyPI at least.
>

What helper? I said you don't need one. Just use subprocess directly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Anssi Saari]

Chris Angelico  writes:

> No helper needed. Safe against command injection. Uses the known
> format of the command's output; if you want other information as well
> as the type, you could get that too.

Can someone let me in on this secret helper module? Doesn't seem to
match the helper module in PyPI at least.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[srinivasan]

Many Thanks a lot , I can use for reliably "lsblk %s -n -o FSTYPE"  in the
reused code of mine as below

cmd = "lsblk %s -n -o FSTYPE" % partition_path
return self._helper.execute_cmd_output_string(cmd)

I really appreciate for all your support w.r.t this..

I feel I have kick started my learning in python :)

Have a great day ahead!


On Wed, Nov 7, 2018 at 3:11 PM Ben Bacarisse  wrote:

> srinivasan  writes:
>
> > Even after changing as per the below
> > "blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
> > or:
> > 'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
> > or:
> > "blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"
> >
> > Still my output is:
> > */dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"*
> >
> > My expected output should be only:
> > *vfat*
> >
> > Could you guys please do the needful?
>
> Why not
>
>   blkid %s -o value --match-tag TYPE
>
> ?  Or, better still,
>
>   lsblk %s -n -o FSTYPE
>
> It's not easy to answer your question about fixing the line you have,
> because the output you are getting it not consistent with what you are
> showing.  (And I can't find the original post that presumably has Python
> code I could run myself.)
>
> The format I get with -o export is:
>
> DEVNAME=/dev/sda1
> UUID=2726bf5f-2655-4986-815d-e4532374f218
> TYPE=ext3
> PARTUUID=000453d3-01
>
> for which
>
>   "blkid %s -o export | grep TYPE | cut -c6-"
>
> would work.  Alternatively I get the result you want from
>
>   "blkid %s -o export | grep TYPE | cut -d= -f2"
>
> Note that "TYPE" and "=" don't really need to be quoted at all.
>
> Someone suggested sed, and that too can be used like this:
>
>   blkid %s -o export | sed -ne '/TYPE=/s///p'
>
> --
> Ben.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Ben Bacarisse]

srinivasan  writes:

> Even after changing as per the below
> "blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
> or:
> 'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
> or:
> "blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"
>
> Still my output is:
> */dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"*
>
> My expected output should be only:
> *vfat*
>
> Could you guys please do the needful?

Why not

  blkid %s -o value --match-tag TYPE

?  Or, better still,

  lsblk %s -n -o FSTYPE

It's not easy to answer your question about fixing the line you have,
because the output you are getting it not consistent with what you are
showing.  (And I can't find the original post that presumably has Python
code I could run myself.)

The format I get with -o export is:

DEVNAME=/dev/sda1
UUID=2726bf5f-2655-4986-815d-e4532374f218
TYPE=ext3
PARTUUID=000453d3-01

for which

  "blkid %s -o export | grep TYPE | cut -c6-"

would work.  Alternatively I get the result you want from

  "blkid %s -o export | grep TYPE | cut -d= -f2"

Note that "TYPE" and "=" don't really need to be quoted at all.

Someone suggested sed, and that too can be used like this:

  blkid %s -o export | sed -ne '/TYPE=/s///p'

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Chris Angelico]

On Wed, Nov 7, 2018 at 11:42 PM srinivasan  wrote:
>
> Some I managed to fix temporarily as below, might be useful for others. Also 
> please correct me if anything wrong or for any improvements in the below
>
> cmd = "blkid -o export %s" % partition_path
> out = self._helper.execute_cmd_output_string(cmd)
> var = out.split("TYPE=", 1)[1]
> quoted = re.compile('(?<=^\")[^"]*')
> for string in quoted.findall(var):
> return string

Leaving aside the fact that MS Comic Sans is known to the State of
California to cause cancer, this code is probably okay if you don't
mind it being overengineered. Here's a much simpler version, albeit
untested:

out = subprocess.check_output(["blkid", "-o", "export", partition_path])
for line in out.split("\n"):
item, value = line.split("=")
if item == "TYPE": return value

No helper needed. Safe against command injection. Uses the known
format of the command's output; if you want other information as well
as the type, you could get that too.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[srinivasan]

Some I managed to fix temporarily as below, might be useful for others.
Also please correct me if anything wrong or for any improvements in the
below

cmd = "blkid -o export %s" % partition_path
out = self._helper.execute_cmd_output_string(cmd)
var = out.split("TYPE=", 1)[1]
quoted = re.compile('(?<=^\")[^"]*')
for string in quoted.findall(var):
return string

On Wed, Nov 7, 2018 at 1:39 PM Chris Angelico  wrote:

> On Wed, Nov 7, 2018 at 11:36 PM Qian Cai  wrote:
> >
> > srinivasan  wrote:
> > > Even after changing as per the below
> > > "blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
> > > or:
> > > 'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
> > > or:
> > > "blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"
> > >
> > > Still my output is:
> > > */dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"*
> > >
> > > My expected output should be only:
> > > *vfat*
> > >
> > > Could you guys please do the needful?
> > >
> > >
> > Perfect place to use sed instead of grep/cut.
>
> ... or to use subprocess.check_output() to run just the blkid command,
> and then do the parsing in Python.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Chris Angelico]

On Wed, Nov 7, 2018 at 11:36 PM Qian Cai  wrote:
>
> srinivasan  wrote:
> > Even after changing as per the below
> > "blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
> > or:
> > 'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
> > or:
> > "blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"
> >
> > Still my output is:
> > */dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"*
> >
> > My expected output should be only:
> > *vfat*
> >
> > Could you guys please do the needful?
> >
> >
> Perfect place to use sed instead of grep/cut.

... or to use subprocess.check_output() to run just the blkid command,
and then do the parsing in Python.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Qian Cai]

srinivasan  wrote:
> Even after changing as per the below
> "blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
> or:
> 'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
> or:
> "blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"
> 
> Still my output is:
> */dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"*
> 
> My expected output should be only:
> *vfat*
> 
> Could you guys please do the needful?
> 
> 
Perfect place to use sed instead of grep/cut.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[srinivasan]

Even after changing as per the below
"blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
or:
'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
or:
"blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"

Still my output is:
*/dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"*

My expected output should be only:
*vfat*

Could you guys please do the needful?


On Wed, Nov 7, 2018 at 11:10 AM Brian J. Oney 
wrote:

> On Wed, 2018-11-07 at 10:22 +0100, srinivasan wrote:
> > blkid -o export %s | grep \'TYPE\' | cut -d\"=\" -f3
>
> You don't need to escape the single quotes.
> Try either:
>
> "blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
> or:
> 'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
> or:
> "blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"
>
> HTH
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Brian J. Oney via Python-list]

On Wed, 2018-11-07 at 10:22 +0100, srinivasan wrote:
> blkid -o export %s | grep \'TYPE\' | cut -d\"=\" -f3

You don't need to escape the single quotes.
Try either:

"blkid -o export %s | grep 'TYPE' | cut -d'=' -f3"
or:
'blkid -o export %s | grep "TYPE" | cut -d"=" -f3'
or:
"blkid -o export %s | grep \"TYPE\" | cut -d\"=\" -f3"

HTH
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[srinivasan]

After changing the line to *"cmd = "blkid -o export %s | grep \'TYPE\' |
cut -d\"=\" -f3" % fs"*, Now I dont see the error "SyntaxError: can't
assign to literal"
This is not returning exactly "*vfat*" instead of this, it is returning as "*
/dev/mmcblk1p1: LABEL="efi" UUID="1084-AA42" TYPE="vfat"* "

Could you please help me  as it seems to be like grep and cut commands are
not working in the above line ie., on *cmd = "blkid -o export %s | grep
\'TYPE\' | cut -d\"=\" -f3" % fs*?

On Wed, Nov 7, 2018 at 9:41 AM Avi Gross  wrote:

> I may be missing something but it looks like the embedded double quotes
> may be a problem in this:
>
> cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % ...
>
> if you use single quotes as in:
>
> cut -d"="
>
> becomes
>
> cut -d'='
>
> or escape the double quotes with \" and so on ...
>
> The above seems to be seen as:
>
> cmd=ALPHA=BETA % ...
>
> where ALPHA="blkid -o export %s | grep 'TYPE' | cut -d"
> and BETA=" -f3"
>
> Other issues may also be there.
> -Original Message-
> From: Tutor  On Behalf Of
> Alan Gauld via Tutor
> Sent: Tuesday, November 6, 2018 7:37 PM
> To: tu...@python.org
> Cc: python-...@python.org
> Subject: Re: [Tutor] SyntaxError: can't assign to literal while using
> ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using
> subprocess module in Python
>
> On 06/11/2018 18:07, srinivasan wrote:
>
> > bash command in python using subprocess module, I ma seeing the below
> > *cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" %
> > (fs)*
>
> In general you should try to do as little as possible using bash and
> subprocess. Especially try to avoid long pipelines since you are starting a
> new OS process for every element in the pipeline. That means, in your case,
> you are running 4 processes to get your result - Python, blkid, grep and cut
>
> Python is designed to do much of what the shell command can do almost as
> easily and much more efficiently (no new processes being started).
>
> In this case just execute the blkid bit in bash because its too difficult
> to replicate simply in Python. Then use Python to search for the TYPE lines
> and slice them to size.
>
> That will, in turn, simplify your command string and remove the issue of
> multiple quotes.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  tu...@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
> ___
> Tutor maillist  -  tu...@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[Rhodri James]

On 06/11/2018 18:10, srinivasan wrote:

root:~/qa/test_library# python3 sd.py
   File "sd.py", line 99
*cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)*
* ^*
*SyntaxError: can't assign to literal*


Look at the 'cut' element of the pipeline.  You have used double quotes 
in a double quoted string without escaping them.  As a result, the 
interpreter thinks you are trying to assign the string literal " -f3" to 
the string literal "blkid -o export %s | grep 'Type' | cut -d"


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

SyntaxError: can't assign to literal while using ""blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)" using subprocess module in Python

[srinivasan]

Dear Python Experts Team,

As am newbie to python development, I am trying to use the below function
to get verify the filesystem type of the SD card parition using bash
command in python using subprocess module, I ma seeing the below Error
"SyntaxError: can't assign to literal"

*CODE:*
**

import helper
from os import path
import subprocess
import os
import otg_ni


class emmc(object):
"""
emmc getters and setters
info:
https://www.kernel.org/doc/Documentation/cpu-freq/user-guide.txt
"""

def __init__(self):
self._helper = helper.helper()
self._otg_ni = otg_ni.otg_ni()


*def get_fstype_of_mounted_partition(self, fs):*
"""
Get the filesystem type of the mounted partition.

:partition_name : Partition path as string (e.g. /dev/mmcblk0p1)
:return: filesystem type as string or None if not found
"""

*cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)*
*return self._helper.execute_cmd_output_string(cmd)*



*def execute_cmd_output_string(self, cmd, enable_shell=False):*
"""
Execute a command and return its output as a string.

:param cmd: abs path of the command with arguments
:param enable_shell : force the cmd to be run as shell script
:return: a string.
"""

try:
result = subprocess.check_output(split(cmd),
 stderr=subprocess.STDOUT,
 shell=enable_shell)

except subprocess.CalledProcessError as e:
s = """While executing '{}' something went wrong.
Return code == '{}'
Return output:\n'{}'
""".format(cmd, e.returncode, e.output, shell=enable_shell)
raise AssertionError(s)

return result.strip().decode("utf-8")
*if __name__ == "__main__":*
m = emmc()
*m.get_fstype_of_mounted_partition("/dev/mmcblk0p1")*
*Error:*
*==*

root:~/qa/test_library# python3 sd.py
  File "sd.py", line 99
*cmd = "blkid -o export %s | grep 'TYPE' | cut -d"=" -f3" % (fs)*
* ^*
*SyntaxError: can't assign to literal*
root:~/qa/test_library#

Kindly do the needful as early as possible, as am stuck with this issue
from past 2 days no clues yet, please redirect me to the correct forum if
this is not the right place for pasting python related queries

Many Thanks in advance,
Srini
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31296] support pty.fork and os.forkpty actions in posix subprocess module

[Gregory P. Smith]

New submission from Gregory P. Smith:

The PyPI pexpect.spawn API is incompatible with multithreaded code thanks to 
its dependency on pty.fork() and running non async signal safe code between 
fork() and exec().
 https://github.com/pexpect/ptyprocess/issues/43
 https://github.com/pexpect/pexpect/issues/369

If subprocess supported pty.fork() and pexpect.pty_spawn.spawn like actions, 
pexpect could use that by default instead of the threading incompatible 
pty.fork().

(this is a brainstorm, see what the pexpect maintainers want in the above 
github issues)

--
components: Library (Lib)
messages: 300968
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: support pty.fork and os.forkpty actions in posix subprocess module
versions: Python 3.7

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31296>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

STINNER Victor added the comment:


New changeset 2097b9e0ef32ab7a0d745edc0f707c615780c006 by Victor Stinner in 
branch '2.7':
[2.7] bpo-30764: test_subprocess uses SuppressCrashReport (#2405) (#2412)
https://github.com/python/cpython/commit/2097b9e0ef32ab7a0d745edc0f707c615780c006


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

STINNER Victor added the comment:


New changeset 9ad50d94599aed0c37beaf78948ec271c8aa3881 by Victor Stinner in 
branch '3.6':
bpo-30764: test_subprocess uses SuppressCrashReport (#2405) (#2410)
https://github.com/python/cpython/commit/9ad50d94599aed0c37beaf78948ec271c8aa3881


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

STINNER Victor added the comment:


New changeset 849b062a82ca2f09e33259d34067faba196c9e23 by Victor Stinner in 
branch '3.5':
bpo-30764: test_subprocess uses SuppressCrashReport (#2405) (#2411)
https://github.com/python/cpython/commit/849b062a82ca2f09e33259d34067faba196c9e23


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2461

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2459

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

STINNER Victor added the comment:


New changeset cdee3f14f7f4c995e7eedb0bf6a67e260c739f7d by Victor Stinner in 
branch 'master':
bpo-30764: test_subprocess uses SuppressCrashReport (#2405)
https://github.com/python/cpython/commit/cdee3f14f7f4c995e7eedb0bf6a67e260c739f7d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2453

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Gregory P. Smith added the comment:

Fixed applied to subprocess32 in 
https://github.com/google/python-subprocess32/commit/0f1958e982bf44db569470def7281bcafa2a8b0e

--
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Gregory P. Smith added the comment:


New changeset 56bc3b768c3cc3817031b56d5e7a279aa1296bc9 by Gregory P. Smith in 
branch 'master':
bpo-29335 - apply suggested test_subprocess simplifications from haypo and 
Zach: (#1757)
https://github.com/python/cpython/commit/56bc3b768c3cc3817031b56d5e7a279aa1296bc9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
pull_requests: +1843

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests:  -916

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Donald Stufft]

Changes by Donald Stufft :


--
pull_requests: +916

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7438] Allow to use a part of subprocess module during building Python

[Chi Hsuan Yen]

Changes by Chi Hsuan Yen :


--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Zach Riggle]

Zach Riggle added the comment:

Neat, though that's not in the standard library.

The current logic for getting a handle to libc could also be simplified via 
ctypes.util.find_library 
(https://docs.python.org/3/library/ctypes.html#finding-shared-libraries).

Darwin:

>>> import ctypes.util
>>> ctypes.util.find_library('c')
'/usr/lib/libc.dylib'

Linux:

>>> import ctypes.util
>>> ctypes.util.find_library('c')
'libc.so.6'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

STINNER Victor added the comment:

If you want crashes, look at the portable faulthandler._sigsegv() :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Zach Riggle]

Zach Riggle added the comment:

Of note, there's no need to actually cause a SIGSEGV to generate the signal.

The tests might be more clear to replace:

libc.printf(ctypes.c_char_p(0xdeadbeef))

with

os.kill(os.getpid(), signal.SIGSEGV)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Gregory P. Smith added the comment:

thanks Ned, I was awaiting interesting buildbot results. :)

fixed in 2.7 and 3.5 onwards.  thanks for the report Zach.

not closing until I also apply the fix to the subprocess32 backport.

--
resolution:  -> fixed
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Roundup Robot]

Roundup Robot added the comment:

New changeset 8e3d412f8e89 by Gregory P. Smith in branch '2.7':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/8e3d412f8e89

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Ned Deily]

Ned Deily added the comment:

Among other buildbot failures:

http://buildbot.python.org/all/builders/x86%20Tiger%203.6/builds/142/steps/test/logs/stdio

==
ERROR: test_child_terminated_in_stopped_state 
(test.test_subprocess.POSIXProcessTestCase)
Test wait() behavior when waitpid returns WIFSTOPPED; issue29335.
--
Traceback (most recent call last):
  File 
"/Users/db3l/buildarea/3.6.bolen-tiger/build/Lib/test/test_subprocess.py", line 
2514, in test_child_terminated_in_stopped_state
libc = ctypes.CDLL(libc_name)
  File "/Users/db3l/buildarea/3.6.bolen-tiger/build/Lib/ctypes/__init__.py", 
line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(libc..dylib, 6): image not found

--
Ran 260 tests in 102.297s


Also, 
http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/240/steps/test/logs/stdio

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Roundup Robot]

Roundup Robot added the comment:

New changeset 269296b2a047 by Gregory P. Smith in branch '3.5':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/269296b2a047

New changeset ed5255a61648 by Gregory P. Smith in branch '3.6':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/ed5255a61648

New changeset 4f5e7d018195 by Gregory P. Smith in branch 'default':
Issue #29335: Fix subprocess.Popen.wait() when the child process has
https://hg.python.org/cpython/rev/4f5e7d018195

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Gregory P. Smith added the comment:

test added.

--
stage: test needed -> patch review
type:  -> behavior
Added file: http://bugs.python.org/file46386/issue29335-gps02.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Gregory P. Smith added the comment:

The attached patch should fix it.

I want to incorporate a bug.py like regression test into test_subprocess.py.

--
keywords: +patch
stage:  -> test needed
Added file: http://bugs.python.org/file46385/issue29335-gps01.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Changes by Gregory P. Smith <g...@krypto.org>:


--
title: Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD -> 
subprocess module does not check WIFSTOPPED on SIGCHLD

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29335>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
assignee:  -> gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
versions: +Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD

[Zach Riggle]

Zach Riggle added the comment:

To further clarify the report:

When the attached proof-of-concept is executed, a RuntimeException is raised, 
which has a comment "Should never happen".

The issue isn't due to SIGCHLD, but rather following a waitpid() call.  The 
code attempts to suss the exit code / reason for waitpid() returning, but does 
not check for WIFSTOPPED in its handler.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD

[Ned Deily]

Changes by Ned Deily :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD

[Zach Riggle]

New submission from Zach Riggle:

The attached script hits some "This should never happen" code in the subprocess 
module.

These lines here:
https://github.com/python/cpython/blob/2.7/Lib/subprocess.py#L1036-L1038

The root cause is a lack of checking WIFSTOPPED and WSTOPSIG in the handler.

When a process elects into being ptraced via PTRACE_TRACEME, it is stopped on 
the SIGSEGV instead of terminating, allowing the user to attach a debugger 
before the kernel destroys the process.

This bug makes it impossible to wait on any process which crashes, which is set 
up to wait for a debugger.

--
components: Library (Lib)
files: bug.py
messages: 285921
nosy: Zach Riggle
priority: normal
severity: normal
status: open
title: Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD
versions: Python 2.7
Added file: http://bugs.python.org/file46363/bug.py

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29335>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Martin Panter]

Changes by Martin Panter :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Roundup Robot]

Roundup Robot added the comment:

New changeset 720865fa61a4 by Martin Panter in branch '3.5':
Issue #26240: Clean up the subprocess module doc string
https://hg.python.org/cpython/rev/720865fa61a4

New changeset 8358c68579e9 by Martin Panter in branch '3.6':
Issue #26240: Merge subprocess doc string from 3.5 into 3.6
https://hg.python.org/cpython/rev/8358c68579e9

New changeset 0dd8b3f133f9 by Martin Panter in branch 'default':
Issue #26240: Merge subprocess doc string from 3.6
https://hg.python.org/cpython/rev/0dd8b3f133f9

New changeset 5a1edf5701f1 by Martin Panter in branch '2.7':
Issue #26240: Clean up the subprocess module doc string
https://hg.python.org/cpython/rev/5a1edf5701f1

--
nosy: +python-dev

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26240>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Martin Panter]

Changes by Martin Panter :


Added file: http://bugs.python.org/file45180/subprocess3-2.7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Martin Panter]

Martin Panter added the comment:

Here are corresponding patches for 3.5 and 2.7.

--
stage: patch review -> commit review
Added file: http://bugs.python.org/file45179/subprocess3-3.5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Martin Panter]

Martin Panter added the comment:

V3 looks good to me

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Tim Mitchell]

Tim Mitchell added the comment:

Changes as per Martins review.

--
Added file: http://bugs.python.org/file45159/subprocess3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Martin Panter]

Martin Panter added the comment:

.
I left some comments on the code review.

Also, I’m not sure about the links to the online documentation. We don’t do 
this for other modules as far as I know. The pydoc module and help() commands 
already add their own links, which can be configured via PYTHONDOCS.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Tim Mitchell]

Tim Mitchell added the comment:

Am now working from tip of default in mercurial (Python 3.6).

* Removed all changes to subprocess.rst.

subprocess.__doc__
* Trimmed down even further - removed function signatures.
* added note to see Python docs for complete description.

Exception docs
* just list attributes rather than describing them - they are pretty 
self-explanatory.

Popen.__doc__ 
* Trimmed down to just list arguments with 1 or 2 liner descriptions. 
* Added note to see Python docs for complete description.
* added encoding and errors arguments

--
Added file: http://bugs.python.org/file45140/subprocess2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Martin Panter]

Martin Panter added the comment:

Thanks for tackling this one Tim. I agree with Berker that the :const:`True` 
changes are out of scope (some introduce errors and inaccuracies).

 class CalledProcessError(SubprocessError):
-"""Raised when a check_call() or check_output() process returns non-zero.
+"""Raised when a process run by check_call() or check_output() process
+returns a non-zero exit status.

New text has stray “process” noun. The old text was good enough IMO.

+  output:  Output of the child process if it was captured by run() or
+   check_output(). Otherwise, None.
 check_output() will also store the output in the output attribute.

I think that last paragraph is redundant and strange now that you already 
described the “output” attribute.

 class TimeoutExpired(SubprocessError):
+Attributes:

The “cmd” attribute is missing from the list.

 class Popen(object):

Your patch seems to be based on 3.6 or 3.7, but you have omitted the new 
“encoding” etc parameters from the signature (and list of constructor 
parameters). What about just relying on the automatic signature for __init__()?

+""" [. . .]
+  universal_newlines:
+If universal_newlines is True, the file objects stdout and stderr are
+opened as a text file, but lines may be terminated by any of '\n',

“opened as text files” would read better I think.

The escape codes will be translated to literal newlines etc in the doc string. 
The best solution is probably to make it a raw string: r""". . .""".

This universal_newlines entry has lots of details about newline translation of 
stdout and stderr, but fails to mention the translation of stdin. And I think 
the details about the child decoding its input should be added to the RST 
documentation before we consider adding it to the doc string.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Tim Mitchell]

Tim Mitchell added the comment:

hg patch with changes forthcoming

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Antony Lee]

Changes by Antony Lee :


--
nosy:  -Antony.Lee

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Berker Peksag]

Berker Peksag added the comment:

Thanks for the patch, Tim.

> Changed true -> :const:`True` in subprocess.rst.

This is out of scope for this issue and I actually prefer the current form.

Having method and class signatures in subprocess.__doc__ would make it less 
maintainable. I'd prefer having a short docstring that describes what the 
modules does, and what its modern API look like.

I don't think we should duplicate documentation of CalledProcessError, 
TimeoutExpired and Popen in their docstrings. Perhaps it would be better to 
just document which parameters are POSIX only.

Also, did you use the GitHub mirror to create the patch? If so, please use the 
official Mercurial repository: 
https://docs.python.org/devguide/setup.html#checkout Rietveld doesn't like 
patches created from the git repository so it was a bit hard to review the 
patch without using Rietveld. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Berker Peksag]

Changes by Berker Peksag :


--
nosy: +berker.peksag
stage: needs patch -> patch review
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Tim Mitchell]

Tim Mitchell added the comment:

Have stripped down the module __doc__ to a list of contents.
I chose to indent the descriptions of each argument to Popen. I know this is 
non-standard but it is such a long ramble otherwise.

Changed true -> :const:`True` in subprocess.rst.

--
keywords: +patch
nosy: +tim.mitchell
Added file: http://bugs.python.org/file45126/subprocess-docs.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26240] Docstring of the subprocess module should be cleaned up

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28165] The 'subprocess' module leaks memory when called in certain ways

[Xavion]

Changes by Xavion :


Added file: http://bugs.python.org/file44763/Test-3a-gc.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28165] The 'subprocess' module leaks memory when called in certain ways

[Xavion]

Changes by Xavion :


Added file: http://bugs.python.org/file44762/Test-3a-no-gc.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28165] The 'subprocess' module leaks memory when called in certain ways

[Xavion]

Xavion added the comment:

Firstly, you've misquoted me.  The quote you attributed to me in your latest 
post was actually made by 'ztane'.

Secondly, your extra thread/event code makes no difference here.  I will attach 
the memory usage logs in subsequent posts.

For consistency, I have removed all of the collateral stuff from your 
"Test-3.py" script and reattached it here as "Test-3a.py".

--
resolution: not a bug -> 
status: closed -> open
title: The 'subprocess' module leaks 4 kiB memory for each thread -> The 
'subprocess' module leaks memory when called in certain ways
Added file: http://bugs.python.org/file44761/Test-3a.py

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28165] The 'subprocess' module leaks 4 kiB memory for each thread

[STINNER Victor]

STINNER Victor added the comment:

Test-2.py has issues:

* it doesn't call Timer.join()
* it uses a weak synchronization between the main thread and the Timer thread: 
see msg276990 for an example using Event

If you use a better synchronization code, call timer.join() and call 
gc.collect(), the memory usage is very stable even after creating more than 100 
000 threads + subprocesses.

Xavion: "As I noted before the problem is not with subprocess leaking 4K memory 
*always*. The issue comes from the fact that subprocess seems to leak 4K memory 
per individual thread."

I'm unable to reproduce a *leak*.

Test-3.py output:

497 thread+subprocess
VmRSS:  9584 kB
986 thread+subprocess
VmRSS:  9596 kB  <== after the warmup, the usage seems stable
1490 thread+subprocess
VmRSS:  9596 kB
(...)
10361 thread+subprocess
VmRSS:  9596 kB
(...)
30282 thread+subprocess
VmRSS:  9596 kB
30695 thread+subprocess
VmRSS:  9672 kB
31160 thread+subprocess
VmRSS:  9684 kB  <=== memory usage decreases :-)
(...)
60768 thread+subprocess
VmRSS:  9684 kB
^C
---

If you really want to say that something is wrong: I don't understand why we 
must call gc.collect() to keep the memory usage stable. But I guess that the GC 
is not always called for performance.

Without the GC it's not that bad:
---
1083 thread+subprocess
VmRSS:  9764 kB
2097 thread+subprocess
VmRSS:  9888 kB
3136 thread+subprocess
VmRSS:  9888 kB
(...)
11750 thread+subprocess
VmRSS:  9888 kB
12668 thread+subprocess
VmRSS:  9940 kB
13705 thread+subprocess
VmRSS:  9940 kB
(...)
70948 thread+subprocess
VmRSS:  9940 kB
^C
---

There is no such "4k leak per function call".

I close the issue. It's a bug in your code, not in Python.

--
resolution:  -> not a bug
status: open -> closed
Added file: http://bugs.python.org/file44749/Test-3.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28165] The 'subprocess' module leaks 4 kiB memory for each thread

[Xavion]

Xavion added the comment:

haypo: So, what is the result when you run "Test-2.py" and monitor the memory 
usage with "Test.sh"?

ztane: The code you've provided is the same as "Test-1.py".  You need to run 
"Test-2.py" in order to see the bug!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28165] The 'subprocess' module leaks 4 kiB memory for each thread

[Antti Haapala]

Antti Haapala added the comment:

The title of the issue is still wrong. As I noted before the problem is not 
with subprocess leaking 4K memory *always*. The issue comes from the fact that 
subprocess seems to leak 4K memory per individual thread. The test code to use 
is thus

def test():
check_output("true")
threading.Timer(1, test, ()).start()

test()

which will invoke subprocess always in a new thread. Using subprocess in a 
loop, or using the timer as above without subprocess will not increase memory 
usage.

I have changed the title accordingly

--
title: The 'subprocess' module leaks memory when called in certain ways -> The 
'subprocess' module leaks 4 kiB memory for each thread

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com