[issue14872] subprocess is not safe from deadlocks

2012-05-22 Thread Ross Lagerwall 

Ross Lagerwall  added the comment:

See also issue1260171.

Closing as a duplicate of that.

--
resolution:  -> duplicate
stage:  -> committed/rejected
status: open -> closed
superseder:  -> subprocess: more general (non-buffering) communication
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14872] subprocess is not safe from deadlocks

2012-05-21 Thread anatoly techtonik 

anatoly techtonik  added the comment:

The problem with memory is more actual for machines with SSD where swap is 
usually turned off and /tmp files are located on memory disk. Hitting memory 
limit often means hard reset.

My process is pretty generic that uses all streams, and I don't know how to use 
threads/polls crossplatform way.

issue1191964 looks interesting.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14872] subprocess is not safe from deadlocks

2012-05-21 Thread Ross Lagerwall 

Ross Lagerwall  added the comment:

Well if you're *certain* that the process is only using one stream, then you 
can just use read/write on that stream.

If not, it probably means you have to use either threads or select/poll.

This is a known issue with subprocess; there are a few proposals on the tracker 
about this. See issue1191964 for example.

--
nosy: +rosslagerwall

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14872] subprocess is not safe from deadlocks

2012-05-21 Thread Daniel Swanson 

Daniel Swanson  added the comment:

what sort of machine has infinite memory?

--
nosy: +weirdink13

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14872] subprocess is not safe from deadlocks

2012-05-21 Thread Chris Rebert 

Changes by Chris Rebert :


--
nosy: +cvrebert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14872] subprocess is not safe from deadlocks

2012-05-21 Thread Richard Oudkerk 

Richard Oudkerk  added the comment:

I think the note for communicate() just means that you might get MemoryError 
(or some other exception) if the output is too big.  But I agree it is 
ambiguous.

communicate() uses select() on Unix and threads on Windows, so deadlocks should 
not be possible.

> So, what should I use?

Use communicate() (on a machine with infinite memory;-)

--
nosy: +sbt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14872] subprocess is not safe from deadlocks

2012-05-21 Thread anatoly techtonik 

New submission from anatoly techtonik :

There is no way to write a program in Python capable to process large/unlimited 
output coming from a subprocess stream without deadlocks.

http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
"Note The data read is buffered in memory, so do not use this method if the 
data size is large or unlimited."

http://docs.python.org/library/subprocess.html#subprocess.Popen.stdin
http://docs.python.org/library/subprocess.html#subprocess.Popen.stdout
http://docs.python.org/library/subprocess.html#subprocess.Popen.stderr
"Warning Use communicate() rather than .stdin.write, .stdout.read or 
.stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling 
up and blocking the child process."


So, what should I use?

--
components: Library (Lib)
messages: 161294
nosy: techtonik
priority: normal
severity: normal
status: open
title: subprocess is not safe from deadlocks
versions: Python 2.7, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com