Re: Subprocess Popen confusion

[Barry Scott]

> On 18 May 2020, at 21:50, Dick Holmes  wrote:

snip

> Per Peter's suggestion I tried readline and it works "as expected". I 
> also discovered that the reason the read operations were stalling was 
> that they followed a write and the write doesn't actually occur until 
> "flush" is called even though the function call returns. There are 
> undoubtedly some subtleties lurking about, but at least I'm making good 
> progress.

You need the flush() because the stream is using buffered I/O.

Each write adds to the buffer inside python and when then buffer if full it is
written out of Python into the OS.

As you found you can force a partial buffer to be written by calling flush().

Barry


> 
> Thanks for the help!
> 
> Dick
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Popen confusion

[Dick Holmes]

In article , 
__pete...@web.de says...
> 
> Dick Holmes wrote:
> 
> > https://occovid19.ochealthinfo.com/coronavirus-in-oc 
> 
> > I'm trying to
> > communicate using a continuing dialog between two
> > processes on the same system.
> 
> I think pexpect
> 
> https://pexpect.readthedocs.io/en/stable/index.html
> 
> does this naturally, but I don't know if Windows support is sufficient for 
> your needs.
> 
> > I've looked at various mechanisms and the
> > class that seems to fit my needs is Popen in the subprocess module, but
> > I can't seem to get more than a single round-trip message through Popen.
> > I first call Popen then poll using the identifier returned from the call
> > and the poll seems to work. I then call the communicate function passing
> > None as the value to send to the companion process stdin. I get the
> > expected result, but I also get "Exception condition detected on fd 0
> > \\n" and "error detected on stdin\\n". Subsequent attempts to
> > read/write/communicate with the subprocess fail because the file (stdxx
> > PIPE) is closed.
> > 
> > I can't tell from the documentation if the communicate function is a
> > one-time operation. 
> 
> Yes, communicate() is one-off, 
> 
> 
> """Interact with process: Send data to stdin and close it.
> Read data from stdout and stderr, until end-of-file is
> reached.  Wait for process to terminate.
> ...
> """
> 
> seems pretty clear. What would you improve?
> 
> > I have tried using read but the read call doesn't
> > return (I'm using winpdb-reborn to monitor the operations).
> 
> Try readline(). Deadlocks may happen ;)
>  
> > I'm using Python 3.7, Windows 10, winpdb-reborn 2.0.0, rpdb2 1.5.0. If
> > it makes any difference, I'm trying to communicate with GDB using the MI
> > interpreter.
> > 

Per Peter's suggestion I tried readline and it works "as expected". I 
also discovered that the reason the read operations were stalling was 
that they followed a write and the write doesn't actually occur until 
"flush" is called even though the function call returns. There are 
undoubtedly some subtleties lurking about, but at least I'm making good 
progress.

Thanks for the help!

Dick


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Popen confusion

[Dick Holmes]

In article , 
__pete...@web.de says...
> 
> Dick Holmes wrote:
> 
> > https://occovid19.ochealthinfo.com/coronavirus-in-oc 
> 
> > I'm trying to
> > communicate using a continuing dialog between two
> > processes on the same system.
> 
> I think pexpect
> 
> https://pexpect.readthedocs.io/en/stable/index.html
> 
> does this naturally, but I don't know if Windows support is sufficient for 
> your needs.
> 
> > I've looked at various mechanisms and the
> > class that seems to fit my needs is Popen in the subprocess module, but
> > I can't seem to get more than a single round-trip message through Popen.
> > I first call Popen then poll using the identifier returned from the call
> > and the poll seems to work. I then call the communicate function passing
> > None as the value to send to the companion process stdin. I get the
> > expected result, but I also get "Exception condition detected on fd 0
> > \\n" and "error detected on stdin\\n". Subsequent attempts to
> > read/write/communicate with the subprocess fail because the file (stdxx
> > PIPE) is closed.
> > 
> > I can't tell from the documentation if the communicate function is a
> > one-time operation. 
> 
> Yes, communicate() is one-off, 
> 
> 
> """Interact with process: Send data to stdin and close it.
> Read data from stdout and stderr, until end-of-file is
> reached.  Wait for process to terminate.
> ...
> """
> 
> seems pretty clear. What would you improve?

Peter - thanks for the clarification. I'm using the 3.6.5 CHM 
documentation and it doesn't mention the phrase "and close it".
> 
> > I have tried using read but the read call doesn't
> > return (I'm using winpdb-reborn to monitor the operations).
> 
> Try readline(). Deadlocks may happen ;)
>  
> > I'm using Python 3.7, Windows 10, winpdb-reborn 2.0.0, rpdb2 1.5.0. If
> > it makes any difference, I'm trying to communicate with GDB using the MI
> > interpreter.
> > 


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Popen confusion

[Terry Reedy]

On 5/13/2020 11:13 PM, Dick Holmes wrote:

https://occovid19.ochealthinfo.com/coronavirus-in-oc I'm trying to
communicate using a continuing dialog between two
processes on the same system


Try multiprocessing and pipes.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Popen confusion

[Peter Otten]

Dick Holmes wrote:

> https://occovid19.ochealthinfo.com/coronavirus-in-oc 

> I'm trying to
> communicate using a continuing dialog between two
> processes on the same system.

I think pexpect

https://pexpect.readthedocs.io/en/stable/index.html

does this naturally, but I don't know if Windows support is sufficient for 
your needs.

> I've looked at various mechanisms and the
> class that seems to fit my needs is Popen in the subprocess module, but
> I can't seem to get more than a single round-trip message through Popen.
> I first call Popen then poll using the identifier returned from the call
> and the poll seems to work. I then call the communicate function passing
> None as the value to send to the companion process stdin. I get the
> expected result, but I also get "Exception condition detected on fd 0
> \\n" and "error detected on stdin\\n". Subsequent attempts to
> read/write/communicate with the subprocess fail because the file (stdxx
> PIPE) is closed.
> 
> I can't tell from the documentation if the communicate function is a
> one-time operation. 

Yes, communicate() is one-off, 


"""Interact with process: Send data to stdin and close it.
Read data from stdout and stderr, until end-of-file is
reached.  Wait for process to terminate.
...
"""

seems pretty clear. What would you improve?

> I have tried using read but the read call doesn't
> return (I'm using winpdb-reborn to monitor the operations).

Try readline(). Deadlocks may happen ;)
 
> I'm using Python 3.7, Windows 10, winpdb-reborn 2.0.0, rpdb2 1.5.0. If
> it makes any difference, I'm trying to communicate with GDB using the MI
> interpreter.
> 
> Thoughts and advice appreciated!
> 
> Dick


-- 
https://mail.python.org/mailman/listinfo/python-list

Subprocess Popen confusion

[Dick Holmes]

https://occovid19.ochealthinfo.com/coronavirus-in-oc I'm trying to 
communicate using a continuing dialog between two 
processes on the same system. I've looked at various mechanisms and the 
class that seems to fit my needs is Popen in the subprocess module, but 
I can't seem to get more than a single round-trip message through Popen. 
I first call Popen then poll using the identifier returned from the call 
and the poll seems to work. I then call the communicate function passing 
None as the value to send to the companion process stdin. I get the 
expected result, but I also get "Exception condition detected on fd 0
\\n" and "error detected on stdin\\n". Subsequent attempts to 
read/write/communicate with the subprocess fail because the file (stdxx 
PIPE) is closed.

I can't tell from the documentation if the communicate function is a 
one-time operation. I have tried using read but the read call doesn't 
return (I'm using winpdb-reborn to monitor the operations). 

I'm using Python 3.7, Windows 10, winpdb-reborn 2.0.0, rpdb2 1.5.0. If 
it makes any difference, I'm trying to communicate with GDB using the MI 
interpreter.

Thoughts and advice appreciated!

Dick
-- 
https://mail.python.org/mailman/listinfo/python-list