On 12Mar2015 12:55, John Nagle <na...@animats.com> wrote:
I have working code from Python 2 which uses "pickle"
to talk to a subprocess via stdin/stdio. I'm trying to
make that work in Python 3.
First, the subprocess Python is invoked with the "-d' option,
so stdin and stdio are supposed to be unbuffered binary streams.
You shouldn't need to use unbuffered streams specificly. It should be enough to
.flush() the output stream (at whichever end) after you have written the pickle
data.
I'm skipping some of your discussion; I can see nothing wong. I don't use
pickle itself so aside from saying that your use seems to conform to the python
3 doco I can't comment more deeply. That said, I do use subprocess a fair bit.
[...]
result = self.reader.load()
TypeError: 'str' does not support the buffer interface
That's as far as traceback goes, so I assume this is
disappearing into C code.
No line numbers at all? Or, I suppose, just the line number from your program
and nothing from the pickle module?
OK, I know I need a byte stream. I tried
self.reader = pickle.Unpickler(self.proc.stdout.buffer)
self.writer = pickle.Pickler(self.proc.stdin.buffer, 2)
You should not need to care about these. They're not required.
That's not allowed. The "stdin" and "stdout" that are
fields of "proc" do not have "buffer". So I can't do that
in the parent process. In the child, though, where
stdin and stdout come from "sys", "sys.stdin.buffer" is valid.
But irrelevant. Besides, the stream buffer may not contain the whole pickle
data anyway; it will be empty before a read and quite possibly incomplete
afterwards. It is just a buffer.
That fixes the ""str" does not support the buffer interface
error."
I'm not sure "fix" is the right characterisation here.
But now I get the pickle error "Ran out of input"
on the process child side. Probably because there's a
str/bytes incompatibility somewhere.
No, probably because the buffer is only ever a snapshot of part of the stream.
str/bytes errors are more glaringly obviously so.
So how do I get clean binary byte streams between parent
and child process?
This is where I'm confused: my experience is that Popen.subprocess gives you
binary streams; I always need to put an encoder/decoder on them to use text.
Did that just the other day.
BTW, this is on some UNIX variant? Should not be very relevant...
Further questions:
What does self.proc.stdout.__class__ say? And for stdin?
Cheers,
Cameron Simpson <c...@zip.com.au>
My opinions are borrowed from someone who no longer needs them.
-- katma...@uga.cc.uga.edu
--
https://mail.python.org/mailman/listinfo/python-list
