Re: Flush stdin

2014-10-22 Thread Chris Angelico
On Wed, Oct 22, 2014 at 4:38 PM, Marko Rauhamaa ma...@pacujo.net wrote: Dan Stromberg drsali...@gmail.com: On Mon, Oct 20, 2014 at 9:41 PM, Marko Rauhamaa ma...@pacujo.net wrote: Nagle affects the communication between the peer OS kernels and isn't directly related to anything the application

Re: Flush stdin

2014-10-22 Thread Marko Rauhamaa
Dan Stromberg drsali...@gmail.com: On Mon, Oct 20, 2014 at 9:41 PM, Marko Rauhamaa ma...@pacujo.net wrote: Terminal devices support line buffering on write. Yes, though that's not the only place it's useful. Line buffering on read is an illusion created by higher-level libraries. The

Re: Flush stdin

2014-10-22 Thread random832
driver via a variety of methods for portability. I wrote it in C before I took an interest in Python. Yes, and 90% of the time, when someone says they want to flush stdin, what they really want to do is go to the next line after they've sloppily read part of the line they're on (and the behavior

Re: Flush stdin

2014-10-22 Thread Marko Rauhamaa
random...@fastmail.us: Yes, and 90% of the time, when someone says they want to flush stdin, what they really want to do is go to the next line after they've sloppily read part of the line they're on (and the behavior they are seeing that they object to is that their next read function reads

Re: Flush stdin

2014-10-21 Thread Dan Stromberg
On Mon, Oct 20, 2014 at 9:41 PM, Marko Rauhamaa ma...@pacujo.net wrote: Dan Stromberg drsali...@gmail.com: Often with TCP protocols, line buffered is preferred to character buffered, Terminal devices support line buffering on write. Yes, though that's not the only place it's useful. Line

Re: Flush stdin

2014-10-21 Thread Cameron Simpson
On 21Oct2014 16:16, Dan Stromberg drsali...@gmail.com wrote: [...snip...] This is tremendously inefficient. It demands a context switch for every character. Inefficiency isn't an issue when you generate one byte a second. Of course, but who's doing one byte per second? You and I in our

Re: Flush stdin

2014-10-21 Thread Nobody
On Sat, 18 Oct 2014 18:42:00 -0700, Dan Stromberg wrote: On Sat, Oct 18, 2014 at 6:34 PM, Dan Stromberg drsali...@gmail.com wrote: Once the nc process actually write()s the data to its standard output (i.e. desriptor 1, not the stdout FILE*) I'm not sure why you're excluding stdout, but even

Re: Flush stdin

2014-10-21 Thread Dan Stromberg
On Tue, Oct 21, 2014 at 7:49 PM, Nobody nobody@nowhere.invalid wrote: On Sat, 18 Oct 2014 18:42:00 -0700, Dan Stromberg wrote: On Sat, Oct 18, 2014 at 6:34 PM, Dan Stromberg drsali...@gmail.com wrote: Once the nc process actually write()s the data to its standard output (i.e. desriptor 1, not

Re: Flush stdin

2014-10-21 Thread Marko Rauhamaa
Dan Stromberg drsali...@gmail.com: On Mon, Oct 20, 2014 at 9:41 PM, Marko Rauhamaa ma...@pacujo.net wrote: Nagle affects the communication between the peer OS kernels and isn't directly related to anything the application does. Actually, Nagle can cause two or more small packets to be

Re: Flush stdin

2014-10-20 Thread Dan Stromberg
On Sun, Oct 19, 2014 at 9:45 PM, Marko Rauhamaa ma...@pacujo.net wrote: I found this comment in CPython's source code (pythonrun.c): /* stdin is always opened in buffered mode, first because it shouldn't make a difference in common use cases, second because TextIOWrapper

Re: Flush stdin

2014-10-20 Thread Dan Stromberg
If I run the following in one tty: nc -l localhost 9000 | /tmp/z ...where /tmp/z has just: #!/usr/bin/python3 import sys for line in sys.stdin.buffer: print(line) And then run the following in another tty on the same computer: while read line; do echo $line; sleep

Re: Flush stdin

2014-10-20 Thread Marko Rauhamaa
Dan Stromberg drsali...@gmail.com: On Sun, Oct 19, 2014 at 9:45 PM, Marko Rauhamaa ma...@pacujo.net wrote: I found this comment in CPython's source code (pythonrun.c): /* stdin is always opened in buffered mode, first because it shouldn't make a difference in common use cases,

Re: Flush stdin

2014-10-20 Thread Marko Rauhamaa
Dan Stromberg drsali...@gmail.com: ...then everything acts line buffered, or perhaps even character buffered [...] That, or we're using two different versions of netcat (there are at least two available). Let's unconfuse the issue a bit. I'll take line buffering, netcat and the OS out of

Re: Flush stdin

2014-10-20 Thread Dan Stromberg
On Mon, Oct 20, 2014 at 4:18 PM, Marko Rauhamaa ma...@pacujo.net wrote: Dan Stromberg drsali...@gmail.com: ...then everything acts line buffered, or perhaps even character buffered [...] That, or we're using two different versions of netcat (there are at least two available). Let's

Re: Flush stdin

2014-10-20 Thread Marko Rauhamaa
Dan Stromberg drsali...@gmail.com: Often with TCP protocols, line buffered is preferred to character buffered, Terminal devices support line buffering on write. Line buffering on read is an illusion created by higher-level libraries. The low-level read function reads in blocks of bytes.

Re: Flush stdin

2014-10-19 Thread Marko Rauhamaa
Cameron Simpson c...@zip.com.au: Even if nc itself does no buffering (handing data to the OS as soon as received, highly desirable for a tool like nc), the OS keeps a buffer for the pipeline between nc and python, Yes, there is a buffer associated with the pipe, but linux/unix never withholds

Re: Flush stdin

2014-10-19 Thread Cameron Simpson
On 18Oct2014 18:42, Dan Stromberg drsali...@gmail.com wrote: On Sat, Oct 18, 2014 at 6:34 PM, Dan Stromberg drsali...@gmail.com wrote: Once the nc process actually write()s the data to its standard output (i.e. desriptor 1, not the stdout FILE*) I'm not sure why you're excluding stdout, but

Flush stdin

2014-10-18 Thread Empty Account
Hi, I am using netcat to listen to a port and python to read stdin and print to the console. nc -l 2003 | python print_metrics.py sys.stdin.flush() doesn’t seem to flush stdin, so I am using the termios module. while True: input = sys.stdin.readline() # do some parsing

Re: Flush stdin

2014-10-18 Thread Chris Angelico
On Fri, Oct 17, 2014 at 10:38 PM, Empty Account empty...@gmail.com wrote: I will be using this script on Unix based systems and I wondered what approach I could use to flush stdin? Why exactly do you need to flush stdin? If you've written a small amount of data to the console, it's stdout

Re: Flush stdin

2014-10-18 Thread Cameron Simpson
On 17Oct2014 12:38, Empty Account empty...@gmail.com wrote: I am using netcat to listen to a port and python to read stdin and print to the console. nc -l 2003 | python print_metrics.py sys.stdin.flush() doesn’t seem to flush stdin, so I am using the termios module.  You're aware

Re: Flush stdin

2014-10-18 Thread Nobody
On Fri, 17 Oct 2014 12:38:54 +0100, Empty Account wrote: I am using netcat to listen to a port and python to read stdin and print to the console. nc -l 2003 | python print_metrics.py sys.stdin.flush() doesn’t seem to flush stdin, You can't flush an input stream. so I am using

Re: Flush stdin

2014-10-18 Thread Tim Chase
On 2014-10-18 17:55, Nobody wrote: On Fri, 17 Oct 2014 12:38:54 +0100, Empty Account wrote: I am using netcat to listen to a port and python to read stdin and print to the console. nc -l 2003 | python print_metrics.py sys.stdin.flush() doesn’t seem to flush stdin, You can't

Re: Flush stdin

2014-10-18 Thread MRAB
On 2014-10-18 17:55, Nobody wrote: On Fri, 17 Oct 2014 12:38:54 +0100, Empty Account wrote: I am using netcat to listen to a port and python to read stdin and print to the console. nc -l 2003 | python print_metrics.py sys.stdin.flush() doesn’t seem to flush stdin, You can't flush an input

Re: Flush stdin

2014-10-18 Thread Cameron Simpson
On 18Oct2014 17:55, Nobody nobody@nowhere.invalid wrote: On Fri, 17 Oct 2014 12:38:54 +0100, Empty Account wrote: I am using netcat to listen to a port and python to read stdin and print to the console. nc -l 2003 | python print_metrics.py sys.stdin.flush() doesn’t seem to flush stdin, You

Re: Flush stdin

2014-10-18 Thread Terry Reedy
sys.stdin.flush() doesn’t seem to flush stdin, You can't flush an input stream. Sure you can. You are collectively confusing three different meaning of 'flush'. Python and its docs are the best authority on what python can and cannot do. One can call .flush() on an imput stream (meaning 1

Re: Flush stdin

2014-10-18 Thread Nobody
sys.stdin.flush() doesn’t seem to flush stdin, You can't flush an input stream. You can't flush it, but you can make it unbuffered. You can either force python to use unbuffered stdio: [snipped] None of this helps in any way, as it's not the behaviour of the Python script which is causing

Re: Flush stdin

2014-10-18 Thread Chris Angelico
On Fri, Oct 17, 2014 at 10:38 PM, Empty Account empty...@gmail.com wrote: I am using netcat to listen to a port and python to read stdin and print to the console. nc -l 2003 | python print_metrics.py After lengthy discussion about what it means to flush stdin, I think it's high time someone

Re: Flush stdin

2014-10-18 Thread Dan Stromberg
to the console. nc -l 2003 | python print_metrics.py sys.stdin.flush() doesn’t seem to flush stdin, You can't flush an input stream. You can't flush it, but you can make it unbuffered. You can either force python to use unbuffered stdio: [snipped] None of this helps in any way

Re: Flush stdin

2014-10-18 Thread Dan Stromberg
On Sat, Oct 18, 2014 at 6:34 PM, Dan Stromberg drsali...@gmail.com wrote: Once the nc process actually write()s the data to its standard output (i.e. desriptor 1, not the stdout FILE*) I'm not sure why you're excluding stdout, but even if nc is using filedes 1 instead of FILE * stdout, isn't