Re: [Rd] pipe(): input to, and output from, a single process
Dirk, > Octave had this already in the 1990s, see documentation for 'popen2' here: thanks. unix that had since the 1970s... :) cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
Simon, > FWIW if you're on unix, you can use named pipes (fifos) for that: i've always wondered what named pipes actually were. thanks! cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
Gabor, thanks. yes, managing the two-way communication is always a bit error-prone, as it depends on the input/output characteristics of the two ends -- they either match, or deadlock. it's too bad if polling is always *required* -- i'd think sometimes a programmer would be happy blocking, though other times one wants better control over when to block. cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
FWIW if you're on unix, you can use named pipes (fifos) for that:
> system("mkfifo my.output")
> p = pipe("sed -l s:hello:oops: > my.output", "w")
> i = file("my.output", "r", blocking=FALSE, raw=TRUE)
> writeLines("hello!\n", p)
> flush(p)
> readLines(i, 1)
[1] "oops!"
Cheers,
Simon
> On 14/03/2020, at 6:26 AM, Greg Minshall wrote:
>
> hi. i'd like to instantiate sed(1), send it some input, and retrieve
> its output, all via pipes (rather than an intermediate file).
>
> my sense from pipe and looking at the sources (sys-unix.c) is that is
> not possible. is that true? are there any thoughts of providing such a
> facility?
>
> cheers, Greg
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
Well, if you want blocking, you can poll with an infinite timeout. This returns if 1) there is output, 2) the process terminates, or 3) you interrupt with CTRL+C / ESC /etc. and then right after the polling, you can read the output. This still works if the process has finished already. Gabor On Mon, Mar 16, 2020 at 7:06 PM Greg Minshall wrote: > > Gabor, thanks. yes, managing the two-way communication is always a bit > error-prone, as it depends on the input/output characteristics of the > two ends -- they either match, or deadlock. it's too bad if polling is > always *required* -- i'd think sometimes a programmer would be happy > blocking, though other times one wants better control over when to > block. cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
On 13 March 2020 at 20:26, Greg Minshall wrote: | hi. i'd like to instantiate sed(1), send it some input, and retrieve | its output, all via pipes (rather than an intermediate file). | | my sense from pipe and looking at the sources (sys-unix.c) is that is | not possible. is that true? are there any thoughts of providing such a | facility? Octave had this already in the 1990s, see documentation for 'popen2' here: https://octave.org/doc/v4.2.1/Controlling-Subprocesses.html As it says 'Start a subprocess with two-way communication'. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
On Fri, 13 Mar 2020 20:26:43 +0300 Greg Minshall wrote: > my sense from pipe and looking at the sources (sys-unix.c) is that is > not possible. is that true? are there any thoughts of providing > such a facility? Pipes (including those created by popen(3), which R pipe() uses internally) are uni-directional data channels. While it could be possible to open two pipes for both stdin and stdout of the child process, doing so correctly is complicated because of differences in buffering provided by the runtime: when stdin/stdout is not a terminal, buffering mode may be set to block-oriented instead of line-oriented, resulting in both parent and child being dead-locked, waiting to fill the buffer instead of returning from the blocking call after the first newline. (Hence the -l flag to sed mentioned by Gábor Csárdi, which avoids this problem for sed). Programs designed to first read stdin until end-of-file, then process the input and print results on the stdout are usually safe to use in this way, but others may be not. Software specifically designed to control other software (e.g. Expect [*]) gets around this limitation by running the child processes inside pseudo-terminals and/or running in event-driven manner, being ready to service the child process whether it wants to read its stdin or write to stdout. Since sed has its -l flag, it should be possible to safely drive it line-by-line with the help of processx, but not via pipe(). -- Best regards, Ivan [*] https://core.tcl-lang.org/expect/index __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pipe(): input to, and output from, a single process
I am not sure if `pipe()` works for this, but if it turns out that it
does not, then you can use the processx package, e.g.:
> p <- processx::process$new("sed", c("-l", "s/a/x/g"), stdin = "|", stdout =
> "|")
> p$write_input("foobar\n")
> p$read_output()
[1] "foobxr\n"
The `-l` sed flag is to make sed line-buffered, otherwise it is will
not produce output until there is enough.
`$write_input()` and `$read_output()` are not easy to program, in particular:
* `$write_input()` returns the chunk of data that it hasn't managed to
write into the pipe. You need to call `$write_input() again, with this
data next, usually.
* `$read_output()` returns an empty string if there is no data to
read, so typically you want to call `p$poll()` first, to make sure
that there is something to read.
* `$read_output()` might not read whole lines, so maybe
`$read_output_lines()` is better for you.
* Close the stdin of the process if you want to quit cleanly:
`close(p$get_input_connection())`.
* There is currently no way to poll the input side of the pipe. :(
HTH, Gabor
On Mon, Mar 16, 2020 at 11:31 AM Greg Minshall wrote:
>
> hi. i'd like to instantiate sed(1), send it some input, and retrieve
> its output, all via pipes (rather than an intermediate file).
>
> my sense from pipe and looking at the sources (sys-unix.c) is that is
> not possible. is that true? are there any thoughts of providing such a
> facility?
>
> cheers, Greg
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] pipe(): input to, and output from, a single process
hi. i'd like to instantiate sed(1), send it some input, and retrieve its output, all via pipes (rather than an intermediate file). my sense from pipe and looking at the sources (sys-unix.c) is that is not possible. is that true? are there any thoughts of providing such a facility? cheers, Greg __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
