On Wed, Sep 23, 2020 at 11:53:10PM -0400, Lawrence Velázquez wrote: > > On Sep 23, 2020, at 11:41 PM, Vito Caputo <vcap...@pengaru.com> wrote: > > > > On Wed, Sep 23, 2020 at 09:12:40AM -0400, Chet Ramey wrote: > >> On 9/22/20 11:23 PM, Vito Caputo wrote: > >>> Hello list, > >>> > >>> Is there any chance we could get a | modifier for enabling O_DIRECT on the > >>> created pipe? "Packet" style pipes have some interesting and potentially > >>> useful properties, it would be nice if bash made them more accessible. > >> > >> Is there a general need, especially since they're Linux-specific? > >> > > > > I'm not sure, but as far as GNU/Linux distros go bash, is kind of the > > canonical shell, and this functionality is kind of inaccessible > > without the shell wiring it up. > > What functionality? I (and I'm sure some others) am not familiar > with packet-style pipes and their benefits. You haven't actually > described *how* exposing them would be useful, and why that would > justify introducing new syntax that only matters/works on Linux. >
Packetized pipes establish well-defined boundaries between writes reproduced at the read side. If the write sizes are kept within PIPE_BUF bounds, then you can be certain what's read is an atomic record including nothing from a subsequent or previous write, with no possibility for partial records. It's useful if you're doing something like say, aggregating data from multiple piped sources into a single bytestream. With the default pipe behavior, you'd have the output interleaved at random boundaries. With packetized pipes, if your sources write say, newline-delimited text records, kept under PIPE_BUF length, the aggregated output would always interleave between the lines, never in the middle of them. If we added this to the shell, I suppose the next thing to explore would be how to get all the existing core shell utilities to detect a packetized pipe on stdout and switch to a line-buffered mode instead of block-buffered, assuming they're using stdio. That should turn their lines into packets on the pipe, and it all becomes generally relevant across the existing shell utils landscape. This heuristic echoes of the terminal output detection for stdout line-buffering already performed according to setvbuf(3). Thanks, Vito Caputo