Rob van der Heij wrote:
On Mon, Nov 2, 2009 at 5:02 AM, BISHOP, Peter <peter.bis...@hp.com> wrote:
To John M - yes, I was thinking in "z/OS" terms, where a single open of the
DDNAME is sufficient for all the datasets in that DDNAME.
To David, Ed and John S - the annoying thing about pipes here is that they incur extra
I/O for the "cat" command, and since these files are very large (approx 20GB
daily) this extra I/O will be very costly, probably excessively so here.
Looks like I'm out of luck here, more thinking required.
What extra I/O do you mean? When "cat" has multiple arguments, it will
open one file after the other, read them to end-of-file and write the
output to stdout. You can pipe that into your processing. If it's your
own application, you could make the application process a list of file
names. As far as I know this is similar to how MVS does extents.
You would have extra I/O when you had to compose the input file on
disk by appending all your input files. But in a lot of cases you can
make the program use stdin and take the data from the pipe.
Rob
I expect Peter's referring to writing and reading the pipes. In OS, the
data goes from disk to buffers to program. Here, it would be going disk
to buffers to cat to buffers(?) to pipe to buffers(?) to program.
I think Peter should measure the performance, but I'm inclined to think
he's right. When I suggested it, I didn't expect it to be "as good."
If there's some useful preprocessing that could be done before the
program, he could replace cat with a program that does that processing
and reduce the overhead.
--
Cheers
John
-- spambait
1aaaa...@coco.merseine.nu z1aaaa...@coco.merseine.nu
-- Advice
http://webfoot.com/advice/email.top.php
http://www.catb.org/~esr/faqs/smart-questions.html
http://support.microsoft.com/kb/555375
You cannot reply off-list:-)
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
