On Wednesday, February 09, 2011 03:19:03 pm you wrote:
> Yeah, it sound weird. What I have is 72 files containing a lot of secuity
> data from our z/OS RACF system. To save space, all these files are
> bzip2'ed - each individually. I am writing some Perl scripts to process
> this data. The Perl script basically reformats the data in such a way that
> I can put it easily into a PostgreSQL database. If I want to run each Perl
> script individually, it is simple:
>
> bzcat data*bz2 | perl script1.pl | psql database
> bzcat data*bz2 | perl script2.pl | psql database
>
> and so on. I don't want to try to merge the scripts together into a single,
> complicated, script. I like what I have in that regard. But I don't like
> running the bzcat twice to feed into each Pel script. Is something like
> the following possible?
>
> mkfifo script1.fifo
> mkfifo script2.fifo
> bzcat data*bz2 | tee script1.fifo >script2.fifo &
> perl script1.pl <script1.fifo &
> perl script2.pl <script2.fifo &
>
> ???
>
> What about more than two scripts concurrently? What about "n" scripts?
Using tee is the right approach, and the above should work OK. Solving this
problem for N outputs is a bit trickier, because you have to have something
that copies its input N times. That could be done with a shell loop. Here's
a function that copies its stdin to each of the files named on its command
line:
Ntee() {
while read line; do
for file; do
echo "$line" >> "$file"
done
done
}
Well, that does it, but it is opening each file and seeking to its end for
each line of input, and that's pretty inefficient. What we'd like to do is
keep the files open. Something like this might do it, but I haven't tested
it:
Ntee() {
fd=3
for file; do
eval $fd'>"$file"'
fd=$((fd + 1))
done
while read line; do
fd=3
for file; do
eval 'echo "$line" 1>'$fd
fd=$((fd + 1))
done
done
}
The first for-loop opens all the files and assigns file descriptors to them,
and the second for-loop writes to those open file descriptors. The eval is
used to expand the $fd (the rest of the command is protected from evaluation
be single-quotes) because the file-redirection syntax requires a number. So,
for example, the first time around the first loop, the command:
3>"$file"
is what gets executed.
I haven't tried to run this, but the idea might help.
- MacK.
-----
Edmund R. MacKenty
Software Architect
Rocket Software
275 Grove Street - Newton, MA 02466-2272 - USA
Tel: +1.617.614.4321
Email: m...@rs.com
Web: www.rocketsoftware.com
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
