On Fri, 5 Feb 2010 12:42:47 -0500 (EST), Bob McGowan wrote: > Stephen Powell wrote: >> This is off topic from the OP's question, but one of the things that I >> miss in the Linux environment that I used to use a lot in the CMS >> environment is CMS Pipelines. The shell supports pipelines, but they >> are *single-stream* pipelines. CMS pipelines supports *multi-stream* >> pipelines. That one feature alone makes CMS Pipelines so much more >> powerful than shell pipelines. I wish the shell supported multi-stream >> pipelines. > > As I'm not familiar with CMS, I have no idea if the following matches or > is completely off target ... > > The default pipeline in shells always operates with stdout/stdin, which > is a bit limiting. > > But, you can work around it, a bit, like this: > > cat abc_does_not_exist 2>&1 | wc > > The 'cat' error message gets sent to wc. The normal cat output is still > to stdout, so it also would be processed by wc, when the file exists. > > The shell also allows you to open your own descriptors to files, using: > > exec 5>abc > date >&5 > cat abc > Fri Feb 5 09:15:15 PST 2010 > > Or, you can set the descriptor to point to another one: > > exec 5>&1 > date >&5 > Fri Feb 5 09:15:15 PST 2010 > > You can also create "named pipes" using 'mknod a_name p'. For example: > > mknod fifo1 p > mknod fifo2 p > tr '[A-Z]' '[a-z]' < fifo1 >fifo2 & > [1] 25269 > echo ABCDEFG > fifo1 > cat fifo2 > abcdefg > [1]+ Done tr '[A-Z]' '[a-z]' <fifo1 >fifo2 > > Perhaps one or another of the above will do what you want?
These are all helpful tips and tricks, and I'll keep them in mind. I really won't know if I can do the kind of things I want with these until I try, I suppose. I don't have a specific application in mind right now. Let me give you a simple example of a CMS pipeline for illustrative purposes, so that you will get some idea of what I'm talking about. This is a REXX EXEC (i.e. a script written in the REXX interpretive language) designed to run on the CMS operating system running in a virtual machine under z/VM: ---------- 'PIPE (ENDCHAR ?)', /* Declare pipeline endchar. */ ' < USER DIRECT|', /* Read directory file. */ ' NFIND *|', /* Discard comments. */ 'LOCM: LOCATE /MDISK/|', /* Select MDISK records */ 'FIN: FANINANY|', /* Collect MDISKs and LINKs. */ ' > OUTPUT FILE A', /* Write them to a file. */ '?', /* End of stream. */ 'LOCM: |', /* Non-MDISK cards to here */ ' LOCATE /LINK/|', /* Select LINK cards. */ 'FIN:' /* Send them to faninany. */ /* End of stream and pipe */ ---------- This is all one command, as viewed by the REXX interpreter. The stuff between /* and */ are comments. They are ignored by the interpreter. Note that all lines end in a comma, which is the continuation character, except the last. The vertical bar, as with shell pipelines, is the stage separator. The apostrophes serve as quote characters, just as in the shell. PIPE is the name of the command to invoke. The rest of the stuff is just arguments to the PIPE command. In other words, the REXX interpreter itself has no concept of a pipeline. It is just passing a string of characters to a CMS command called PIPE. It is the PIPE command that understands what a pipeline is. The question mark is defined as an end-of-stream character. It marks the end of a section of pipe, if you will. The pipe consists of a number of stages. The stage numbers and their names are as follows: 1. < 2. NFIND 3. LOCATE 4. FANINANY 5. > 6. LOCATE Don't be confused by "<" and ">". They are not redirection operators. There's no such thing as a redirection operator in REXX or in CMS Pipelines. "<" and ">" are the names of stages, just like the others. Stage 2 and stage 6 are two different instances of the LOCATE stage. They have separate inputs and outputs. LOCM and FIN are labels. They are names given to some of the stages so that they can be referenced later. You can tell they are labels because they end with a colon. Stage 1, "<", reads records (lines) from a file, the name of which is USER DIRECT *. (The asterisk, meaning "on the first accessed disk you can find it on" is implied by default.) Input stream 1 is unconnected and not used, since it is the first stage. Records from the file are written to output stream 1. Stage 2, NFIND, has its input stream 1 connected to output stream 1 of stage 1. Records which do not begin with an asterisk in column 1 are written to output stream 1. Records which do begin with an asterisk are written to output stream 2, if it is connected. Otherwise, they are discarded. Since output stream 2 is not connected, they are discarded. Stage 3, LOCATE, has its input stream 1 connected to output stream 1 of stage 2. Records containing the character string "MDISK" are written to output stream 1. Records which do not contain the character string "MDISK" are written to output stream 2, if it is connected. Otherwise, they are discarded. As we will later see, output stream 2 is connected; so these records are not discarded. This stage is given the label LOCM so that it can be referred to later. Stage 4, FANINANY, has two connected input streams. Input stream 1 is connected to output stream 1 from stage 3. We will see later what is connected to input stream 2. FANINANY reads from whichever input stream has a record ready and writes it to output stream 1. Stage 5, ">", has its input stream 1 connected to output stream 1 of stage 4. ">" writes these records to a file, whose name is OUTPUT FILE on the "A" disk. Output stream 1 is unconnected, since it is followed by the end-of-stream character. It is not used. Stage 6, the second LOCATE stage, has its input stream 1 connected to output stream 2 of stage 3, since it is preceded by "LOCM: |". Therefore input records from stage 3 which do not contain the character string "MDISK" end up coming in on input stream 1 of stage 6. If any of these records contain the character string "LINK", they are written to output stream 1. Records which do not contain the character string "LINK" are written to output stream 2, if it is connected. Otherwise, they are discarded. Since output stream 2 is not connected, these records are discarded. The trailing string "|FIN:" indicates that ouput stream 1 is to be connected to input stream 2 (since it is the second reference to the label) of the FANINANY stage, stage 4. This pipeline can be illustrated graphically as follows: +---+ +-------+ +--------+ +----------+ +---+ |-| < |-->| NFIND |-->| LOCATE |------>| FANINANY |-->| > |-| +---+ +-------+ +--------+ 1 1 +----------+ +---+ | 2 | 2 V A | +--------+ | LOCM:-->| LOCATE |-->-FIN: +--------+ (Obviously, you must be viewing this using a non-proportional font or it will look terrible.) The net result is that, using a single pass through the input file, records which contain the character string MDISK or LINK, not including records that have been commented out, are extracted from the input file and written to the output file. This is a very simple example of a multi-stream pipeline. Much more complex pipeline topologies can be defined, as you can imagine. And there are lots of input device stages, output device stages, filter stages, and stages which do various other kinds of processing and transformations. Perhaps this gives you a flavor for CMS Pipelines. I don't see anything like that available in shell pipelines. Sorry for the long post, but I thought a specific example would help. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org