On 5/8/07, Paul Raulerson <[EMAIL PROTECTED]> wrote:
No, I have looked, and CMS Pipelines are nice indeed. But then so are pipes under UNIX; indeed, pipes are the very core of UNIX. If you are not annoyed by discussing it, I would love to hear your opinions on what is so primitive about UNIX. :)
As I said: leaky garden hose. The analogy holds just as long as you use terms like "nice" * CMS Pipelines has multi stream pipelines which means that you can divert part of the input stream and have that go through a different segment of the pipeline and further down the pipe the streams can join again when desired. The closest you get in UNIX is something like the "tee" program. * The stages in CMS Pipelines are not limited by a single input and output (and stderr), but can have many streams which allows for building complex refineries without the need for endless copies of the data. * The way records are moving through the pipeline and the way stages interact means that you can reason about where records are and guarantee the order in which data is produced and consumed in parallel pipeline segments. * Dynamic changes to the topology of the pipeline where a stage can replace itself by a newly composed segment either permanently or temporary (a sipping pipeline). Combined with the strict order in which data is consumed, you control what part of the data flows through the modified pipeline. I do believe I am one of those many VM people who embraced Linux and the concepts are not alien to me (I avoid the term "transition" because that would suggest going from one to the other). Recently I wrote a simple Perl program - ptime - to take lines from stdin and write them out prefixed with the local time. To my surprise the following did not work to tag vmstat output with the time as I intended: vmstat 10 | ptime Turns out that something is doing an undetermined amount of buffering (and yes, I learned that I can set the "$|" variable (?) to change that). And there's many more cases where the tools violate the Principe of Least Astonishment. Things like njpipes and OS/2 pipe ran short of that and turned out to be far less useful. I understand I have the option to write a C program from scratch to do what I want, or maybe copy an old one from when I wanted almost the same. We've done so with Rexx for quite some time. However, I find it way more productive to compose a pipeline out of many built-in stages and maybe a few reusable ones from myself in Rexx. Rob
