This is awesome work you've done.
There are two or three strong implementations of Pipelines for
non-CMS/TSO environments. I don't mean to dis any of them. I'm
looking for something written in C for maximum efficiency and
portability. Having a core that is pure C does not prohibit some of
the STAGES being written in C++ or anything else.
Some months back, I began to assemble the following into runnable
code. It is not complete, but the idea is ...
two Unix pipe() pairs for each effective Pipelines connection
wrapper functions output(), peekto(), and readto() are illustrated below
The two Unix pipes go opposite directions. One goes forward and
carries data (duh!) and also any meta data (such as length of the
record available, if any) from producer to consumer. The other goes
backward and carries requests and feedback from consumer to producer.
So a record flows something like ...
producer calls output()
consumer calls peekto()
under the covers, peekto() sends "STAT" backward
output() sends metadata forward (most interesting being size
of the record)
assuming the buffer can hold the record, peekto() sends "PEEK" backward
output() then sends the data
A readto() does all of the above plus sends "NEXT" at the end and
output() can return. When output() returns, the producer is unblocked
and we get proper record flow. (You can dealy the record ... or not!)
The design starts with two Unix pipes but can use shared mem if the
platform supports that. The code is accomodating. (A NULL pointer to
a shared mem struct means don't use shared mem.)
-- R; <><
On Tue, Aug 31, 2010 at 04:17, James Laing <james_la...@hotmail.co.uk> wrote:
> Pipelines was never intended to represent a port of CMS Pipelines - nor was
> it originally intended for anything other than a useful tool for those times
> when you might need to modify/translate some data.
>
> My background was as a VM/TPF systems programmer and one day I realised that
> I could really use a tool 'like' CMS Pipelines - but there wasn't one on my
> Windows platform. So I wrote Pipelines.
>
> James.
>
