Peter Alexander wrote: > Hi Martin, > > On Wed, Jun 9, 2010 at 12:09 PM, Martin Sustrik <sust...@250bpm.com> wrote > >> I believe the "send_requests" is the initial node of the pipeline. It >> produces request. Down the stream there's arbitrary number of >> "component"s which do the actual processing and generate replies. >> Further down there's "receive_replies" component that gathers all the >> replies. > > A pipeline shouldn't be considered in the request/reply paradigm, but > rather in the context of "pushing data". In other words, the pattern > is data centric.
Yes. The "send_requests" and "receive_replies" are not particularly well chosen names. > The entire string of components should be thought of as a virtual > single process that has an input and an output where the first "root" > node has the responsibility of obtaining the data (input) which is to > be passed along to the following nodes. The string/pipeline is a > directed, normally acyclic, graph. The last "leaf" node(s) is then > responsible as an output of the data in it's final form to, for > instance, a file, database, stdout, or a display (like a graph/plot). > > Each inner component/node, within the string/pipeline, has a > responsibility to massage or derive-from the dataset buffer which has > been passed to it. > > Each component/node knows apriori, how to handle this "chunk" of data > that it has been handed. If its a raw chunk of data, for instance, > then it is familiar with a dataset descriptor that allows it to > interpret the chunk into some sort of workable structure. > > Each component/node is a _single_ "black box" process that has an > input(s) and an output(s) just like a typical function call. > > Request/Reply is not needed because that is _implied_ by the existence > of the component/node itself.. within the string/pipeline. > > Finally, a pipeline can be an assembly of other (pre-composed) "sub" > pipelines. Think "cut and paste".. ;) > > This is a very.. very flexible and powerful programming paradigm. I think we are missing this kind of discussion of pipeline paradigm in the documentation. Once I wrote this article: http://www.zeromq.org/tutorials:butterfly However is misses the overall introduction (and the code examples use historic version of 0MQ with completely different API). Maybe the text above can be tweaked to serve as an introduction. That can be followed by discussion of performance issues (as in my article), and an example of actual code. What do you think? Martin _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
