I have collected some open topics/questions for discussion already from folks who already reviewed the code
1.The name, name of the Stream and the StreamSource 2. Build dag in an incremental way vs lazy population. Incremental way is easier to implement (what I did right now) and it create one edge for the dag for each transformation method. Lazy population means keep the method chain in memory until it needs to submit the dag either locally or to the cluster, in this way, some optimization(change order of transforation ex.) might be done because you have an overall picture. 3. How to easily extend the Stream interface and it's implementation 4. How to deal with operator with multiple input ports/output ports. Again, I appreciate any ideas and suggestion for those questions above. And feel free to ask more questions you have Regards, Siyuan On Mon, Apr 4, 2016 at 11:30 PM, Siyuan Hua <[email protected]> wrote: > Hi community, > > I have submitted my first commit of stream api into my public repository > here > https://github.com/siyuanh/incubator-apex-malhar/tree/stream > > You can think this is the prototype of the Java Stream API proposal here > > https://docs.google.com/document/d/163LmQjX860b61NDe3ZzR0hRTPtE-4GF0iHaVhmHQssY/edit#heading=h.aytn6rz7u1e4 > > A simple walkthrough of the code: > > ApexStream is the core interface to build a dag in stream style. Default > implementation is in ApexStreamImpl > > Function is a super interface for all simple transformation, it has > several sub interfaces like MapFunction, ReduceFunction etc. > > FunctionOperator is a wrapper for functions that pass param from input > port to function and deliver the return value to output port. > > And you can find the word count demo code below > > > https://github.com/siyuanh/incubator-apex-malhar/blob/stream/demos/wordcount/src/main/java/com/datatorrent/demos/wordcount/ApplicationWithStreamAPI.java > > > > As we want to release this API asap. We want the whole community to help > define a clear scope of what we want to achieve in the first cut. Any > suggestions, ideas are very welcome. > > Please please do contribute to this :) > > Thanks! > Siyuan > > >
