On Thu Feb 12 2015 at 11:09:35 AM Robert Douglas < [email protected]> wrote:
> Hey Steve, I think this is a great idea. Streams has a ton of use cases and > I think to a new user, they can be a bit daunting. I'd be happy to help > shore up some of the examples. > I would recommend having infra setup a separate git repository and adding modules to that from scratch, rather than importing them from another repository. > > -- Robert > > On Wed, Feb 11, 2015 at 9:53 AM, Steve Blackmon <[email protected]> > wrote: > > > Reviving an old discussion: > > > > I propose that we create an official streams-examples repository in the > > apache git repository with the following characteristics: > > > > - maven multi-level multi-module project > > - reference streams-project as parent > > - first level : runtime (local, dropwizard, pig, storm, etc...) > > - below, individual examples modules built upon that runtime. > > > > Each example would: > > > > - Contain a primary class which uses a StreamsBuilder to launch an > > example stream, or set of complementary streams. > > - Alternatively, contain a script suitable for execution within a > > specific runtime (pig/spark shell for example) > > - Optionally, contain processors and utility classes suited to a > > specific purpose within > > - Optionally, override the behavior of providers, processors, utility > > classes from the core project to achieve a specific purpose > > - Contain a configuration class which declares all required and > optional > > configuration > > - Contain a README.md conforming to a template TBD, describing what > the > > example does > > - Contain at least one example configuration in src/main/resources and > > in README.md > > - Contain a machine readable graph definition (dot or suitable > > alternative) and png version visible in README.md > > - Contain integration test(s) demonstrating that a small input set > > traverses the stream, resulting in expected outputs and (optionally) > > properly handled exceptions. > > - Describe requirements for a successful deployment and execution in > > README.md. > > - Build an executable uber-jar > > - Build an executable docker image > > > > https://github.com/steveblackmon/streams-examples is a collection of > > modules that conform in varying degrees to this pattern. Less organized > > and less tested than I'm proposing, but containing code & poms that are > > minimal, readable, and useful "out-of-the-box". I'll commit to > increasing > > the quality of 5-10 of them and submitting them to > > > > Anyone agree this is a good idea? Have concerns? Want to help? > > > > Steve Blackmon > > [email protected] > > >
