It could be interesting to do the streaming on top of Apache Kafka because both systems work well with Avro serialization.
On Fri, Sep 14, 2012 at 11:17 AM, Josh Wills <[email protected]> wrote: > I like the idea of having themes for releases. In my head, the theme of > this release could be either > > a) Hacking the new MSCRPlanner code, esp. to add the ability to fuse > different MSCR jobs into a single instance that it enables, or > b) data access/integration points-- things like solr, hcatalog, hbase, > cassandra, jdbc, etc. as input and output sources for Crunch pipelines, or > c) API refactoring-- the crunch-api/crunch-impl/crunch-lib split, or > d) working on a PStream API that would let people apply DoFns to streams > and would build on top of things like WalMart's mupd8 or Storm or whatever. > > Of course, this is in addition to whatever fixes and new lib functions we > want to add over time. I don't want anything heavyweight, but those are > some of the larger-scale things that we'll need to tackle as a community, > and I would think of completing each of those big things as corresponding > to a release. > > Just my two cents. > > J > > On Fri, Sep 14, 2012 at 10:23 AM, Matthias Friedrich <[email protected]> wrote: > > > Hi, > > > > should we discuss the focus of our next release? Maybe make a list > > of things we want to achieve? Or would this be too much process? > > > > Regards, > > Matthias > > > > > > -- > Director of Data Science > Cloudera <http://www.cloudera.com> > Twitter: @josh_wills <http://twitter.com/josh_wills> >
