On Fri, Sep 14, 2012 at 11:30 AM, Joseph Adler <[email protected]>wrote:
> It could be interesting to do the streaming on top of Apache Kafka because > both systems work well with Avro serialization. > Good point-- I like the idea of supporting Apache projects wherever possible-- hcat and solr being great examples of this. I'd love some thoughts on Crunch-for-Kafka from you and the rest of the Kafka community. > On Fri, Sep 14, 2012 at 11:17 AM, Josh Wills <[email protected]> wrote: > > > I like the idea of having themes for releases. In my head, the theme of > > this release could be either > > > > a) Hacking the new MSCRPlanner code, esp. to add the ability to fuse > > different MSCR jobs into a single instance that it enables, or > > b) data access/integration points-- things like solr, hcatalog, hbase, > > cassandra, jdbc, etc. as input and output sources for Crunch pipelines, > or > > c) API refactoring-- the crunch-api/crunch-impl/crunch-lib split, or > > d) working on a PStream API that would let people apply DoFns to streams > > and would build on top of things like WalMart's mupd8 or Storm or > whatever. > > > > Of course, this is in addition to whatever fixes and new lib functions we > > want to add over time. I don't want anything heavyweight, but those are > > some of the larger-scale things that we'll need to tackle as a community, > > and I would think of completing each of those big things as corresponding > > to a release. > > > > Just my two cents. > > > > J > > > > On Fri, Sep 14, 2012 at 10:23 AM, Matthias Friedrich <[email protected]> > wrote: > > > > > Hi, > > > > > > should we discuss the focus of our next release? Maybe make a list > > > of things we want to achieve? Or would this be too much process? > > > > > > Regards, > > > Matthias > > > > > > > > > > > -- > > Director of Data Science > > Cloudera <http://www.cloudera.com> > > Twitter: @josh_wills <http://twitter.com/josh_wills> > > > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
