Hm. My understanding was that both are aimed at basically the same thing - Kafka on YARN. From Samza site: "Apache Samza is a distributed stream processing framework. It uses Apache Kafka <http://kafka.apache.org/> for messaging, and Apache Hadoop YARN <http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html> to provide fault tolerance, processor isolation, security, and resource management."
And KOYA: "KOYA is a YARN application that launches Kafka within YARN. It then manages the resource negotiation with Resource Manager, and ensures that Kafka operates in a YARN native way. For an external publisher or subscriber, KOYA would not look any different than Kafka since the same code is being run as a YARN application." It looks like they were never mentioned together.... until now :) Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Fri, Jan 16, 2015 at 3:34 PM, Chris Riccomini < criccom...@linkedin.com.invalid> wrote: > Hey Otis, > > I'm not terribly familiar with KOYA, but my understanding is that it's a > tool for deploying Kafka brokers to YARN, and administering them. I don't > think that it has any stream processing functionality built into it. As > such, it seems to me that KOYA and Samza could be used together: you could > use KOYA to deploy Kafka in YARN, and Samza to read/write messages from > the brokers that have been deployed. > > Samza provides containers that have consumers/producers in them, and allow > you to plug in processing logic as new messages arrive. Samza's goal is to > provide features that are useful when you're processing the messages, such > as fault tolerance (restarting consumers/producers when they fail), > checkpointing (saving offsets), state management (if you're counting > messages, you want to make sure your count is accurate even if you fail), > etc. > > Cheers, > Chris > > On 1/16/15 12:14 PM, "Otis Gospodnetic" <otis.gospodne...@gmail.com> > wrote: > > >Hi, > > > >I was wondering if anyone can compare and contrast KOYA and Samza? > > > >Thanks, > >Otis > >-- > >Monitoring * Alerting * Anomaly Detection * Centralized Log Management > >Solr & Elasticsearch Support * http://sematext.com/ > >
