Re: Samza job having failed containers in YARN

2016-11-22 Thread Suraj choudhary
Hi Jagadish, Thanks for your response. *Question 1*: Do you write your output to kafka at all? (For example, is it possible for a message you write to an output topic to be more than 1M?) Yes, we write changelogs for RocksDB stores, metrics for monitoring purpose and offset checkpoints to diffe

Re: Samza job having failed containers in YARN

2016-11-18 Thread Jagadish Venkatraman
Hi Suraj, A couple of questions: 1. Do you write your output to kafka at all? (For example, is it possible for a message you write to an output topic to be more than 1M?) 2. Is it possible that an entry written to the changelog is more than 1M? (Possibly, because you had a > 1M write to the local

Samza job having failed containers in YARN

2016-11-18 Thread Suraj choudhary
Hey there, We are having a Samza job running in YARN which reads from Kafka and writes to Cassandra. We are also enabling the job to write the metrics to a Kafka topic, the changelogs for the RocksDB stores and checkpointing of offsets are also written to Kafka. We are having failed containers i