Re: ElasticsearchSystemProducer Crashes Samza Job

The code that you showed below is part of the BulkProcessor.Listener interface so if that listener were pluggable, you could override the default behavior (which is to only ignore version conflicts). On Tue, Feb 16, 2016 at 12:27 PM, jeremiah adams wrote: > The root of

Re: ElasticsearchSystemProducer Crashes Samza Job

Hi Jeremiah, There's currently no way to do that. I think the best way to modify the existing ElasticsearchSystemProducer would be to add a config option for a callback to let you customize this behavior. Basically, a pluggable listener (

ElasticsearchSystemProducer Crashes Samza Job

We have a samza job configured to run in a yarn cluster. This job consumes multiple kafka topics and routes the messages to elasticsearch for indexing. When enough batch-updates to elasticsearch fail using the ElasticsearchSystemProducer, the entire samza job dies. Due to checkpointing + yarn, the