Previously, I was using storm topology to build my pipeline to suffice my use case which was - Reading from Kafka topic through Kafka spout, process these tuples in Bolt, then emit them to ElasticSearch bolt to index my tuples. In storm topology, I was able to acknowledge each tuple i was indexing and these acknowledgments were sent from EsBolt to spout and finally to Zookeeper.
Later I realised that Storm is not the appropriate component for my use case and I switched to Spark. I am doing the same thing with Spark - Reading from Kafka topic, reading messages in Spark stream, pass this stream to a component which converts this stream to RDD and finally indexes these RDDs. But now, there is no mechanism for acknowledgements. But I still want to send acknowledgements for each messages being indexed to the previous component. My code looks like this - JavaEsSpark.saveJsonToEs(jrd, "index_{date}/type"); How can I get acknowledgements in Spark-ElasticSearch framework? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f73a57fd-1f5a-41f3-b6ec-077961668687%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.