Mohammad,
We store our offsets in Cassandra, and use that for tracking. This solved a few issues for us, as it provides a good persistence mechanism even when you're reading from multiple clusters. Bryan Jeffrey Get Outlook for Android On Tue, Feb 14, 2017 at 7:03 PM -0500, "Mohammad Kargar" <mkar...@phemi.com> wrote: As explained here, direct approach of integration between spark streaming and kafka does not update offsets in Zookeeper, hence Zookeeper-based Kafka monitoring tools will not show progress (details). We followed the recommended workaround to update the zookeeper with the latest offset after each batch, but no success. Wondering if there's any end-to-end example we can use? Thanks, Mohammad
