As explained here <https://spark.apache.org/docs/latest/streaming-kafka-0-8-integration.html>, direct approach of integration between spark streaming and kafka does not update offsets in Zookeeper, hence Zookeeper-based Kafka monitoring tools will not show progress (details). We followed the recommended workaround to update the zookeeper with the latest offset after each batch, but no success. Wondering if there's any end-to-end example we can use?
Thanks, Mohammad
