Hello, 1. My Kafka consumer name is randomly being generated by spark structured streaming. Can I override this? 2. When testing in development, when I stop my streaming job for Kafka consumer job for couple of days and try to start back again, the job keeps failing for missing offsets as the offsets get expired after 4 hours. I read that when restarting to consume messages SS always tries to get the earliest offset but not latest offset. How to handle this problem?
Regards Srini
