Github user shuyang-truex commented on the issue:
https://github.com/apache/spark/pull/21929
Thanks @brucezhao11. Unfortunately this patch did cause a side effect for
us, where the receiver process would die randomly after running for a while
(ranges from a few minutes to many hours, not during checkpoint recovery).
Symptom being the streaming job still kept running, but with all empty batches
and no API calls to Kinesis. We had to rollback and explore options like
structured streaming in the end.
It does seem surprising that this important component doesn't work
properly, but looks like the non-structured streaming is pretty much
deprecated, the maintainers probably won't be very motivated to get it fixed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
