* The fact that I seem to get all of my data is currently leading me to discard and ignore this error
Please ignore this statement, I typed this email as I was testing a theory, I meant to delete this line. This is still a very real issue for me. I was looking to try a work around tomorrow, I saw that the Kafka 11 consumer supported transactions for exactly once processing, I was going to read about that and see if I could somehow fail a read that I couldn’t deserialize and try again, and if that might make a difference (can I just retry this ?). I’m not sure how that’ll go. If you’ve got an idea for a work around, I’d be all ears too. From: Philip Doctor <philip.doc...@physiq.com> Date: Tuesday, February 27, 2018 at 10:02 PM To: "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>, Fabian Hueske <fhue...@gmail.com> Cc: "user@flink.apache.org" <user@flink.apache.org> Subject: Re: Flink Kafka reads too many bytes .... Very rarely Honestly this has been a very frustrating issue to dig in to. The fact that I seem to get all of my data is currently leading me to discard and ignore this error, it’s rare, flink still seems to work, but something is very hard to debug here and despite some confusing observations, most of my evidence suggests that this originates in the flink kafka consumer.
