Re: "Got wrong record after seeking to offset" issue

Yeah I saw that after I sent that e-mail out. Iactually remembered another ticket that I had commented on that turned out to be unrelated to the issue I was seeing at the time. It may be related to the current issue: https://issues.apache.org/jira/browse/SPARK-17147

Re: "Got wrong record after seeking to offset" issue

https://kafka.apache.org/documentation/#compaction On Thu, Jan 18, 2018 at 1:17 AM, Justin Miller wrote: > By compacted do you mean compression? If so then we did recently turn on lz4 > compression. If there’s another meaning if there’s a command I can run to >

Re: "Got wrong record after seeking to offset" issue

By compacted do you mean compression? If so then we did recently turn on lz4 compression. If there’s another meaning if there’s a command I can run to check compaction I’m happy to give that a shot too. I’ll try consuming from the failed offset if/when the problem manifests itself again. Thanks!

Re: "Got wrong record after seeking to offset" issue

That means the consumer on the executor tried to seek to the specified offset, but the message that was returned did not have a matching offset. If the executor can't get the messages the driver told it to get, something's generally wrong. What happens when you try to consume the particular

"Got wrong record after seeking to offset" issue

Greetings all, I’ve recently started hitting on the following error in Spark Streaming in Kafka. Adjusting maxRetries and spark.streaming.kafka.consumer.poll.ms even to five minutes doesn’t seem to be helping. The problem only manifested in the last few days, restarting with a new consumer