Re: Structured streaming: Tried to fetch $offset but the returned record offset was ${record.offset}"

On 2018/04/17 22:34:25, Cody Koeninger wrote: > Is this possibly related to the recent post on > https://issues.apache.org/jira/browse/SPARK-18057 ? > > On Mon, Apr 16, 2018 at 11:57 AM, ARAVIND SETHURATHNAM < > asethurath...@homeaway.com.invalid> wrote: > > > Hi, > > > > We have several

Re: Structured streaming: Tried to fetch $offset but the returned record offset was ${record.offset}"

Is this possibly related to the recent post on https://issues.apache.org/jira/browse/SPARK-18057 ? On Mon, Apr 16, 2018 at 11:57 AM, ARAVIND SETHURATHNAM < asethurath...@homeaway.com.invalid> wrote: > Hi, > > We have several structured streaming jobs (spark version 2.2.0) consuming > from kafka

Structured streaming: Tried to fetch $offset but the returned record offset was ${record.offset}"

Hi, We have several structured streaming jobs (spark version 2.2.0) consuming from kafka and writing to s3. They were running fine for a month, since yesterday few jobs started failing and I see the below exception in the failed jobs log, ```Tried to fetch 473151075 but the returned record