Re: How to gracefully handle Kafka OffsetOutOfRangeException

2017-03-10 Thread Justin Miller
) wrapped in a >> Try construct, but since the exception happens in the executor, the Try >> construct didn't take effect. Do you have any ideas of how to handle this? >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-li

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2017-03-10 Thread Michael Armbrust
If you have a reproduction you should open a JIRA. It would be great if there is a fix. I'm just saying I know a similar issue does not exist in structured streaming. On Fri, Mar 10, 2017 at 7:46 AM, Justin Miller < justin.mil...@protectwise.com> wrote: > Hi Michael, > > I'm experiencing a

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2017-03-10 Thread Justin Miller
andle this? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-gracefully-handle-Kafka-OffsetOutOfRangeException-tp26534.html > > <http://apache-spark-user-list.1001560.n3.nabble.com/H

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2017-03-10 Thread Michael Armbrust
how to handle this? > > > > -- > View this message in context: http://apache-spark-user-list. > 1001560.n3.nabble.com/How-to-gracefully-handle-Kafka- > OffsetOutOfRangeException-tp26534.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2017-03-10 Thread Ramkumar Venkataraman
@n3.nabble.com> wrote: > Did you find out how ? > > -- > If you reply to this email, your message will be added to the discussion > below: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to- > gracefully-handle-Kafka-OffsetOutOfRangeE

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2016-03-21 Thread Ramkumar Venkataraman
to handle this and not have my spark job crash? I > have > >> > no > >> > option of increasing the kafka retention period. > >> > > >> > I tried to have the DStream returned by createDirectStream() wrapped > in > >> > a > >

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2016-03-21 Thread Cody Koeninger
s and not have my spark job crash? I have >> > no >> > option of increasing the kafka retention period. >> > >> > I tried to have the DStream returned by createDirectStream() wrapped in >> > a >> > Try construct, but since the exception happens i

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2016-03-21 Thread Ramkumar Venkataraman
am() wrapped in a > > Try construct, but since the exception happens in the executor, the Try > > construct didn't take effect. Do you have any ideas of how to handle > this? > > > > > > > > -- > > View this message

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2016-03-19 Thread Sebastian Piu
t; I tried to have the DStream returned by createDirectStream() wrapped in a > Try construct, but since the exception happens in the executor, the Try > construct didn't take effect. Do you have any ideas of how to handle this? > > > > -- > View this message in context: &g

How to gracefully handle Kafka OffsetOutOfRangeException

2016-03-19 Thread Ramkumar Venkataraman
gracefully-handle-Kafka-OffsetOutOfRangeException-tp26534.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-ma

Re: How to gracefully handle Kafka OffsetOutOfRangeException

2016-03-19 Thread Cody Koeninger
ect. Do you have any ideas of how to handle this? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-gracefully-handle-Kafka-OffsetOutOfRangeException-t