[ https://issues.apache.org/jira/browse/SPARK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-12693. ------------------------------- Resolution: Not A Problem > OffsetOutOfRangeException caused by retention > --------------------------------------------- > > Key: SPARK-12693 > URL: https://issues.apache.org/jira/browse/SPARK-12693 > Project: Spark > Issue Type: Bug > Components: Streaming > Affects Versions: 1.6.0 > Environment: Ubuntu 64bit, Intel i7 > Reporter: Rado Buransky > Priority: Minor > Labels: kafka > Attachments: kafka-log.txt, log.txt > > > I am running Kafka server locally with extremely low retention of 3 seconds > and with 1 second segmentation. I create direct Kafka stream with > auto.offset.reset = smallest. > In case of bad luck (happens actually quite often in my case) the smallest > offset retrieved druing stream initialization doesn't already exists when > streaming actually starts. > Complete source code of the Spark Streaming application is here: > https://github.com/pygmalios/spark-checkpoint-experience/blob/cb27ab83b7a29e619386b56e68a755d7bd73fc46/src/main/scala/com/pygmalios/sparkCheckpointExperience/spark/SparkApp.scala > The application ends in an endless loop trying to get that non-existing > offset and has to be killed. Check attached logs from Spark and also from > Kafka server. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org