Zhang Jianguo created SPARK-34594: ------------------------------------- Summary: OrcColumnarBatchReader uncaught exception Key: SPARK-34594 URL: https://issues.apache.org/jira/browse/SPARK-34594 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 2.4.5 Reporter: Zhang Jianguo
RecordReaderInterator#hasNext doesn't handle exception thrown from OrcColumnarBatchReader#nextKeyValue ```scala override def hasNext: Boolean = { if (!finished && !havePair) { finished = !rowReader.nextKeyValue // Exception thrown from here if (finished) { // Close and release the reader here; close() will also be called when the task // completes, but for tasks that read from many files, it helps to release the // resources early. close() } havePair = !finished } !finished } ``` https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37 -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org