[jira] [Updated] (SPARK-34594) OrcColumnarBatchReader uncaught exception

2021-03-02 Thread Zhang Jianguo (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-34594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Jianguo updated SPARK-34594:
--
Description: 
RecordReaderInterator#hasNext doesn't handle exception thrown from 
OrcColumnarBatchReader#nextKeyValue
{code:java}
// code placeholder
override def hasNext: Boolean = {
  if (!finished && !havePair) {
finished = !rowReader.nextKeyValue // Exception thrown from here
if (finished) {
  // Close and release the reader here; close() will also be called when 
the task
  // completes, but for tasks that read from many files, it helps to 
release the
  // resources early.
  close()
}
havePair = !finished
  }
  !finished
}

{code}
[https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37]

 

!report.PNG!

  was:
RecordReaderInterator#hasNext doesn't handle exception thrown from 
OrcColumnarBatchReader#nextKeyValue


{code:java}
// code placeholder
override def hasNext: Boolean = {
  if (!finished && !havePair) {
finished = !rowReader.nextKeyValue // Exception thrown from here
if (finished) {
  // Close and release the reader here; close() will also be called when 
the task
  // completes, but for tasks that read from many files, it helps to 
release the
  // resources early.
  close()
}
havePair = !finished
  }
  !finished
}

{code}

 
[https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37]


> OrcColumnarBatchReader uncaught exception
> -
>
> Key: SPARK-34594
> URL: https://issues.apache.org/jira/browse/SPARK-34594
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 2.4.5
>Reporter: Zhang Jianguo
>Priority: Major
>
> RecordReaderInterator#hasNext doesn't handle exception thrown from 
> OrcColumnarBatchReader#nextKeyValue
> {code:java}
> // code placeholder
> override def hasNext: Boolean = {
>   if (!finished && !havePair) {
> finished = !rowReader.nextKeyValue // Exception thrown from here
> if (finished) {
>   // Close and release the reader here; close() will also be called when 
> the task
>   // completes, but for tasks that read from many files, it helps to 
> release the
>   // resources early.
>   close()
> }
> havePair = !finished
>   }
>   !finished
> }
> {code}
> [https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37]
>  
> !report.PNG!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-34594) OrcColumnarBatchReader uncaught exception

2021-03-02 Thread Zhang Jianguo (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-34594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Jianguo updated SPARK-34594:
--
Description: 
RecordReaderInterator#hasNext doesn't handle exception thrown from 
OrcColumnarBatchReader#nextKeyValue


{code:java}
// code placeholder
override def hasNext: Boolean = {
  if (!finished && !havePair) {
finished = !rowReader.nextKeyValue // Exception thrown from here
if (finished) {
  // Close and release the reader here; close() will also be called when 
the task
  // completes, but for tasks that read from many files, it helps to 
release the
  // resources early.
  close()
}
havePair = !finished
  }
  !finished
}

{code}

 
[https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37]

  was:
RecordReaderInterator#hasNext doesn't handle exception thrown from 
OrcColumnarBatchReader#nextKeyValue
```scala
  override def hasNext: Boolean = {
if (!finished && !havePair) {
  finished = !rowReader.nextKeyValue // Exception thrown from here
  if (finished) {
// Close and release the reader here; close() will also be called when 
the task
// completes, but for tasks that read from many files, it helps to 
release the
// resources early.
close()
  }
  havePair = !finished
}
!finished
  }
```
https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37


> OrcColumnarBatchReader uncaught exception
> -
>
> Key: SPARK-34594
> URL: https://issues.apache.org/jira/browse/SPARK-34594
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 2.4.5
>Reporter: Zhang Jianguo
>Priority: Major
>
> RecordReaderInterator#hasNext doesn't handle exception thrown from 
> OrcColumnarBatchReader#nextKeyValue
> {code:java}
> // code placeholder
> override def hasNext: Boolean = {
>   if (!finished && !havePair) {
> finished = !rowReader.nextKeyValue // Exception thrown from here
> if (finished) {
>   // Close and release the reader here; close() will also be called when 
> the task
>   // completes, but for tasks that read from many files, it helps to 
> release the
>   // resources early.
>   close()
> }
> havePair = !finished
>   }
>   !finished
> }
> {code}
>  
> [https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-34594) OrcColumnarBatchReader uncaught exception

2021-03-02 Thread Zhang Jianguo (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-34594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Jianguo updated SPARK-34594:
--
Attachment: report_01.PNG

> OrcColumnarBatchReader uncaught exception
> -
>
> Key: SPARK-34594
> URL: https://issues.apache.org/jira/browse/SPARK-34594
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 2.4.5
>Reporter: Zhang Jianguo
>Priority: Major
> Attachments: report_01.PNG
>
>
> RecordReaderInterator#hasNext doesn't handle exception thrown from 
> OrcColumnarBatchReader#nextKeyValue
> {code:java}
> // code placeholder
> override def hasNext: Boolean = {
>   if (!finished && !havePair) {
> finished = !rowReader.nextKeyValue // Exception thrown from here
> if (finished) {
>   // Close and release the reader here; close() will also be called when 
> the task
>   // completes, but for tasks that read from many files, it helps to 
> release the
>   // resources early.
>   close()
> }
> havePair = !finished
>   }
>   !finished
> }
> {code}
> [https://github.com/apache/spark/blob/96f51378b598501cc5dc5791f794c8586a48b355/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/RecordReaderIterator.scala#L37]
>  
> !report.PNG!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org