[
https://issues.apache.org/jira/browse/HIVE-24530?focusedWorklogId=524322&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-524322
]
ASF GitHub Bot logged work on HIVE-24530:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Dec/20 08:56
Start Date: 15/Dec/20 08:56
Worklog Time Spent: 10m
Work Description: szlta commented on a change in pull request #1775:
URL: https://github.com/apache/hive/pull/1775#discussion_r543159074
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
##########
@@ -1204,8 +1205,10 @@ private void closeWriters(boolean abort) throws
HiveException {
private void closeRecordwriters(boolean abort) {
for (RecordWriter writer : rowOutWriters) {
Review comment:
So now there is a null check for the RecordWriter instances in this
list, but shouldn't we also check for the list itself (rowOutWriters) being
null here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 524322)
Time Spent: 20m (was: 10m)
> Potential NPE in FileSinkOperator.closeRecordwriters method
> -----------------------------------------------------------
>
> Key: HIVE-24530
> URL: https://issues.apache.org/jira/browse/HIVE-24530
> Project: Hive
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: Marta Kuczora
> Assignee: Marta Kuczora
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> During testing a NPE occurred in the FileSinkOperator.closeRecordwriters
> method.
> After investigating, turned out there was an underlaying IOException during
> executing the FileSinkOperator.process method. It got caught by the following
> code part:
> {noformat}
> } catch (IOException e) {
> closeWriters(true);
> throw new HiveException(e);
> } catch (SerDeException e) {
> closeWriters(true);
> throw new HiveException(e);
> }
> {noformat}
> First the closeWriters method was called:
> {noformat}
> private void closeWriters(boolean abort) throws HiveException {
> fpaths.closeWriters(true);
> closeRecordwriters(true);
> }
> private void closeRecordwriters(boolean abort) {
> for (RecordWriter writer : rowOutWriters) {
> try {
> LOG.info("Closing {} on exception", writer);
> writer.close(abort);
> } catch (IOException e) {
> LOG.error("Error closing rowOutWriter" + writer, e);
> }
> }
> {noformat}
> If the writers had got closed successfully, a HiveException would have been
> thrown with the original IOException.
> But when the IOException occurred the writers in the rowOutWriters were not
> yet initialised, so a NPE occurred. This was very misleading as the NPE was
> not the real issue, but the original IOException was hidden.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
