[
https://issues.apache.org/jira/browse/HIVE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290076#comment-16290076
]
Jason Dere commented on HIVE-18271:
-----------------------------------
Dug into some of the details with [~ashutoshc] and [~bslim]. The
FileSinkOperator should not even be trying to call Utilities.mvToFinalPath(),
because it is a non-native table - there is actually logic in place for that.
The problem is that FileSinkOperator.isNativeTable is dependent on being set
during initializeOp() .. and it appears that initializeOp() is never being
called for the operators in the TezTask (client side)! Making a patch to fix
the immediate problem, by making isNativeTable into a method, that does not
depend on initializeOp().
We may want to look into whether the operators within TezTask should have
initialize() called on them as a future item.
> Druid Insert into fails with exception when committing files
> ------------------------------------------------------------
>
> Key: HIVE-18271
> URL: https://issues.apache.org/jira/browse/HIVE-18271
> Project: Hive
> Issue Type: Bug
> Reporter: Nishant Bangarwa
> Assignee: Nishant Bangarwa
> Fix For: 3.0.0
>
> Attachments: HIVE-18271.patch
>
>
> Exception -
> {code}
> 000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty to:
> hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty.moved)'
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move:
> hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty
> to:
> hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty.moved
> at org.apache.hadoop.hive.ql.exec.Utilities.rename(Utilities.java:1129)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.mvFileToFinalPath(Utilities.java:1460)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.jobCloseOp(FileSinkOperator.java:1135)
> at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:765)
> at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:770)
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.close(TezTask.java:588)
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:286)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1211)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1204)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:350)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
