[
https://issues.apache.org/jira/browse/HIVE-27688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764575#comment-17764575
]
liang yu edited comment on HIVE-27688 at 9/13/23 10:06 AM:
-----------------------------------------------------------
I traced the code of moveTask, and find that when we execute sql "insert
overwrite into table partition (XX) select ....", it will throw HiveException
whose exception message is overwritten by method getHiveException(e, msg). Here
is the chain of how it gets overwritten:
{code:java}
execute(handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line
273)> handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 392)>
loadPartition(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 472)
->replaceFiles(org/apache/hadoop/hive/ql/metadata/Hive.java: line 1817) ->
moveFile(org/apache/hadoop/hive/ql/metadata/Hive.java: line 4136) ->
needToCopy(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3681) ->
throw HiveException(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3856)
{code}
but this HiveException's message was overwritten by method
moveFile(org/apache/hadoop/hive/ql/metadata/Hive.java: line 4136), it catches
the exception and replaced the message with("Unable to move source
/path/to/source to destination /path/to/dest, which is a very regular error
message")
But when I execute sql "insert into table partition (XX) select ...", it will
throw HiveException which is not overwritten. Here is the chain of how it
throws the correct Exception:
{code:java}
execute(handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line
273)> handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 392)>
loadPartition(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 472) ->
copyFiles(org/apache/hadoop/hive/ql/metadata/Hive.java: line 1821) ->
copyFiles(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3937) ->
needToCopy(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3389) ->
throw HiveException(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3856)
{code}
this HiveException was thrown and caught by method loadPartition
(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 472) whose error message is
not overwritten.
was (Author: JIRAUSER299608):
I traced the code of moveTask, and find that when we execute sql "insert
overwrite into table partition (XX) select ....", it will throw HiveException
whose exception message is overwritten by method getHiveException(e, msg). Here
is the chain of how it gets overwritten:
execute(handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line
273)-> handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line
392)->
loadPartition(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 472)
->replaceFiles(org/apache/hadoop/hive/ql/metadata/Hive.java: line 1817) ->
moveFile(org/apache/hadoop/hive/ql/metadata/Hive.java: line 4136) ->
needToCopy(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3681) ->
throw HiveException(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3856)
but this HiveException's message was overwritten by method
moveFile(org/apache/hadoop/hive/ql/metadata/Hive.java: line 4136), it catches
the exception and replaced the message with("Unable to move source
/path/to/source to destination /path/to/dest, which is a very regular error
message")
But when I execute sql "insert into table partition (XX) select ...", it will
throw HiveException which is not overwritten. Here is the chain of how it
throws the correct Exception:
execute(handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line
273)-> handleStaticParts(org/apache/hadoop/hive/ql/exec/MoveTask.java:line
392)->
loadPartition(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 472) ->
copyFiles(org/apache/hadoop/hive/ql/metadata/Hive.java: line 1821) ->
copyFiles(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3937) ->
needToCopy(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3389) ->
throw HiveException(org/apache/hadoop/hive/ql/metadata/Hive.java: line 3856)
this HiveException was thrown and caught by method loadPartition
(org/apache/hadoop/hive/ql/exec/MoveTask.java:line 472) whose error message is
not overwritten.
> hive MoveTask cannot show the correct exception message
> -------------------------------------------------------
>
> Key: HIVE-27688
> URL: https://issues.apache.org/jira/browse/HIVE-27688
> Project: Hive
> Issue Type: Bug
> Reporter: liang yu
> Assignee: liang yu
> Priority: Major
> Attachments: image-2023-09-13-17-39-27-864.png,
> image-2023-09-13-17-40-02-981.png
>
>
> Setting hive.load.data.owner to hive.
> When I try to insert overwrite to an existing table partition, I get the
> exception that: Unable to move source /path/to/source to destination
> /path/to/dest, which is a very regular error message, gives me no helpful
> information.
> !image-2023-09-13-17-40-02-981.png!
>
> But when I try to insert into an existing table partition, I get the
> exception that: Unable to move source /path/to/source to destination
> /path/to/dest as the file is not owned by hive and load data is also not ran
> as hive. which gives me a very helpful error message that I should change the
> hive.load.data.owner to hive.
> !image-2023-09-13-17-39-27-864.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
