[
https://issues.apache.org/jira/browse/HIVE-25025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
WangHualei updated HIVE-25025:
------------------------------
Attachment: HIVE-25025.patch
Labels: patch pull-request-available (was: pull-request-available)
Status: Patch Available (was: In Progress)
distcp method different from FileUtil.copy, because distcp method ignore parent
directory, then may overwrite stats files,cause stats info lost.?
modify dst value, add parent directory to dst path, make distcp method run
result as same as?FileUtil.copy.
> Distcp In MoveTask may cause stats info lost
> --------------------------------------------
>
> Key: HIVE-25025
> URL: https://issues.apache.org/jira/browse/HIVE-25025
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: WangHualei
> Assignee: WangHualei
> Priority: Major
> Labels: pull-request-available, patch
> Attachments: HIVE-25025.patch
>
> Original Estimate: 72h
> Time Spent: 10m
> Remaining Estimate: 71h 50m
>
> after set _Run_ _as_ _end_ _user_ _instead_ _of_ _Hive_ _user_ , when
> execute insert overwrite , In MoveTask ,if source byte >
> HIVE_EXEC_COPYFILE_MAXSIZE and source file count>
> HIVE_EXEC_COPYFILE_MAXNUMFILES , HIve will use distcp method, it may cause
> tmp stats file lost.
> example:
> set hive.exec.copyfile.maxsize=0;
> set hive.exec.copyfile.maxnumfiles=0;
> insert overwrite table abc_new select * from abc;
> select count(1) from abc_new ;
> select * from abc_new ;
> then the count(1) result will be 0, but select * will display real data,
> because stats info lost.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
