[
https://issues.apache.org/jira/browse/HIVE-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147146#comment-13147146
]
jirapos...@reviews.apache.org commented on HIVE-1996:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1610/
-----------------------------------------------------------
(Updated 2011-11-09 16:44:55.708392)
Review request for hive, Carl Steinbach and John Sichi.
Changes
-------
Load rename made it configurable.
Summary
-------
"LOAD DATA INPATH" fails when the table already contains a file of the same
name. If any name confilcts occurs it will rename the file, After file name got
changed it is trying to load with the old name because of this load is failed.
Now we have changed the code like, load with the changed filename for that
introduced a map it will maintain the old name and changed filename as key
value pair and while loading need to use this map.
This addresses bug HIVE-1996.
https://issues.apache.org/jira/browse/HIVE-1996
Diffs (updated)
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1198626
trunk/conf/hive-default.xml 1198626
trunk/data/conf/hive-site.xml 1198626
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1198626
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1198626
trunk/ql/src/test/queries/clientpositive/input47.q PRE-CREATION
trunk/ql/src/test/results/clientpositive/input47.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1610/diff
Testing
-------
Added a test case for this scenario.
Thanks,
chinna
> "LOAD DATA INPATH" fails when the table already contains a file of the same
> name
> --------------------------------------------------------------------------------
>
> Key: HIVE-1996
> URL: https://issues.apache.org/jira/browse/HIVE-1996
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Kirk True
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1996.1.Patch, HIVE-1996.2.Patch, HIVE-1996.Patch
>
>
> Steps:
> 1. From the command line copy the kv2.txt data file into the current user's
> HDFS directory:
> {{$ hadoop fs -copyFromLocal /path/to/hive/sources/data/files/kv2.txt
> kv2.txt}}
> 2. In Hive, create the table:
> {{create table tst_src1 (key_ int, value_ string);}}
> 3. Load the data into the table from HDFS:
> {{load data inpath './kv2.txt' into table tst_src1;}}
> 4. Repeat step 1
> 5. Repeat step 3
> Expected:
> To have kv2.txt renamed in HDFS and then copied to the destination as per
> HIVE-307.
> Actual:
> File is renamed, but {{Hive.copyFiles}} doesn't "see" the change in {{srcs}}
> as it continues to use the same array elements (with the un-renamed, old file
> names). It crashes with this error:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:1725)
> at org.apache.hadoop.hive.ql.metadata.Table.copyFiles(Table.java:541)
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1173)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:197)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1060)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:897)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:745)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
