Namit Jain created HIVE-3451: -------------------------------- Summary: map-reduce jobs does not work for a partition containing sub-directories Key: HIVE-3451 URL: https://issues.apache.org/jira/browse/HIVE-3451 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain
Consider the following test: -- The test verifies that sub-directories are supported for versions of hadoop -- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23. -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23) CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING); CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING) LOCATION 'pfile:${system:test.tmp.dir}/fact_tz'; INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1') SELECT key+11 FROM src WHERE key=484; ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE'); ALTER TABLE fact_daily ADD PARTITION (ds='1') LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1'; set mapred.input.dir.recursive=true; SELECT * FROM fact_daily WHERE ds='1'; SELECT count(1) FROM fact_daily WHERE ds='1'; Say, the above file was named: recursive_dir.q and we ran the test for hadoop 23: by executing: ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true -Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha -Dtestcase=TestCliDriver -Dqfile=recursive_dir.q The select * from the table works fine, but the last command does not work since it requires a map-reduce job. This will prevent other features which are creating sub-directories to add any tests which requires a map-reduce job. The work-around is to issue queries which do not require map-reduce jobs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira