[ https://issues.apache.org/jira/browse/HIVE-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486597#comment-13486597 ]
Chris Drome commented on HIVE-3638: ----------------------------------- @Shreepadma: I had traced the code a while back and found that Hive was creating an empty file for these types of queries. Hadoop20 didn't care that the file was empty and would create a split, which would get a mapper. With Hadoop23 I noticed that there is a condition which checks to see whether the file is empty or not. If it is empty it doesn't create a split and hence doesn't get a mapper. In this way Hive could trick Hadoop20 into running an MR job, but this tactic doesn't work on Hadoop23. I don't remember the classes off-hand. Here is the diff of the generated vs expected output. If I remember correctly, when no splits are generated it returns NULL. Line 84: select max(ds) from TEST1; (no partitions exist) Line 211: alter table TEST1 add partition (ds='1'); select max(ds) from TEST1; Line 337: select count(distinct ds) from TEST1; Line 1080: alter table TEST2 add partition (ds='1', hr='1'); alter table TEST2 add partition (ds='1', hr='2'); alter table TEST2 add partition (ds='1', hr='3'); select ds, count(distinct hr) from TEST2 group by ds; Line 1453: alter table TEST1 add partition (ds='2'); select max(ds) from TEST1; [junit] diff -a /export/crawlspace/cdrome/workspace/hive/build/ql/test/logs/clientpositive/metadataonly1.q.out /export/crawlspace/cdrome/workspace/hive/ql/src/test/results/clientpositive/metadataonly1.q.out [junit] 84c84 [junit] < NULL [junit] --- [junit] > [junit] 211c211 [junit] < NULL [junit] --- [junit] > 1 [junit] 337c337 [junit] < 0 [junit] --- [junit] > 1 [junit] 1080a1081 [junit] > 1 3 [junit] 1453c1454 [junit] < NULL [junit] --- [junit] > 2 > metadataonly1.q test fails with Hadoop23 > ---------------------------------------- > > Key: HIVE-3638 > URL: https://issues.apache.org/jira/browse/HIVE-3638 > Project: Hive > Issue Type: Bug > Components: Tests > Affects Versions: 0.10.0, 0.9.1 > Reporter: Chris Drome > > Hive creates an empty file as a hack to get Hadoop to run a mapper. > This no longer works with Hadoop23, causing this test to fail. Note that this > tests empty partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira