Thomas Friedrich created HIVE-8790:
--------------------------------------
Summary: UT: fix hook_context_cs test case
Key: HIVE-8790
URL: https://issues.apache.org/jira/browse/HIVE-8790
Project: Hive
Issue Type: Sub-task
Components: Tests
Affects Versions: spark-branch
Reporter: Thomas Friedrich
Priority: Minor
The test hook_context_cs fails while checking for expected value in
VerifyContentSummaryCacheHook:
TestSparkCliDriver.testCliDriver_hook_context_cs:120->runTest:146 Unexpected
exception junit.framework.AssertionFailedError: expected:<1> but was:<0>
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.failNotEquals(Assert.java:329)
at junit.framework.Assert.assertEquals(Assert.java:78)
at junit.framework.Assert.assertEquals(Assert.java:234)
at junit.framework.Assert.assertEquals(Assert.java:241)
at
org.apache.hadoop.hive.ql.hooks.VerifyContentSummaryCacheHook.run(VerifyContentSummaryCacheHook.java:34)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1512)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1216)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1033)
The problem is that the post-hook check in
org.apache.hadoop.hive.ql.hooks.VerifyContentSummaryCacheHook fails because
inputToCS.size is 0 instead of 1:
Assert.assertEquals(1, inputToCS.size());
I ran the same test with TestCliDriver where Hive will create MapRedTask
objects. For a MapRedTask, the hookContext.getInputPathToContentSummary() is
set in the setNumberOfReducers method, line 400:
inputSummary = Utilities.getInputSummary(driverContext.getCtx(),
work.getMapWork(), null);
(the path is added in Utilities.getInputSummary(Context, MapWork, PathFilter)
line: 2584).
Since we have a SparkTask, the array will be empty, hence the check for
length==1 fails.
It looks somewhat related to HIVE-8504 where the test case also configures a
hive.exec.post.hooks, but since it's a different post-hook class altogether, I
wanted to open a separate JIRA for it.
Maybe something needs to be added to the SparkTask.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
