[
https://issues.apache.org/jira/browse/HIVE-7530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095089#comment-14095089
]
Brock Noland commented on HIVE-7530:
------------------------------------
We've been hitting this one by one which is quite slow thus I think we should
knock these out. I also think we should evaluate where we are casting to Tez
specific classes (as evidenced by HIVE-7591).
{noformat}
./ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:
TezWork work = (TezWork)task.getWork();
./ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:
((TezWork)work).add(cplan);
./ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:
TezWork work = (TezWork) currTask.getWork();
./ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java:
work = (MapWork) ((TezWork) mrTask.getWork()).getAllWork().get(0);
{noformat}
> Go thru the common code to find references to HIVE_EXECUCTION_ENGINE to make
> sure conditions works with Spark
> -------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-7530
> URL: https://issues.apache.org/jira/browse/HIVE-7530
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Xuefu Zhang
>
> In common code, such as Utilities.java, I found a lot of references to this
> conf variable and special handling to a specific engine such as following:
> {code}
> if (!HiveConf.getVar(job,
> ConfVars.HIVE_EXECUTION_ENGINE).equals("tez")
> && isEmptyPath(job, path, ctx)) {
> path = createDummyFileForEmptyPartition(path, job, work,
> hiveScratchDir, alias, sequenceNumber++);
> }
> {code}
> We need to make sure the condition still holds after a new execution engine
> such as "spark" is introduced.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
