[
https://issues.apache.org/jira/browse/HIVE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258885#comment-14258885
]
Rui Li commented on HIVE-9153:
------------------------------
Hi [~brocknoland] and [~xuefuz],
Sorry maybe I was being confusing. The patch here is to reduce the call to
{{Utilities.getBaseWork()}}, which is quite similar to HIVE-9127. Changes to
{{Utilities.getBaseWork()}} is just to remove redundant code:
{code}
Path localPath;
if (conf.getBoolean("mapreduce.task.uberized", false) &&
name.equals(REDUCE_PLAN_NAME)) {
localPath = new Path(name);
} else if (ShimLoader.getHadoopShims().isLocalMode(conf)) {
localPath = path;
} else {
LOG.info("***************non-local mode***************");
localPath = new Path(name);
}
localPath = path;
LOG.info("local path = " + localPath);
{code}
Seems those if-else is unnecessary because localPath = path anyway, which makes
localPath redundant too. But I can revert this change if you feel uncertain
about it.
BTW, the path should be a trunk patch, I'll upload a trunk version to test
again.
> Evaluate CombineHiveInputFormat versus HiveInputFormat [Spark Branch]
> ---------------------------------------------------------------------
>
> Key: HIVE-9153
> URL: https://issues.apache.org/jira/browse/HIVE-9153
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Affects Versions: spark-branch
> Reporter: Brock Noland
> Assignee: Rui Li
> Attachments: HIVE-9153.1-spark.patch, HIVE-9153.1-spark.patch,
> screenshot.PNG
>
>
> The default InputFormat is {{CombineHiveInputFormat}} and thus HOS uses this.
> However, Tez uses {{HiveInputFormat}}. Since tasks are relatively cheap in
> Spark, it might make sense for us to use {{HiveInputFormat}} as well. We
> should evaluate this on a query which has many input splits such as {{select
> count(\*) from store_sales where something is not null}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
