[
https://issues.apache.org/jira/browse/HIVE-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-16483:
----------------------------
Attachment: HIVE-16483.1.patch
Re-attach patch to trigger test.
> HoS should populate split related configurations to HiveConf
> ------------------------------------------------------------
>
> Key: HIVE-16483
> URL: https://issues.apache.org/jira/browse/HIVE-16483
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Chao Sun
> Assignee: Chao Sun
> Attachments: HIVE-16483.1.patch
>
>
> There are several split related configurations, such as
> {{MAPREDMINSPLITSIZE}}, {{MAPREDMINSPLITSIZEPERNODE}},
> {{MAPREDMINSPLITSIZEPERRACK}}, etc., that should be populated to HiveConf.
> Currently we only do this for {{MAPREDMINSPLITSIZE}}.
> All the others, if not set, will be using the default value, which is 1.
> Without these, Spark sometimes will not merge small files for file formats
> such as text.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
