[
https://issues.apache.org/jira/browse/HIVE-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joydeep Sen Sarma updated HIVE-543:
-----------------------------------
Attachment: hive-543.patch.1
a few fixes for better local mode execution:
- provide alternate log4j file configuration for capturing local mode execution
log (and default to hive log4j if none provided). this cleans up the goop on
the cli but allows captuing execution time logs in a separate location if
desired
- bypass distributed cache for local mode submissions. saves on hdfs time
- some cleanup on the set/get MapRedWork code path. it seems to have been
messed up after the parallel execution changes
- getMRScratchDir - now returns a local scratch dir when executing in local
mode. so we don't hit hdfs unnecessarily in local mode.
- fix to fileutils.makequalified because of the above. there was a subtle bug
in this that was causing file paths to get messed up when using local paths for
interemediate data
- bypassed query plan serialization/deserialization except for test mode. from
past experience - xml serialization/deserialization is pretty expensive and
makes no sense to subject every query to it.
> provide option to run hive in local mode
> ----------------------------------------
>
> Key: HIVE-543
> URL: https://issues.apache.org/jira/browse/HIVE-543
> Project: Hadoop Hive
> Issue Type: Improvement
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Attachments: hive-543.patch.1
>
>
> this is a little bit more than just mapred.job.tracker=local
> when run in this mode - multiple jobs are an issue since writing to same tmp
> directories is an issue. the following options:
> hadoop.tmp.dir
> mapred.local.dir
> need to be randomized (perhaps based on queryid).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
