[
https://issues.apache.org/jira/browse/WHIRR-146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12964856#action_12964856
]
Tom White commented on WHIRR-146:
---------------------------------
+1 looks good
> In order to JUnit test it, probably we would need to write a job which runs
> in integration tests.
Adding jobs to the benchmark suites that will be introduced in WHIRR-92 is
probably the way to do this.
> I'm not sure if we are changing only the install scripts which are also
> changeable when you would like to personalize the setup
Changing the install scripts is not very user friendly at the moment. WHIRR-55
will make this easier.
> Changing the mapred.child.java.opts value does not change the heap size from
> a default one.
> -------------------------------------------------------------------------------------------
>
> Key: WHIRR-146
> URL: https://issues.apache.org/jira/browse/WHIRR-146
> Project: Whirr
> Issue Type: Bug
> Environment: Amazon EC2, Amazon Linux images.
> Reporter: Tibor Kiss
> Assignee: Tibor Kiss
> Attachments: whirr-146.patch
>
>
> Even if I change the value for mapred.child.java.opts the task is started
> with -Xmx200m.
> Since the mapred.child.java.opts and mapred.child.ulimit has been deprecated,
> we need to set the mapred.map.child.java.opts, mapred.reduce.child.java.opts
> respectively the mapred.map.child.ulimit and mapred.reduce.child.ulimit in
> order to have any effect.
> Unfortunately the /scripts/cdh/install and /scripts/apache/install which
> generates the /etc/hadoop/conf.dist/hadoop-site.xml is not synchronized with
> this deprecation as a result we are not able to use mappers and reducers
> which does not fit in 200M heap size.
> How to reproduce:
> 1. Start a cluster on large instances where we are using 64bit jvm and run a
> simple distcp, you will experience Child jvm crash.
> 2. Or run a job with mappers or reducers which does not fit in 200M heap, it
> will experience OutOfMemoryError in child processes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
