Changing the mapred.child.java.opts value does not change the heap size from a
default one.
-------------------------------------------------------------------------------------------
Key: WHIRR-146
URL: https://issues.apache.org/jira/browse/WHIRR-146
Project: Whirr
Issue Type: Bug
Environment: Amazon EC2, Amazon Linux images.
Reporter: Tibor Kiss
Assignee: Tibor Kiss
Even if I change the value for mapred.child.java.opts the task is started with
-Xmx200m.
Since the mapred.child.java.opts and mapred.child.ulimit has been deprecated,
we need to set the mapred.map.child.java.opts, mapred.reduce.child.java.opts
respectively the mapred.map.child.ulimit and mapred.reduce.child.ulimit in
order to have any effect.
Unfortunately the /scripts/cdh/install and /scripts/apache/install which
generates the /etc/hadoop/conf.dist/hadoop-site.xml is not synchronized with
this deprecation as a result we are not able to use mappers and reducers which
does not fit in 200M heap size.
How to reproduce:
1. Start a cluster on large instances where we are using 64bit jvm and run a
simple distcp, you will experience Child jvm crash.
2. Or run a job with mappers or reducers which does not fit in 200M heap, it
will experience OutOfMemoryError in child processes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
