Matthias,
Good to know! Remember the caveat that the Generic Options (but don't
use that Class directly, its API is not marked for public consumption,
use Tool and ToolRunner instead) need to be the preset at the
beginning, followed by any app-specific arguments. Otherwise, they may
not work. The
Thanks Harsh!
That did the trick after some other search. This link was helpful:
http://hadoop.apache.org/common/docs/r1.0.3/api/org/apache/hadoop/util/GenericOptionsParser.html
The right way to do it was the following:
hadoop jar some.jar some.class.to.execute
-Dmapred.map.child.java.opts="-Xmx2
That is a per job property and you can raise it when submitting a job itself.
You can pass it via -D args (See
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/util/Tool.html)
or via a conf.set(name, value) from inside the code. Mostly, any prop
that is not of the format mapred(u
I want to change the mapreduce.(map|reduce).java.opts for a bigger heap
space of my map and reduce parts.
2012/6/13 Harsh J
> Matthias,
>
> It depends on what config you're trying to change. There are several
> per-job configs that do not require changes on JT/TT mapred-site.xml
> and can be pas
Matthias,
It depends on what config you're trying to change. There are several
per-job configs that do not require changes on JT/TT mapred-site.xml
and can be passed via -D parameters to a job's configuration directly
from CLI (if you use the Tool and ToolRunner methods of writing a
driver), or ma
Hi,
I've got a question regarding hadoop configuration. Is it possible to pass
configuration parameters on job start up?
Something like that:
hadoop -HADOOP_HEAPSIZE=4G jar some.jar some.class.to.execute param1 param2
Or do I have to restart the hadoop cluster every time I want to change
somethi
