I guess then the question is, how do I set the (system?) property and what
property exactly is this?
"export HADOOP_HEAPSIZE=4096" - Works from the environment.
And none of the following seem to work from the command line.
-Xmx4G
-Dmapred.child.java.opts=-Xmx4G
-DHADOOP_HEAPSIZE=4096
-DHADOOP_OPTS=-Xmx4G
I tried this from Java code as well and no luck.
System.setProperty("HADOOP_HEAPSIZE", "4096");
What am I missing?
Thanks,
Andy
On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <shahab.yu...@gmail.com>wrote:
> You can pass hadoop conf properties through the -D option. Have you seen
> this?
>
> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>
> This is not for system properties. The assumption is that you want to
> specify hadoop conf property through the command line.
>
> Regards,
> Shahab
>
>
> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <andy.sr...@gmail.com> wrote:
>
>> Thanks for the tip. May be it's a different setting I need. Let me
>> explain the problem.
>>
>> I get an heap error on the command line - "Exception in thread "main"
>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>> command line (hadoop jar ...)? Next choice would be to do it
>> programmatically in Java using hadoop conf. I can't really change the XML
>> files across the cluster.
>>
>> Regards,
>> Andy
>>
>>
>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <azury...@gmail.com> wrote:
>>
>>> Do you want add "-Xmx4g" to your MR tasks? if so, just add it as
>>> "mapred.child.java.opts" in the mapred-site.xml
>>>
>>>
>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <andy.sr...@gmail.com>wrote:
>>>
>>>> Quick question. How would I pass the following JVM option to the Hadoop
>>>> command line?
>>>>
>>>> "-Xmx4G"
>>>>
>>>> hadoop jar <jar file> <class_name> <args>
>>>>
>>>> Thanks,
>>>> Andy
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Andy
>>
>>
>
--
Thanks,
Andy
