Hi Ted,
As far as i can recollect, I onl configured these parameters
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx2048m</value>
<description>this number is the number of megabytes of memory that
each mapper and each reducers will have available to use. If jobs start
running out of heap space, this may need to be increased.</description>
</property>
<property>
<name>mapred.child.ulimit</name>
<value>3145728</value>
<description>this number is the number of kilobytes of memory that
each mapper and each reducer will have available to use. If jobs start
running out of heap space, this may need to be increased.</description>
</property>
On Mon, Mar 25, 2013 at 6:57 AM, Ted <r6squee...@gmail.com> wrote:
> did you set the min heap size == your max head size? if you didn't,
> free memory only shows you the difference between used and commit, not
> used and max.
>
> On 3/24/13, nagarjuna kanamarlapudi <nagarjuna.kanamarlap...@gmail.com>
> wrote:
> > Hi,
> >
> > I configured my child jvm heap to 2 GB. So, I thought I could really
> read
> > 1.5GB of data and store it in memory (mapper/reducer).
> >
> > I wanted to confirm the same and wrote the following piece of code in the
> > configure method of mapper.
> >
> > @Override
> >
> > public void configure(JobConf job) {
> >
> > System.out.println("FREE MEMORY -- "
> >
> > + Runtime.getRuntime().freeMemory());
> >
> > System.out.println("MAX MEMORY ---" + Runtime.getRuntime().maxMemory());
> >
> > }
> >
> >
> > Surprisingly the output was
> >
> >
> > FREE MEMORY -- 341854864 = 320 MB
> > MAX MEMORY ---1908932608 = 1.9 GB
> >
> >
> > I am just wondering what processes are taking up that extra 1.6GB of
> > heap which I configured for the child jvm heap.
> >
> >
> > Appreciate in helping me understand the scenario.
> >
> >
> >
> > Regards
> >
> > Nagarjuna K
> >
>
>
> --
> Ted.
>
