; >> >> nodemanager
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> yarn.nodemanager.address
>>> >> >> 0.0.0.0:18050
>>> >> >&g
gt; >> >> yarn.nodemanager.remote-app-log-dir
>> >> >>
>> /opt/hadoop-2.0.4-alpha/temp/hadoop/yarn_nm_app-logs
>> >> >> directory on hdfs where the application logs are
>> moved
>> >> >> to
>> >> >>
>&
the directories used by Nodemanagers as log
>> >> >> directories
>> >> >>
>> >> >>
>> >> >>
>> >> >> yarn.nodemanager.aux-services
>> >> >> mapreduce.shuffle
>> &g
> >> >>
> >> >>
> >> >> yarn.nodemanager.vcores-pcores-ratio
> >> >> 3
> >> >>
> >> >>
> >> >>
> >> >> yarn.nodeman
>> >>
>> >>
>> >> 2013/6/7 Harsh J
>> >>>
>> >>> Not tuning configurations at all is wrong. YARN uses memory resource
>> >>> based scheduling and hence MR2 would be requesting 1 GB minimum by
>> >>> defau
>>> > tune
> >>> > their configurations at all. So I got above test results. After
> >>> > analyzing
> >>> > the test result, no doubt, I will configure them and do comparison
> >>> > again.
> >>> >
> &
; > And any detailed suggestions/comments/materials on Yarn performance
>>> > tunning?
>>> >
>>> > Thanks!
>>> >
>>> >
>>> > 2013/6/7 Marcos Luis Ortiz Valmaseda
>>> >>
>>> >> Why not to tune
terasort on Yarn and MRv1 for comprison.
>> >>>
>> >>> My env is three nodes cluster, and each node has similar hardware: 2
>> >>> cpu(4 core), 32 mem. Both Yarn and MRv1 cluster are set on the same
>> en
gt;>
> >>> MRv1: Hadoop-1.1.1
> >>> Yarn: Hadoop-2.0.4
> >>>
> >>> (A) Teragen: generate 10 GB data:
> >>> - MRv1: 193 sec
> >>> - Yarn: 69 sec
> >>> Yarn is 2.8 times better than MRv1
> >>>
> >&g
differences:
>>>
>>> MRv1: Hadoop-1.1.1
>>> Yarn: Hadoop-2.0.4
>>>
>>> (A) Teragen: generate 10 GB data:
>>> - MRv1: 193 sec
>>> - Yarn: 69 sec
>>> Yarn is 2.8 times better than MRv1
>>>
>>> (B) Terasort: so
Yarn: 1136 sec
> Yarn is 2.5 times worse than MRv1
>
> After a fast analysis, I think the direct cause might be that Yarn is much
> faster than MRv1 on Map phase, but much worse on Reduce phase.
>
> Here I have two questions:
> - Why my tests shows Yarn is worse than
;>
>>>> (A) Teragen: generate 10 GB data:
>>>> - MRv1: 193 sec
>>>> - Yarn: 69 sec
>>>> *Yarn is 2.8 times better than MRv1*
>>>>
>>>> (B) Terasort: sort 10 GB data:
>>>> - MRv1: 451 sec
>&
rt 10 GB data:
>>> - MRv1: 451 sec
>>> - Yarn: 1136 sec
>>> *Yarn is 2.5 times worse than MRv1*
>>>
>>> After a fast analysis, I think the direct cause might be that Yarn is
>>> much faster than MRv1 on Map phase, but much worse on Reduce phase.
&
ec
>> *Yarn is 2.5 times worse than MRv1*
>>
>> After a fast analysis, I think the direct cause might be that Yarn is
>> much faster than MRv1 on Map phase, but much worse on Reduce phase.
>>
>> Here I have two questions:
>> *- Why my tests shows Yarn is worse than MRv1 for terasort?
>> *
>> *- What's the stratage for tuning Yarn performance? Is any materials?*
>>
>> Thanks!
>>
>
>
>
> --
> Marcos Ortiz Valmaseda
> Product Manager at PDVSA
> http://about.me/marcosortiz
>
>
> faster than MRv1 on Map phase, but much worse on Reduce phase.
>
> Here I have two questions:
> *- Why my tests shows Yarn is worse than MRv1 for terasort?
> *
> *- What's the stratage for tuning Yarn performance? Is any materials?*
>
> Thanks!
>
--
Marcos Ortiz Valmaseda
Product Manager at PDVSA
http://about.me/marcosortiz
that Yarn is much
faster than MRv1 on Map phase, but much worse on Reduce phase.
Here I have two questions:
*- Why my tests shows Yarn is worse than MRv1 for terasort?
*
*- What's the stratage for tuning Yarn performance? Is any materials?*
Thanks!
16 matches
Mail list logo
