[
https://issues.apache.org/jira/browse/MAPREDUCE-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated MAPREDUCE-3788:
----------------------------------------
Fix Version/s: (was: 0.24.0)
> [Gridmix] Investigate if Gridmix can be made YARN aware
> -------------------------------------------------------
>
> Key: MAPREDUCE-3788
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3788
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/gridmix
> Affects Versions: 0.24.0
> Reporter: Amar Kamat
> Labels: gridmix, yarn
>
> Gridmix was written keeping in mind the monolithic JobTracker. Calls to the
> single JobTracker were throttled to avoid excess load. Also, polling was
> faster in JobTracker as the job statuses were cached even if the job was
> complete. In the YARN world, the situation is slightly different. To make
> Gridmix scalable and really a YARN scale-benchmarking tool, Gridmix should be
> enhanced. Some directions worth investigating are:
> 1. Investigate if Gridmix can cache the AM handles and poll the AM directly
> for map/reduce task progress.
> 2. Can the job monitor be made multi-threaded? Each thread can poll a bunch
> of AMs.
> 3. Check if there are better ways for getting job progress updates and get
> away with the busy-waiting logic in Gridmix.
> 4. Can Gridmix be made container aware. The definition of cluster load should
> be container aware.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
