[
https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated MAPREDUCE-1904:
----------------------------------------
Attachment: MAPREDUCE-1904-trunk.patch
Attaching the patch for trunk version.
> Reducing locking contention in TaskTracker.MapOutputServlet's
> LocalDirAllocator
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1904
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.20.1
> Reporter: Rajesh Balamohan
> Attachments: MAPREDUCE-1904-RC10.patch, MAPREDUCE-1904-trunk.patch,
> profiler output after applying the patch.jpg, TaskTracker- yourkit profiler
> output .jpg, Thread profiler output showing contention.jpg
>
>
> While profiling tasktracker with Sort benchmark, it was observed that threads
> block on LocalDirAllocator.getLocalPathToRead() in order to get the index
> file and temporary map output file.
> As LocalDirAllocator is tied up with ServetContext, only one instance would
> be available per tasktracker httpserver. Given the jobid & mapid,
> LocalDirAllocator retrieves index file path and temporary map output file
> path. getLocalPathToRead() is internally synchronized.
> Introducing a LRUCache for this lookup reduces the contention heavily
> (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the
> LRUCache can be varied based on the environment and I observed a throughput
> improvement in the order of 4-7% with the introduction of LRUCache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
