[jira] Updated: (MAPREDUCE-1904) Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator
[ https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated MAPREDUCE-1904: Attachment: LocalDirAllocator.JPG LocalDirAllocator_Monitor.JPG CPU profiler output of a TaskTracker "without" the patch. Profiler output shows that ~4% of time being spent on internal methods of LocalDirAllocator.*.getLocalPathToRead(). Rest of them time is spent on the method itself. This along with the monitor profiling shows synchonization to be the bottleneck in getLocalPathToRead(). > Reducing locking contention in TaskTracker.MapOutputServlet's > LocalDirAllocator > --- > > Key: MAPREDUCE-1904 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tasktracker >Affects Versions: 0.20.1 >Reporter: Rajesh Balamohan > Attachments: LocalDirAllocator.JPG, LocalDirAllocator_Monitor.JPG, > MAPREDUCE-1904-RC10.patch, MAPREDUCE-1904-trunk.patch, profiler output after > applying the patch.jpg, TaskTracker- yourkit profiler output .jpg, Thread > profiler output showing contention.jpg > > > While profiling tasktracker with Sort benchmark, it was observed that threads > block on LocalDirAllocator.getLocalPathToRead() in order to get the index > file and temporary map output file. > As LocalDirAllocator is tied up with ServetContext, only one instance would > be available per tasktracker httpserver. Given the jobid & mapid, > LocalDirAllocator retrieves index file path and temporary map output file > path. getLocalPathToRead() is internally synchronized. > Introducing a LRUCache for this lookup reduces the contention heavily > (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the > LRUCache can be varied based on the environment and I observed a throughput > improvement in the order of 4-7% with the introduction of LRUCache. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1904) Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator
[ https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated MAPREDUCE-1904: Attachment: MAPREDUCE-1904-trunk.patch Attaching the patch for trunk version. > Reducing locking contention in TaskTracker.MapOutputServlet's > LocalDirAllocator > --- > > Key: MAPREDUCE-1904 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tasktracker >Affects Versions: 0.20.1 >Reporter: Rajesh Balamohan > Attachments: MAPREDUCE-1904-RC10.patch, MAPREDUCE-1904-trunk.patch, > profiler output after applying the patch.jpg, TaskTracker- yourkit profiler > output .jpg, Thread profiler output showing contention.jpg > > > While profiling tasktracker with Sort benchmark, it was observed that threads > block on LocalDirAllocator.getLocalPathToRead() in order to get the index > file and temporary map output file. > As LocalDirAllocator is tied up with ServetContext, only one instance would > be available per tasktracker httpserver. Given the jobid & mapid, > LocalDirAllocator retrieves index file path and temporary map output file > path. getLocalPathToRead() is internally synchronized. > Introducing a LRUCache for this lookup reduces the contention heavily > (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the > LRUCache can be varied based on the environment and I observed a throughput > improvement in the order of 4-7% with the introduction of LRUCache. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1904) Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator
[ https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated MAPREDUCE-1904: Attachment: Thread profiler output showing contention.jpg > Reducing locking contention in TaskTracker.MapOutputServlet's > LocalDirAllocator > --- > > Key: MAPREDUCE-1904 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tasktracker >Affects Versions: 0.20.1 >Reporter: Rajesh Balamohan > Attachments: MAPREDUCE-1904-RC10.patch, profiler output after > applying the patch.jpg, TaskTracker- yourkit profiler output .jpg, Thread > profiler output showing contention.jpg > > > While profiling tasktracker with Sort benchmark, it was observed that threads > block on LocalDirAllocator.getLocalPathToRead() in order to get the index > file and temporary map output file. > As LocalDirAllocator is tied up with ServetContext, only one instance would > be available per tasktracker httpserver. Given the jobid & mapid, > LocalDirAllocator retrieves index file path and temporary map output file > path. getLocalPathToRead() is internally synchronized. > Introducing a LRUCache for this lookup reduces the contention heavily > (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the > LRUCache can be varied based on the environment and I observed a throughput > improvement in the order of 4-7% with the introduction of LRUCache. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1904) Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator
[ https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated MAPREDUCE-1904: Attachment: profiler output after applying the patch.jpg Contention on LocalDirAllocator is very less. Close to 0% > Reducing locking contention in TaskTracker.MapOutputServlet's > LocalDirAllocator > --- > > Key: MAPREDUCE-1904 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tasktracker >Affects Versions: 0.20.1 >Reporter: Rajesh Balamohan > Attachments: MAPREDUCE-1904-RC10.patch, profiler output after > applying the patch.jpg, TaskTracker- yourkit profiler output .jpg, Thread > profiler output showing contention.jpg > > > While profiling tasktracker with Sort benchmark, it was observed that threads > block on LocalDirAllocator.getLocalPathToRead() in order to get the index > file and temporary map output file. > As LocalDirAllocator is tied up with ServetContext, only one instance would > be available per tasktracker httpserver. Given the jobid & mapid, > LocalDirAllocator retrieves index file path and temporary map output file > path. getLocalPathToRead() is internally synchronized. > Introducing a LRUCache for this lookup reduces the contention heavily > (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the > LRUCache can be varied based on the environment and I observed a throughput > improvement in the order of 4-7% with the introduction of LRUCache. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1904) Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator
[ https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated MAPREDUCE-1904: Attachment: TaskTracker- yourkit profiler output .jpg LocalDirAllocator.AllocatorPerContext is heavily contended. > Reducing locking contention in TaskTracker.MapOutputServlet's > LocalDirAllocator > --- > > Key: MAPREDUCE-1904 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tasktracker >Affects Versions: 0.20.1 >Reporter: Rajesh Balamohan > Attachments: MAPREDUCE-1904-RC10.patch, TaskTracker- yourkit profiler > output .jpg > > > While profiling tasktracker with Sort benchmark, it was observed that threads > block on LocalDirAllocator.getLocalPathToRead() in order to get the index > file and temporary map output file. > As LocalDirAllocator is tied up with ServetContext, only one instance would > be available per tasktracker httpserver. Given the jobid & mapid, > LocalDirAllocator retrieves index file path and temporary map output file > path. getLocalPathToRead() is internally synchronized. > Introducing a LRUCache for this lookup reduces the contention heavily > (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the > LRUCache can be varied based on the environment and I observed a throughput > improvement in the order of 4-7% with the introduction of LRUCache. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1904) Reducing locking contention in TaskTracker.MapOutputServlet's LocalDirAllocator
[ https://issues.apache.org/jira/browse/MAPREDUCE-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated MAPREDUCE-1904: Attachment: MAPREDUCE-1904-RC10.patch Patch for RC10 release is attached here. > Reducing locking contention in TaskTracker.MapOutputServlet's > LocalDirAllocator > --- > > Key: MAPREDUCE-1904 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1904 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tasktracker >Affects Versions: 0.20.1 >Reporter: Rajesh Balamohan > Attachments: MAPREDUCE-1904-RC10.patch > > > While profiling tasktracker with Sort benchmark, it was observed that threads > block on LocalDirAllocator.getLocalPathToRead() in order to get the index > file and temporary map output file. > As LocalDirAllocator is tied up with ServetContext, only one instance would > be available per tasktracker httpserver. Given the jobid & mapid, > LocalDirAllocator retrieves index file path and temporary map output file > path. getLocalPathToRead() is internally synchronized. > Introducing a LRUCache for this lookup reduces the contention heavily > (LRUCache with key =jobid +mapid and value=PATH to the file). Size of the > LRUCache can be varied based on the environment and I observed a throughput > improvement in the order of 4-7% with the introduction of LRUCache. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
