Gopal V created TEZ-1491:
----------------------------
Summary: Tez reducer-side merge's counter update is slow
Key: TEZ-1491
URL: https://issues.apache.org/jira/browse/TEZ-1491
Project: Apache Tez
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Gopal V
Assignee: Gopal V
TezMerger$MergeQueue::next() shows up in profiles due a synchronized block in a
tight loop.
Part of the slow operation was due to DataInputBuffer issues identified earlier
in HADOOP-10694, but along with that approx 11% of my lock prefix calls were
originating from the following line.
{code}
mergeProgress.set(totalBytesProcessed * progPerByte);
{code}
in two places within the core loop.
!perf-top-counters.png!
--
This message was sent by Atlassian JIRA
(v6.2#6252)
