rbalamohan commented on code in PR #314:
URL: https://github.com/apache/tez/pull/314#discussion_r1370987305
##########
tez-runtime-internals/src/main/java/org/apache/tez/runtime/task/TaskReporter.java:
##########
@@ -262,6 +263,13 @@ private synchronized ResponseWrapper
heartbeat(Collection<TezEvent> eventsArg) t
sendCounters = true;
prevCounterSendHeartbeatNum = nonOobHeartbeatCounter.get();
}
+ try {
+ task.checkTaskLimits();
Review Comment:
Doing this in heartbeat can significantly increase its runtime. As in, even
small increase of few milliseconds can have impact on cluster usage. You may
want to run this every 10 seconds here or in some other way. This should
ideally handle sorter spills and merges, in which case it can handle another
ticket which was created on similar lines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
