TaskTracker.addDiagnostics(String file, int num, String tag) could exit early
if num==0
---------------------------------------------------------------------------------------
Key: HADOOP-3439
URL: https://issues.apache.org/jira/browse/HADOOP-3439
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Affects Versions: 0.18.0
Reporter: Steve Loughran
Priority: Minor
When a TaskTracker job finishes, taskFinished() is invoked.
as part of its work it
1. loads in a conf option (that is not in hadoop-default, incidentally) ,
mapred.debug.out.lines , default value -1;
2. calls addDiagnostics passing in that line count
addDiagnostics either builds a string buffer of all the output, or creates a
linear array of lines and runs adds them, shuffling them up if there are more
lines than expected.
This is all unneeded if the number of lines to print == 0; the entire reading
in of the output file can be skipped. This may speed up termination slightly on
a run with a large output file and mapred.debug.out.lines ==0.
Note also that a circular buffer would handle the lines>0 problem without
having to copy all the strings around.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
