Hello list,
I use a Map/Reduce application processing several GB data on nine
different machines. To monitor exceptional cases, I use a combination of
counter enums (inreased by a Reporter) and console logging (with
java.util.logging).
In the Hadoop web interface I see that a specific counter enum was
increased exactly once during the mapping process. I would like to
inspect the corresponding logs. However, I can't figure out a
possibility to locate the corresponding log file.
My problem is that there are about 1200 map processes which I don't want
to inspect by hand.
Does anyone know a simple way to identify which map task increased a
specific counter or which task produced a given log message?
Best regards
Thomas