On Aug 12, 2008, at 11:52 AM, Stuart Sierra wrote:
Hello, list, I've seen this question before, but haven't found an answer. If I run a Hadoop job on a cluster (EC2), how can I download the stdout/stderr logs from each mapper/reducer task? Are they stored somewhere in HDFS, or just on the machine that ran the task?
They are usually just present on the nodes, however if you running Hadoop on Demand (HoD - http://hadoop.apache.org/core/docs/current/hod.html) you get that feature with it.
OTOH - you might be interested in Chukwa (http://issues.apache.org/jira/browse/HADOOP-3719 ) - it's coming in hadoop-0.19 though, which is a while away.
Arun
