Hi Shrinivas, Yes, this is the behavior of the task logs when using JVM Reuse. You should notice in the log directories for the other tasks a "log index" file which specifies the byte offsets into the log files where the task starts and stops. When viewing logs through the web UI, it will use these index files to show you the right portion of the logs.
-Todd On Wed, Mar 30, 2011 at 1:17 PM, Shrinivas Joshi <jshrini...@gmail.com>wrote: > It seems like when JVM reuse is enabled map task log data is not getting > written to their corresponding log files; log data from certain map tasks > gets appended to log files corresponding to some other map task. > > For example, I have a case here where 8 map JVMs are running simultaneously > and all syslog data from map task 9, 17 and 25 gets appended in to log file > for map task 0. Whereas no syslog file gets generated in > attempt_*m_000009_0/ , attempt_*m_000017_0/ and attempt_*m_000025_0/ > folders. This job creates 32 map tasks. > This behavior might also be applicable to reduce log files, however, in our > case total # of reduce tasks is not more than max reduce JVMs running at > the > same time and hence it might not be manifesting. > > BTW, this is on Apache distro 0.21.0. > > -Shrinivas > -- Todd Lipcon Software Engineer, Cloudera
