[
https://issues.apache.org/jira/browse/MAPREDUCE-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12840003#action_12840003
]
Amareshwari Sriramadasu commented on MAPREDUCE-927:
---------------------------------------------------
bq. Creation of the localFS should not be in a static block. In the past also,
we did this and realized it creates a kind of circular initialization of
loggers and results in NPE while creating log objects which can be seen in
task-logs.
Can you elaborate this? How will static initialization code result into
circular initialization of loggers? How will this result into NPE?
bq. Also can't we simply construct a local-filesystem here itself, instead of
calling TaskLog.getLocalFileSystem()?
I'm getting the log file system through TaskLog.getLogFileSystem() to make sure
that it is the same FileSystem used in TaskLog creation and deletion.
bq. Rename the class to TaskLogsMonitor, so that we are consistent going
forward with MAPREDUCE-1100.
bq. The logscleanup thread is not joined/killed in the TaskTracker.close(). So,
there will be zombie threads in the system on a re-init and may well interfere
with the new thread.
TaskLogMonitor introduced in MAPREDUCE-1100 is not a daemon thread, but
TaskLogCleanupThread needs to be. I don't think we can merge the functionality
of TaskLogCleanupThread and TaskLogMonitor into one thread.
TaskLogCleanupThread is not started after every re-init. It is started only
once similar to taskCleanupThread, directoryCleanupThread. So, this should not
be joined in TaskTracker.close().
> Cleanup of task-logs should happen in TaskTracker instead of the Child
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-927
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-927
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: security, tasktracker
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: patch-927-1.txt, patch-927.txt
>
>
> Task logs' cleanup is being done in Child now. This is undesirable atleast
> for two reasons: 1) failures while cleaning up will affect the user's tasks,
> and 2) the task's wall time will get affected due to operations that TT
> actually should own.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
