[
https://issues.apache.org/jira/browse/HADOOP-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peeyush Bishnoi updated HADOOP-5022:
------------------------------------
Attachment: hadoop-5022-1.txt
Thanks! for the comments. I incorporated the changes in the patch as per the
comments provided and uploaded the new patch . This new patch has "-r " and
"--retain-masters-logs" as option that need to be passed with value either
"true" or "false" while running the logcondense.py script . By default the
option is set to 'false' . It means it will delete the complete job directory
inside hod-logs in HDFS. But if option is set to 'true' it will delete only the
tasktracker logs . It will delete the Datanode logs , if "--dynamicdfs" is
'true'.
For example:
python logcondense.py -p ~/hadoop-0.17.0/bin/hadoop -d 7 -c ~/hadoop-conf -l
/user -r true
---
> [HOD] logcondense should delete all hod logs for a user, including jobtracker
> logs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-5022
> URL: https://issues.apache.org/jira/browse/HADOOP-5022
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Reporter: Hemanth Yamijala
> Assignee: Peeyush Bishnoi
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: hadoop-5022-1.txt, hadoop-5022.txt
>
>
> Currently, logcondense.py does not delete jobtracker logs that it uploads to
> the DFS when the HOD cluster is deallocated. This will result in the hod-logs
> directory to slowly accumulate a whole bunch of jobtracker logs. Particularly
> for users who run a lot of user jobs, this could fill up the namespace.
> Further these directories will cause the logcondense program to keep
> repeatedly looking at these directories stressing out the namenode. So,
> logcondense.py should optionally also delete the jobtracker logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
