[jira] [Updated] (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-1716: - Target Version/s: 0.22.0 > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.20.3, 0.21.0, 0.21.1, 0.22.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1100-20091216.2.txt, > mapreduce-1716-testcase-race.txt, patch-1100-fix-ydist.2.txt, > patch-log-truncation-bugs-20100514.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-1716: --- Priority: Major (was: Blocker) Unblocking this. Target for 0.22.1 > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.20.3, 0.21.0, 0.21.1, 0.22.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1100-20091216.2.txt, > mapreduce-1716-testcase-race.txt, patch-1100-fix-ydist.2.txt, > patch-log-truncation-bugs-20100514.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-1716: --- Attachment: mapreduce-1716-testcase-race.txt This test case has a bit of a race, since it assumes that the logs will be truncated immediately upon job completion. This isn't the case, since the logs aren't added to the truncation manager until the JVM finished, which can be several seconds after the last task finishes (eg when sleepTimeBeforeSigKill is 5 seconds). So, I found that this test was flaky as is. You can show this by adding a 10 second sleep before truncating the logs, for example. This delta patch has the tests loop for 20 seconds while checking the logs for truncation. It only fails if the logs aren't truncated after 20 seconds. > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.20.3, 0.21.0, 0.21.1, 0.22.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1100-20091216.2.txt, > mapreduce-1716-testcase-race.txt, patch-1100-fix-ydist.2.txt, > patch-log-truncation-bugs-20100514.txt > > -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-1716: --- Priority: Blocker (was: Major) Making a blocker since it's parent task is. > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.20.3, 0.21.0, 0.21.1, 0.22.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1100-20091216.2.txt, > patch-1100-fix-ydist.2.txt, patch-log-truncation-bugs-20100514.txt > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-1716: --- Fix Version/s: 0.22.0 Affects Version/s: 0.20.3 0.21.1 0.22.0 > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.20.3, 0.21.0, 0.21.1, 0.22.0 >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1100-20091216.2.txt, > patch-1100-fix-ydist.2.txt, patch-log-truncation-bugs-20100514.txt > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod K V updated MAPREDUCE-1716: - Attachment: patch-log-truncation-bugs-20100514.txt Attaching one more patch for ydist that (1) fixes truncation problem when binary data is written to the logs and (2) adds a header at the beginning of a truncated log file: "[ ... this log file was truncated because of excess length]\n" > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.21.0 >Reporter: Vinod K V >Assignee: Vinod K V > Attachments: MAPREDUCE-1100-20091216.2.txt, > patch-1100-fix-ydist.2.txt, patch-log-truncation-bugs-20100514.txt > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod K V updated MAPREDUCE-1716: - Attachment: MAPREDUCE-1100-20091216.2.txt patch-1100-fix-ydist.2.txt Attaching the ydist patches for this issue. These were already uploaded on MAPREDUCE-1100 before this sub-task was created. > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.21.0 >Reporter: Vinod K V >Assignee: Vinod K V > Attachments: MAPREDUCE-1100-20091216.2.txt, patch-1100-fix-ydist.2.txt > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1716) Truncate logs of finished tasks to prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod K V updated MAPREDUCE-1716: - Affects Version/s: 0.21.0 Component/s: task tasktracker > Truncate logs of finished tasks to prevent node thrash due to excessive > logging > --- > > Key: MAPREDUCE-1716 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: task, tasktracker >Affects Versions: 0.21.0 >Reporter: Vinod K V >Assignee: Vinod K V > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
