[jira] [Updated] (YARN-171) NodeManager should serve logs directly if log-aggregation is not enabled
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated YARN-171: Attachment: YARN-171_3.txt Uploading a newer, but still very dated version of the patch - which I had sitting around on my system. In case someone is taking over this jira - this could be used as a starting point, or not. NodeManager should serve logs directly if log-aggregation is not enabled Key: YARN-171 URL: https://issues.apache.org/jira/browse/YARN-171 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Affects Versions: 0.23.3 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Attachments: YARN-171_3.txt, YARN171_WIP.txt NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available. We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-171) NodeManager should serve logs directly if log-aggregation is not enabled
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-171: - Issue Type: Sub-task (was: Bug) Parent: YARN-431 NodeManager should serve logs directly if log-aggregation is not enabled Key: YARN-171 URL: https://issues.apache.org/jira/browse/YARN-171 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Affects Versions: 0.23.3 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Attachments: YARN171_WIP.txt NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available. We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-171) NodeManager should serve logs directly if log-aggregation is not enabled
[ https://issues.apache.org/jira/browse/YARN-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated YARN-171: Attachment: YARN171_WIP.txt Partial patch. Will get back to this after several days if noone has taken it over by then. The current patch is persisting per app ACLs via the LogHandler, and into NM private directories. The acl files could go along with logs - but with the current localization model, that's a bigger change - which could be done at a later point. Also, the log handler is probably not the best place for ACL verification. The log handler writing ACLs as well as verifying ACLs does have an advantage in terms of being able to deal with logs after an NM restart. NodeManager should serve logs directly if log-aggregation is not enabled Key: YARN-171 URL: https://issues.apache.org/jira/browse/YARN-171 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 0.23.3 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Attachments: YARN171_WIP.txt NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available. We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
