[jira] [Updated] (YARN-10406) YARN log processor
[ https://issues.apache.org/jira/browse/YARN-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated YARN-10406: -- Target Version/s: 3.5.0 (was: 3.4.0) > YARN log processor > -- > > Key: YARN-10406 > URL: https://issues.apache.org/jira/browse/YARN-10406 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: Adam Antal >Assignee: Hudáky Márton Gyula >Priority: Critical > > YARN currently does not have any utility that would enable cluster > administrators to display previous actions in a Hadoop YARN cluster in an > offline fashion. > HDFS has the > [OIV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html]/ > > [OEV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html] > which does not require a running cluster to look and modify the filesystem. > A corresponding tool would be very helpful in the context of YARN. > Since ATS is not widespread (is not available for older clusters) and there > isn't a single file or entity that would collect all the > application/container etc. related information, we thought our best option to > parse and process the output of the YARN daemon log files and reconstruct the > history of the cluster from that. We designed and implemented a CLI based > solution that after parsing the log file enables users to query app/container > related information (listing, filtering by certain properties) and search for > common errors like CE failures/error codes, AM preemption or stack traces. > The tool can be integrated into the YARN project as a sub-project. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10406) YARN log processor
[ https://issues.apache.org/jira/browse/YARN-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated YARN-10406: -- Description: YARN currently does not have any utility that would enable cluster administrators to display previous actions in a Hadoop YARN cluster in an offline fashion. HDFS has the [OIV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html]/ [OEV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html] which does not require a running cluster to look and modify the filesystem. A corresponding tool would be very helpful in the context of YARN. Since ATS is not widespread (is not available for older clusters) and there isn't a single file or entity that would collect all the application/container etc. related information, we thought our best option to parse and process the output of the YARN daemon log files and reconstruct the history of the cluster from that. We designed and implemented a CLI based solution that after parsing the log file enables users to query app/container related information (listing, filtering by certain properties) and search for common errors like CE failures/error codes, AM preemption or stack traces. The tool can be integrated into the YARN project as a sub-project. was: YARN currently does not have any utility that would enable cluster administrators to display previous actions in a Hadoop YARN cluster in an offline fashion. HDFS has the OIV/OEV which does not require a running cluster to look and modify the filesystem. A corresponding tool would be very helpful in the context of YARN. Since ATS is not widespread (is not available for older clusters) and there isn't a single file or entity that would collect all the application/container etc. related information, we thought our best option to parse and process the output of the YARN daemon log files and reconstruct the history of the cluster from that. We designed and implemented a CLI based solution that after parsing the log file enables users to query app/container related information (listing, filtering by certain properties) and search for common errors like CE failures/error codes, AM preemption or stack traces. The tool can be integrated into the YARN project as a sub-project. > YARN log processor > -- > > Key: YARN-10406 > URL: https://issues.apache.org/jira/browse/YARN-10406 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: Adam Antal >Assignee: Hudáky Márton Gyula >Priority: Critical > > YARN currently does not have any utility that would enable cluster > administrators to display previous actions in a Hadoop YARN cluster in an > offline fashion. > HDFS has the > [OIV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html]/ > > [OEV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html] > which does not require a running cluster to look and modify the filesystem. > A corresponding tool would be very helpful in the context of YARN. > Since ATS is not widespread (is not available for older clusters) and there > isn't a single file or entity that would collect all the > application/container etc. related information, we thought our best option to > parse and process the output of the YARN daemon log files and reconstruct the > history of the cluster from that. We designed and implemented a CLI based > solution that after parsing the log file enables users to query app/container > related information (listing, filtering by certain properties) and search for > common errors like CE failures/error codes, AM preemption or stack traces. > The tool can be integrated into the YARN project as a sub-project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10406) YARN log processor
[ https://issues.apache.org/jira/browse/YARN-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated YARN-10406: -- Description: YARN currently does not have any utility that would enable cluster administrators to display previous actions in a Hadoop YARN cluster in an offline fashion. HDFS has the OIV/OEV which does not require a running cluster to look and modify the filesystem. A corresponding tool would be very helpful in the context of YARN. Since ATS is not widespread (is not available for older clusters) and there isn't a single file or entity that would collect all the application/container etc. related information, we thought our best option to parse and process the output of the YARN daemon log files and reconstruct the history of the cluster from that. We designed and implemented a CLI based solution that after parsing the log file enables users to query app/container related information (listing, filtering by certain properties) and search for common errors like CE failures/error codes, AM preemption or stack traces. The tool can be integrated into the YARN project as a sub-project. was: YARN currently does not have any utility that would enable cluster administrators to re-play actions in a Hadoop YARN cluster in an offline fashion. HDFS has the OIV/OEV which does not require a running cluster to look and modify the filesystem. A corresponding tool would be very helpful in the context of YARN. Since ATS is not widespread (is not available for older clusters) and there isn't a single file or entity that would collect all the application/container etc. related information, we thought our best option to parse and process the output of the YARN daemon log files and reconstruct the history of the cluster from that. We designed and implemented a CLI based solution that after parsing the log file enables users to query app/container related information (listing, filtering by certain properties) and search for common errors like CE failures/error codes, AM preemption or stack traces. The tool can be integrated into the YARN project as a sub-project. > YARN log processor > -- > > Key: YARN-10406 > URL: https://issues.apache.org/jira/browse/YARN-10406 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn >Reporter: Adam Antal >Assignee: Hudáky Márton Gyula >Priority: Critical > > YARN currently does not have any utility that would enable cluster > administrators to display previous actions in a Hadoop YARN cluster in an > offline fashion. > HDFS has the OIV/OEV which does not require a running cluster to look and > modify the filesystem. A corresponding tool would be very helpful in the > context of YARN. > Since ATS is not widespread (is not available for older clusters) and there > isn't a single file or entity that would collect all the > application/container etc. related information, we thought our best option to > parse and process the output of the YARN daemon log files and reconstruct the > history of the cluster from that. We designed and implemented a CLI based > solution that after parsing the log file enables users to query app/container > related information (listing, filtering by certain properties) and search for > common errors like CE failures/error codes, AM preemption or stack traces. > The tool can be integrated into the YARN project as a sub-project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
