[
https://issues.apache.org/jira/browse/YARN-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shilun Fan updated YARN-10406:
------------------------------
Target Version/s: 3.5.0 (was: 3.4.0)
> YARN log processor
> ------------------
>
> Key: YARN-10406
> URL: https://issues.apache.org/jira/browse/YARN-10406
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Adam Antal
> Assignee: Hudáky Márton Gyula
> Priority: Critical
>
> YARN currently does not have any utility that would enable cluster
> administrators to display previous actions in a Hadoop YARN cluster in an
> offline fashion.
> HDFS has the
> [OIV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html]/
>
> [OEV|https://hadoop.apache.org/docs/current3/hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html]
> which does not require a running cluster to look and modify the filesystem.
> A corresponding tool would be very helpful in the context of YARN.
> Since ATS is not widespread (is not available for older clusters) and there
> isn't a single file or entity that would collect all the
> application/container etc. related information, we thought our best option to
> parse and process the output of the YARN daemon log files and reconstruct the
> history of the cluster from that. We designed and implemented a CLI based
> solution that after parsing the log file enables users to query app/container
> related information (listing, filtering by certain properties) and search for
> common errors like CE failures/error codes, AM preemption or stack traces.
> The tool can be integrated into the YARN project as a sub-project.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
