[ https://issues.apache.org/jira/browse/MAPREDUCE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Kanter updated MAPREDUCE-6415: ------------------------------------- Attachment: MAPREDUCE-6415_branch-2_prelim_002.patch MAPREDUCE-6415_prelim_002.patch The prelim_002 patch: - Uses {{YARN_SHELL_ID}} from YARN-3950 instead of parsing {{CONTAINER_ID}} - Runs 'hadoop archive' and the FileSystem commands from a Java program, so we can limit the JVM startup cost > Create a tool to combine aggregated logs into HAR files > ------------------------------------------------------- > > Key: MAPREDUCE-6415 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6415 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Affects Versions: 2.8.0 > Reporter: Robert Kanter > Assignee: Robert Kanter > Attachments: HAR-ableAggregatedLogs_v1.pdf, > MAPREDUCE-6415_branch-2_prelim_001.patch, > MAPREDUCE-6415_branch-2_prelim_002.patch, MAPREDUCE-6415_prelim_001.patch, > MAPREDUCE-6415_prelim_002.patch > > > While we wait for YARN-2942 to become viable, it would still be great to > improve the aggregated logs problem. We can write a tool that combines > aggregated log files into a single HAR file per application, which should > solve the too many files and too many blocks problems. See the design > document for details. > See YARN-2942 for more context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)