[ https://issues.apache.org/jira/browse/MAPREDUCE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639513#comment-14639513 ]
Robert Kanter commented on MAPREDUCE-6415: ------------------------------------------ I that case, I suppose I could write a Java program that calls the 'hadoop archive' command programmatically, and then the equivalent 'hadoop fs' operations with the Java API. This would only require the one JVM startup. > Create a tool to combine aggregated logs into HAR files > ------------------------------------------------------- > > Key: MAPREDUCE-6415 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6415 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Affects Versions: 2.8.0 > Reporter: Robert Kanter > Assignee: Robert Kanter > Attachments: HAR-ableAggregatedLogs_v1.pdf, > MAPREDUCE-6415_branch-2_prelim_001.patch, MAPREDUCE-6415_prelim_001.patch > > > While we wait for YARN-2942 to become viable, it would still be great to > improve the aggregated logs problem. We can write a tool that combines > aggregated log files into a single HAR file per application, which should > solve the too many files and too many blocks problems. See the design > document for details. > See YARN-2942 for more context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)