[
https://issues.apache.org/jira/browse/YARN-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937515#comment-16937515
]
Szilard Nemeth commented on YARN-9808:
--------------------------------------
Thanks [~adam.antal] for this patch!
Really appreciate that you extracted the formatContainerLogHeader method in
LogToolUtils and that you spent time to extend the testcases as [~shuzirra]
said before.
+1 on the latest patch, committing this to trunk soon!
[~adam.antal]:
I noticed duplicated code in
org.apache.hadoop.yarn.logaggregation.LogToolUtils#outputContainerLog,
duplicated in
org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat.LogReader#readContainerLogs.
This is not related to your patch, but could you please file a jira for that?
Most likely we could have similar jiras upcoming as the code here is very
bloated and dirty.
> Zero length files in container log output haven't got a header
> --------------------------------------------------------------
>
> Key: YARN-9808
> URL: https://issues.apache.org/jira/browse/YARN-9808
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: log-aggregation, yarn
> Affects Versions: 3.2.0
> Reporter: Adam Antal
> Assignee: Adam Antal
> Priority: Major
> Attachments: YARN-9808.001.patch, YARN-9808.002.patch,
> YARN-9808.003.patch
>
>
> Using the Yarn logs CLI for containers that have zero length files produces
> output similar to this:
> {noformat}
> End of LogType:stderr
> ***********************************************************************
> End of LogType:prelaunch.err
> ******************************************************************************
> Container: container_e25_1567431105510_0001_01_000002 on host-1
> LogAggregationType: AGGREGATED
> =======================================================================================================
> LogType:container.log
> LogLastModifiedTime:Mon Sep 02 06:34:48 -0700 2019
> LogLength:5442
> LogContents:
> ...
> ...
> {noformat}
> Note that stderr and prelaunch.err are both zero length files. Though the
> output is not misleading, the header is missing.
> I suggest to add the header for zero length files as well, primarily for the
> following reasons:
> - for applications having multiple files with the same name you may want to
> distinguish them by host - if many of those are of zero length, you can not
> extract this information from here. Note that this is a common case for
> stderr and prelaunch.err.
> - you may want to see the modification time (which corresponds to the
> creation time of the zero length file)
> - would explicitly display the "LogLength:0" line, which would avoid any
> confusion from end user side.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
