[
https://issues.apache.org/jira/browse/YARN-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146032#comment-16146032
]
Jason Lowe commented on YARN-6876:
----------------------------------
Previously the IOUtils.cleanup calls were passed a logger that would be used to
report errors if the close failed. That was lost as part of this change, so
now closes can fail (and data lost in the aggregated log files) without any
indication in the NM logs. Maybe I'm mistaken, but I do not see a case where
we would want to call close on a writer or stream in log aggregation and
quietly suppress errors that arise during the close process.
I agree that for the closeStream case we should be calling cleanupWithLogger so
at least the error is logged somewhere rather than silently suppressed.
> Create an abstract log writer for extendability
> -----------------------------------------------
>
> Key: YARN-6876
> URL: https://issues.apache.org/jira/browse/YARN-6876
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Xuan Gong
> Assignee: Xuan Gong
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-6876-branch-2.001.patch, YARN-6876-trunk.001.patch,
> YARN-6876-trunk.002.patch, YARN-6876-trunk.003.patch,
> YARN-6876-trunk.004.patch, YARN-6876-trunk.005.patch,
> YARN-6876-trunk.006.patch
>
>
> Currently, TFile log writer is used to aggregate log in YARN. We need to add
> an abstract layer, and pick up the correct log writer based on the
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
