[
https://issues.apache.org/jira/browse/YARN-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606291#comment-16606291
]
Vrushali C edited comment on YARN-3841 at 9/6/18 7:25 PM:
----------------------------------------------------------
Thanks [~abmodi] ! Patch looks good overall. A couple of minor comments:
- let's use File.separator instead of an actual "/"
- For FileSystemTimelineWriterImpl.java, I think we may not want to do a
fs.close(); at line 261. This will close the FileSystem handle for all threads
in that process since this is a static instance.
- For line281, instead of
{{ LOG.info("Retrying operation on FS. Retry no. " + retry); }}
we could perhaps update it to
{{ "Will retry operation on FS. Retry no. " + retry + " after sleeping for "
+ fsRetryInterval + " seconds" ); }}
Will be a better indication of the sleep & retry. What do you think?
was (Author: vrushalic):
Thanks [~abmodi] ! Patch looks good overall. A couple of minor comments:
- let's use File.separator instead of an actual "/"
- For FileSystemTimelineWriterImpl.java, I think we may not want to do a
fs.close(); at line 261. This will close the FileSystem handle for all threads
in that process since this is a static instance.
- For line281, instead of
{{monospaced}} LOG.info("Retrying operation on FS. Retry no. " + retry);
{{monospaced}}
we could perhaps update it to
{{monospaced}} "Will retry operation on FS. Retry no. " + retry + " after
sleeping for " + fsRetryInterval + " seconds" ); {{monospaced}}
Will be a better indication of the sleep & retry. What do you think?
> [Storage implementation] Adding retry semantics to HDFS backing storage
> -----------------------------------------------------------------------
>
> Key: YARN-3841
> URL: https://issues.apache.org/jira/browse/YARN-3841
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Tsuyoshi Ozawa
> Assignee: Abhishek Modi
> Priority: Major
> Labels: YARN-5355
> Attachments: YARN-3841-YARN-7055.002.patch, YARN-3841.001.patch,
> YARN-3841.002.patch, YARN-3841.003.patch, YARN-3841.004.patch
>
>
> HDFS backing storage is useful for following scenarios.
> 1. For Hadoop clusters which don't run HBase.
> 2. For fallback from HBase when HBase cluster is temporary unavailable.
> Quoting ATS design document of YARN-2928:
> {quote}
> In the case the HBase
> storage is not available, the plugin should buffer the writes temporarily
> (e.g. HDFS), and flush
> them once the storage comes back online. Reading and writing to hdfs as the
> the backup storage
> could potentially use the HDFS writer plugin unless the complexity of
> generalizing the HDFS
> writer plugin for this purpose exceeds the benefits of reusing it here.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
