[
https://issues.apache.org/jira/browse/STORM-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700291#comment-14700291
]
ASF GitHub Bot commented on STORM-969:
--------------------------------------
Github user dossett commented on a diff in the pull request:
https://github.com/apache/storm/pull/664#discussion_r37241329
--- Diff:
external/storm-hdfs/src/main/java/org/apache/storm/hdfs/bolt/HdfsBolt.java ---
@@ -80,6 +86,11 @@ public HdfsBolt addRotationAction(RotationAction action){
return this;
}
+ public HdfsBolt withTickTupleIntervalSeconds(int interval) {
--- End diff --
I like that change since it separates the function from the implementation.
A similar change could be made to HiveBolt, to which I also added a tick tuple
functionality recently. If this PR is otherwise ok, I'd like to open a new
JIRA to change this bolt and HiveBolt at the same time.
> HDFS Bolt can end up in an unrecoverable state
> ----------------------------------------------
>
> Key: STORM-969
> URL: https://issues.apache.org/jira/browse/STORM-969
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-hdfs
> Reporter: Aaron Dossett
> Assignee: Aaron Dossett
>
> The body of the HDFSBolt.execute() method is essentially one try-catch block.
> The catch block reports the error and fails the current tuple. In some
> cases the bolt's FSDataOutputStream object (named 'out') is in an
> unrecoverable state and no subsequent calls to execute() can succeed.
> To produce this scenario:
> - process some tuples through HDFS bolt
> - put the underlying HDFS system into safemode
> - process some more tuples and receive a correct ClosedChannelException
> - take the underlying HDFS system out of safemode
> - subsequent tuples continue to fail with the same exception
> The three fundamental operations that execute takes (writing, sync'ing,
> rotating) need to be isolated so that errors from each are specifically
> handled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
