[
https://issues.apache.org/jira/browse/FLUME-762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099699#comment-13099699
]
[email protected] commented on FLUME-762:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1739/#review1806
-----------------------------------------------------------
I think if we get an interrupted exception that percolates up, we need to bail
out. Can you add a test for this? (also, since it may be the third time source
is implemented, maybe refactor?).
Also, to avoid having 2 second long tests, you can have next return null after
some count. This means that source is out of data and should exit the driver
"cleanly".
Nits: are you using tabs? please replace them with spaces.
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
<https://reviews.apache.org/r/1739/#comment4148>
I'm pretty convinced we need to bail out completely like before if we get
InterruptedExceptions here.
IO and any other runtime should retry once.
In either case, I think we should check the stopped variable before
retrying.
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
<https://reviews.apache.org/r/1739/#comment4149>
nit: formatting looks funny.
- LOG not indented properly.
- extra trailing spaces
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
<https://reviews.apache.org/r/1739/#comment4153>
Maybe say "Unexpected exception in source" ...
Also, use LOG.warn("string", eI). Doing eI.printStackTrace might not make
the log file!
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
<https://reviews.apache.org/r/1739/#comment4151>
Add something about "attempting to recover" in the log message?
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
<https://reviews.apache.org/r/1739/#comment4150>
nit: is this a tab?
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
<https://reviews.apache.org/r/1739/#comment4155>
Same comments about exception logging and Interrupted Exception.
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
<https://reviews.apache.org/r/1739/#comment4154>
Might want to update here saying that we attempted to recover but could
not.
flume-core/src/test/java/com/cloudera/flume/core/connector/TestDirectDriverExp.java
<https://reviews.apache.org/r/1739/#comment4156>
funny indents.
flume-core/src/test/java/com/cloudera/flume/core/connector/TestDirectDriverExp.java
<https://reviews.apache.org/r/1739/#comment4157>
Need to check something to make this a valid test? (or add comment/log
make it clear that success is if no exception happens).
flume-core/src/test/java/com/cloudera/flume/core/connector/TestDirectDriverExp.java
<https://reviews.apache.org/r/1739/#comment4158>
formatting funny again.
- jmhsieh
On 2011-09-07 23:18:39, Prasad Mujumdar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1739/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-07 23:18:39)
bq.
bq.
bq. Review request for jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. If the source or sink throws an exception, close, reopen and retry it.
This way the flow can continue after minor/recoverable errors.
bq.
bq.
bq. This addresses bug FLUME-762.
bq. https://issues.apache.org/jira/browse/FLUME-762
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
flume-core/src/main/java/com/cloudera/flume/core/connector/DirectDriver.java
0466394
bq.
flume-core/src/test/java/com/cloudera/flume/core/connector/TestDirectDriverExp.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/1739/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Prasad
bq.
bq.
> DirectDriver should reopen the source/sink if the append or next throws an
> exception
> ------------------------------------------------------------------------------------
>
> Key: FLUME-762
> URL: https://issues.apache.org/jira/browse/FLUME-762
> Project: Flume
> Issue Type: Improvement
> Components: Node
> Reporter: Prasad Mujumdar
> Assignee: Prasad Mujumdar
> Attachments: flume-762.patch
>
>
> Currently, if source or sink throws an exception, the driver bails out
> resulting the everything to shut down. Alternately, we can try to close and
> reopen the source or sink, and then attempt to continue. This way we can
> avoid blocking the flow. If the reopen fails, then it can bail out.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
