[
https://issues.apache.org/jira/browse/FLUME-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250570#comment-13250570
]
[email protected] commented on FLUME-1089:
------------------------------------------------------
bq. On 2012-04-10 00:01:17, Arvind Prabhakar wrote:
bq. > Thanks for the patch Brock. I think what this patch does is forces a
state transition on close no matter what. This has the potential of covering up
for programmatic problems that could lead to resource/tx leaks in the system
which I feel should not happen. If a component is buggy, the other components
around it should not try to coverup.
bq. >
bq. > Another way to look at it is - the close() method should not throw an
exception ever. This can be further reinforced by having a thread local
transaction that is discarded on close.
bq.
bq. Brock Noland wrote:
bq. I can agree with that.
bq.
bq. The new code would do the state transition (which means a new
transaction is gotten on getTransaction()) and then call doClose(). Correct?
bq.
bq. Arvind Prabhakar wrote:
bq. My view on it is that there are two parts to this problem:
bq.
bq. 1. If someone calls close() when the tx is not in the correct state,
that should fail with an exception. This signals a bad/buggy implementation
that should be identified aggressively and fixed.
bq.
bq. 2. If someone calls close() when the tx is in the correct state, that
should never fail. This will ensure that good code is not penalized for
implementation issues of the tx provider.
bq.
bq.
bq.
bq. Brock Noland wrote:
bq. In my understanding from the email chain "Channel/Transaction States"
was that like a DB statement, you should be able to call close() should be safe
to call at any point in time. If work is uncommitted that work is thrown away.
bq.
bq. If we require rollback or commit to be called before close, then every
source/sink needs to catch Throwable, call rollback and rethrow so that close
can be called in the finally block. Thoughts?
bq.
bq. Arvind Prabhakar wrote:
bq. The use of transaction must be done in an idiomatic manner as
described in it's api:
bq.
bq. * Channel ch = ...
bq. * Transaction tx = ch.getTransaction();
bq. * try {
bq. * tx.begin();
bq. * ...
bq. * // ch.put(event) or ch.take()
bq. * ...
bq. * tx.commit();
bq. * } catch (Exception ex) {
bq. * tx.rollback();
bq. * ...
bq. * } finally {
bq. * tx.close();
bq. * }
bq.
bq. If the caller is using this idiom, then it is a guarantee that the
state transition will occur correctly, and that for every begin there is a
close. As you can see from this idiom, the close should not be throwing an
exception (and implicitly the begin too).
bq.
bq. Brock Noland wrote:
bq. The issue with the idom above is that if anything is thrown which not
an Exception (e.g. subclass of Error), an exception will be thrown in the
finally clause and that more serious problem will be eaten. The only way this
can been handled is:
bq.
bq. * boolean readyForClose = false;
bq. * Channel ch = ...
bq. * Transaction tx = ch.getTransaction();
bq. * try {
bq. * tx.begin();
bq. * ...
bq. * // ch.put(event) or ch.take()
bq. * ...
bq. * tx.commit();
bq. * readyForClose = true;
bq. * } catch (Exception ex) {
bq. * tx.rollback();
bq. * readyForClose = true;
bq. * ...
bq. * } finally {
bq. * if(readyForClose) {
bq. * tx.close();
bq. * } else {
bq. * tx.rollback();
bq. * tx.close();
bq. * }
bq.
bq. It seems quite a lot of effort to push on our users and is quite bug
prone.
bq.
bq. Brock Noland wrote:
bq. Or as an alternative to the above you can catch Error, rollback and
then re-throw...
bq.
bq. Arvind Prabhakar wrote:
bq. I feel that if the close() method never throws an exception, the idiom
is perfectly fine in all cases. Besides, if an Error type does occur, then it
is ok to leak tx resources. I do acknowledge that requiring all clients of this
API to follow this idiom is a bit of a drag, but it ensures easy switching of
the channel when necessary. It also gives an easy way to use
telescoping/reference-counting semantics where necessary.
These two JUnit examples shows what I mean. Below a serious error is thrown:
@Test
public void testExample() throws Exception {
Event event = EventBuilder.withBody("test event".getBytes());
Channel channel = new MemoryChannel();
Context context = new Context();
Configurables.configure(channel, context);
Transaction tx = channel.getTransaction();
try {
tx.begin();
channel.put(event);
if(true) {
throw new Error("Error class means a serious problem occurred");
}
tx.commit();
} catch (Exception ex) {
tx.rollback();
throw ex;
} finally {
tx.close();
}
}
But all we get is:
java.lang.IllegalStateException: close() called when transaction is OPEN - you
must either commit or rollback first
at
com.google.common.base.Preconditions.checkState(Preconditions.java:172)
at
org.apache.flume.channel.BasicTransactionSemantics.close(BasicTransactionSemantics.java:179)
at
org.apache.flume.channel.TestMemoryChannel.testExample(TestMemoryChannel.java:64)
In order to handle this correctly we have to take additional action like so:
@Test
public void testExample() throws Exception {
Event event = EventBuilder.withBody("test event".getBytes());
Channel channel = new MemoryChannel();
Context context = new Context();
Configurables.configure(channel, context);
Transaction tx = channel.getTransaction();
try {
tx.begin();
channel.put(event);
if(true) {
throw new Error("Error class means a serious problem occurred");
}
tx.commit();
} catch (Exception ex) {
tx.rollback();
throw ex;
} catch (Error error) {
tx.rollback();
throw error;
} finally {
tx.close();
}
}
Now we get the real error:
java.lang.Error: Error class means a serious problem occurred
at
org.apache.flume.channel.TestMemoryChannel.testExample(TestMemoryChannel.java:57)
- Brock
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4655/#review6810
-----------------------------------------------------------
On 2012-04-05 03:05:51, Brock Noland wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4655/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-05 03:05:51)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Allowing the calling of transaction.close() at any point of time.
bq.
bq.
bq. This addresses bug FLUME-1089.
bq. https://issues.apache.org/jira/browse/FLUME-1089
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
flume-ng-core/src/main/java/org/apache/flume/channel/BasicTransactionSemantics.java
403cbca
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestBasicChannelSemantics.java
80020fc
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannelTransaction.java
bc81f26
bq.
bq. Diff: https://reviews.apache.org/r/4655/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Brock
bq.
bq.
> Transaction.close should be safe to call at any point
> -----------------------------------------------------
>
> Key: FLUME-1089
> URL: https://issues.apache.org/jira/browse/FLUME-1089
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.2.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-1089-0.patch, FLUME-1089-1.patch
>
>
> We are struggling with error handling in regards to transactions. The general
> consensus is that it should be safe to call close on a transaction at any
> point.
> Discussion on flume-dev here:
> http://mail-archives.apache.org/mod_mbox/incubator-flume-dev/201203.mbox/%3CCAFukC=5X99U=aOZ5qMR_OoF=pz9f7yhl6ofkyzu08ut4or0...@mail.gmail.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
