[
https://issues.apache.org/jira/browse/FLUME-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251474#comment-13251474
]
[email protected] commented on FLUME-1089:
------------------------------------------------------
bq. On 2012-04-10 00:01:17, Arvind Prabhakar wrote:
bq. > Thanks for the patch Brock. I think what this patch does is forces a
state transition on close no matter what. This has the potential of covering up
for programmatic problems that could lead to resource/tx leaks in the system
which I feel should not happen. If a component is buggy, the other components
around it should not try to coverup.
bq. >
bq. > Another way to look at it is - the close() method should not throw an
exception ever. This can be further reinforced by having a thread local
transaction that is discarded on close.
bq.
bq. Brock Noland wrote:
bq. I can agree with that.
bq.
bq. The new code would do the state transition (which means a new
transaction is gotten on getTransaction()) and then call doClose(). Correct?
bq.
bq. Arvind Prabhakar wrote:
bq. My view on it is that there are two parts to this problem:
bq.
bq. 1. If someone calls close() when the tx is not in the correct state,
that should fail with an exception. This signals a bad/buggy implementation
that should be identified aggressively and fixed.
bq.
bq. 2. If someone calls close() when the tx is in the correct state, that
should never fail. This will ensure that good code is not penalized for
implementation issues of the tx provider.
bq.
bq.
bq.
bq. Brock Noland wrote:
bq. In my understanding from the email chain "Channel/Transaction States"
was that like a DB statement, you should be able to call close() should be safe
to call at any point in time. If work is uncommitted that work is thrown away.
bq.
bq. If we require rollback or commit to be called before close, then every
source/sink needs to catch Throwable, call rollback and rethrow so that close
can be called in the finally block. Thoughts?
bq.
bq. Arvind Prabhakar wrote:
bq. The use of transaction must be done in an idiomatic manner as
described in it's api:
bq.
bq. * Channel ch = ...
bq. * Transaction tx = ch.getTransaction();
bq. * try {
bq. * tx.begin();
bq. * ...
bq. * // ch.put(event) or ch.take()
bq. * ...
bq. * tx.commit();
bq. * } catch (Exception ex) {
bq. * tx.rollback();
bq. * ...
bq. * } finally {
bq. * tx.close();
bq. * }
bq.
bq. If the caller is using this idiom, then it is a guarantee that the
state transition will occur correctly, and that for every begin there is a
close. As you can see from this idiom, the close should not be throwing an
exception (and implicitly the begin too).
bq.
bq. Brock Noland wrote:
bq. The issue with the idom above is that if anything is thrown which not
an Exception (e.g. subclass of Error), an exception will be thrown in the
finally clause and that more serious problem will be eaten. The only way this
can been handled is:
bq.
bq. * boolean readyForClose = false;
bq. * Channel ch = ...
bq. * Transaction tx = ch.getTransaction();
bq. * try {
bq. * tx.begin();
bq. * ...
bq. * // ch.put(event) or ch.take()
bq. * ...
bq. * tx.commit();
bq. * readyForClose = true;
bq. * } catch (Exception ex) {
bq. * tx.rollback();
bq. * readyForClose = true;
bq. * ...
bq. * } finally {
bq. * if(readyForClose) {
bq. * tx.close();
bq. * } else {
bq. * tx.rollback();
bq. * tx.close();
bq. * }
bq.
bq. It seems quite a lot of effort to push on our users and is quite bug
prone.
bq.
bq. Brock Noland wrote:
bq. Or as an alternative to the above you can catch Error, rollback and
then re-throw...
bq.
bq. Arvind Prabhakar wrote:
bq. I feel that if the close() method never throws an exception, the idiom
is perfectly fine in all cases. Besides, if an Error type does occur, then it
is ok to leak tx resources. I do acknowledge that requiring all clients of this
API to follow this idiom is a bit of a drag, but it ensures easy switching of
the channel when necessary. It also gives an easy way to use
telescoping/reference-counting semantics where necessary.
bq.
bq. Brock Noland wrote:
bq. These two JUnit examples shows what I mean. Below a serious error is
thrown:
bq.
bq. @Test
bq. public void testExample() throws Exception {
bq. Event event = EventBuilder.withBody("test event".getBytes());
bq. Channel channel = new MemoryChannel();
bq. Context context = new Context();
bq. Configurables.configure(channel, context);
bq. Transaction tx = channel.getTransaction();
bq. try {
bq. tx.begin();
bq. channel.put(event);
bq. if(true) {
bq. throw new Error("Error class means a serious problem occurred");
bq. }
bq. tx.commit();
bq. } catch (Exception ex) {
bq. tx.rollback();
bq. throw ex;
bq. } finally {
bq. tx.close();
bq. }
bq. }
bq.
bq. But all we get is:
bq.
bq. java.lang.IllegalStateException: close() called when transaction is
OPEN - you must either commit or rollback first
bq. at
com.google.common.base.Preconditions.checkState(Preconditions.java:172)
bq. at
org.apache.flume.channel.BasicTransactionSemantics.close(BasicTransactionSemantics.java:179)
bq. at
org.apache.flume.channel.TestMemoryChannel.testExample(TestMemoryChannel.java:64)
bq.
bq. In order to handle this correctly we have to take additional action
like so:
bq.
bq. @Test
bq. public void testExample() throws Exception {
bq. Event event = EventBuilder.withBody("test event".getBytes());
bq. Channel channel = new MemoryChannel();
bq. Context context = new Context();
bq. Configurables.configure(channel, context);
bq. Transaction tx = channel.getTransaction();
bq. try {
bq. tx.begin();
bq. channel.put(event);
bq. if(true) {
bq. throw new Error("Error class means a serious problem occurred");
bq. }
bq. tx.commit();
bq. } catch (Exception ex) {
bq. tx.rollback();
bq. throw ex;
bq. } catch (Error error) {
bq. tx.rollback();
bq. throw error;
bq. } finally {
bq. tx.close();
bq. }
bq. }
bq.
bq. Now we get the real error:
bq.
bq. java.lang.Error: Error class means a serious problem occurred
bq. at
org.apache.flume.channel.TestMemoryChannel.testExample(TestMemoryChannel.java:57)
bq.
bq. Arvind Prabhakar wrote:
bq. My apologies for dragging this out so far, but I do see your point.
One way to address both these concerns is to catch a Throwable instead. Do you
think that would work?
bq.
bq.
bq. Will McQueen wrote:
bq. I agree with what Arvind mentioned earlier about Error being thrown,
"if an Error type does occur, then it is ok to leak tx resources". According to
JavaDocs for Error, "An Error is a subclass of Throwable that indicates serious
problems that a reasonable application should not try to catch". My
understanding is that the JVM can cause an Error (or one of its subclasses) to
be thrown within any or all of your threads (effectively inserting a "throw
<Error>" into any thread, and at any time... and so after any bytecode
instruction). Not only that, but I believe that the JVM can throw an Error
multiple times (eg, OutOfMemoryException). So when we encounter an Error, I
feel we should just propagate it without taking any additional action.
bq.
bq. Will McQueen wrote:
bq. Here's some code that shows the state of the txn at various places
within the code. There are 4 states: NEW, OPEN, COMPLETED, CLOSED. "ISE" means
"IllegalStateException", which can be thrown by rollback() or by close(). The
methods foo_1(), foo_2(), and foo_3() are each assumed to throw a Throwable (or
subclass of Throwable). The comments after each method show valid state
transitions and invalid (with XXX) ones. This sample code with comments shows
some cases that we may need to consider.
bq.
bq. try {
bq. //NEW
bq. foo_1(); //No ISE thrown for "(NEW ---close()---> CLOSED)", and
ISE thrown for "(NEW ---XXXrollback()XXX---> XXX)"
bq. tx.begin();
bq. //OPEN
bq. foo_2(); //ISE thrown for "(OPEN ---XXXclose()XXX---> XXX)", and
ISE thrown for "(OPEN ---XXXrollback()XXX---> XXX)"
bq. tx.commit();
bq. //COMPLETED
bq. foo_3(); //No ISE thrown for "(COMPLETED ---close()---> CLOSED)",
and ISE thrown for "(COMPLETED ---XXXrollback()XXX---> XXX)"
bq. } catch (Exception ex) {
bq. //NEW (due to foo_1), OPEN (due to foo_2), or COMPLETED (due to
foo_3) when Exception thrown
bq. tx.rollback(); //ISE thrown if got to this catch block due to
foo_1 or foo_3 throwing Exception. The ISE will replace the Exception.
bq. //COMPLETED
bq. throw ex;
bq. } catch (Error error) {
bq. //NEW (due to foo_1), OPEN (due to foo_2), or COMPLETED (due to
foo_3) when Error thrown
bq. tx.rollback(); //ISE thrown if got to this catch block due to
foo_1 or foo_3 throwing Error. The ISE will replace the Error.
bq. //COMPLETED (due to foo_2)
bq. throw error;
bq. } finally {
bq. //NEW (due to foo_1), OPEN (due to foo_2), COMPLETED (due to
foo_3)
bq. tx.close(); //ISE thrown if got to this catch block due to foo_2
bq. //CLOSED
bq. }
bq.
bq. Will McQueen wrote:
bq. A possible solution is shown below. I could not think of a more
elegant way to do this, while still accounting for all cases. The goals I had
in mind when coding this were:
bq. 1) Propagate any Error without taking any additional action (eg,
closing resources, etc)
bq. 2) Use a causal chain of exceptions where needed, to traceback to
original Throwable.
bq.
bq. Note that the Throwables class is from Guava.
bq.
bq.
bq. Throwable throwable = null;
bq. try {
bq. foo_1();
bq. // txn.begin();
bq. foo_2();
bq. // txn.commit();
bq. foo_3();
bq. } catch (Throwable t1) {
bq. throwable = t1; // for finally clause to know if exception
was thrown
bq. Throwables.propagateIfInstanceOf(t1, Error.class);
bq. try {
bq. // txn.rollback();
bq. } catch (Throwable t2) {
bq. t2.initCause(t1);
bq. throwable = t2; // for finally clause
bq. throw t2;
bq. }
bq. throw t1;
bq. } finally {
bq. if (!(throwable instanceof Error)) {
bq. try {
bq. // txn.close();
bq. } catch (Throwable t) {
bq. if (throwable != null) {
bq. t.initCause(throwable);
bq. }
bq. throw t;
bq. }
bq. }
bq. }
bq.
bq. Will McQueen wrote:
bq. When I referred to OutOfMemoryException above, I meant
OutOfMemoryError.
Catching Error and Throwable are all very dangerous things todo and should be
done sparingly. It's too easy to hide serious errors. Working in OPS and DevOPS
I have seen developers mess this up far too many times.
I think we should look at JDBC, the most common transactional system, and
improve upon that.
- Brock
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4655/#review6810
-----------------------------------------------------------
On 2012-04-05 03:05:51, Brock Noland wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4655/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-05 03:05:51)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Allowing the calling of transaction.close() at any point of time.
bq.
bq.
bq. This addresses bug FLUME-1089.
bq. https://issues.apache.org/jira/browse/FLUME-1089
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
flume-ng-core/src/main/java/org/apache/flume/channel/BasicTransactionSemantics.java
403cbca
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestBasicChannelSemantics.java
80020fc
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannelTransaction.java
bc81f26
bq.
bq. Diff: https://reviews.apache.org/r/4655/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Brock
bq.
bq.
> Transaction.close should be safe to call at any point
> -----------------------------------------------------
>
> Key: FLUME-1089
> URL: https://issues.apache.org/jira/browse/FLUME-1089
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.2.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-1089-0.patch, FLUME-1089-1.patch
>
>
> We are struggling with error handling in regards to transactions. The general
> consensus is that it should be safe to call close on a transaction at any
> point.
> Discussion on flume-dev here:
> http://mail-archives.apache.org/mod_mbox/incubator-flume-dev/201203.mbox/%3CCAFukC=5X99U=aOZ5qMR_OoF=pz9f7yhl6ofkyzu08ut4or0...@mail.gmail.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
