[
https://issues.apache.org/jira/browse/FLUME-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289157#comment-13289157
]
Hari Shreedharan edited comment on FLUME-1255 at 6/5/12 5:18 AM:
-----------------------------------------------------------------
Hi Juhani,
Yes, it is more of a config error. The issue here is that because take throws
exceptions, the HbaseSink ends up throwing an exception and break out of
process(), *without* committing or rolling back the transaction(and thus
closing it). Due to this, the next time txn.begin() is called, an exception is
thrown because the txn was never closed. Even for sinks like HDFSEventSink,
which catches the exception, it is problematic because it rolls back the
txn(This is probably the right thing to do, since a channel throwing an
exception indicates an error condition, and the sink does not need to care what
error it was - and reaching txn capacity is certainly not an error - so we
should not throw an exception in that case) - which means the sink will never
actually commit a transaction. This is primarily a correctness issue, not a
user-friendliness one.
There is no reason we should fail with an exception if the transaction capacity
of the channel is smaller than the batch size of the sink - since such a
configuration is absolutely valid.
was (Author: hshreedharan):
Hi Juhani,
Yes, it is more of a config error. The issue here is that because take throws
exceptions, our sinks end up throwing an exception and break out of process(),
*without* committing or rolling back the transaction(and thus closing it). Due
to this, the next time txn.begin() is called, an exception is thrown because
the txn was never closed. Even for sinks like HDFSEventSink, which catches the
exception, it is problematic because it rolls back the txn(This is probably the
right thing to do, since a channel throwing an exception indicates an error
condition, and the sink does not need to care what error it was - and reaching
txn capacity is certainly not an error - so we should not throw an exception in
that case) - which means the sink will never actually commit a transaction.
This is primarily a correctness issue, not a user-friendliness one.
There is no reason we should fail with an exception if the transaction capacity
of the channel is smaller than the batch size of the sink - since such a
configuration is absolutely valid.
> Channels should not throw exceptions in channel.take if it has reached txn
> capacity
> -----------------------------------------------------------------------------------
>
> Key: FLUME-1255
> URL: https://issues.apache.org/jira/browse/FLUME-1255
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
>
> Channels like Memory channel throw exception when reaching txn capacity. If
> sink's batchsize is more than txn capacity the take function throws exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
