[
https://issues.apache.org/jira/browse/FLUME-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266151#comment-13266151
]
[email protected] commented on FLUME-1121:
------------------------------------------------------
bq. On 2012-04-30 19:22:02, Arvind Prabhakar wrote:
bq. > Thanks for the patch Brock, changes look good. Some minor feedback
follows.
bq. >
bq. > Please rebase the patch when convenient and attach to the Jira.
Thank you for your review! Attached to the JIRA!
bq. On 2012-04-30 19:22:02, Arvind Prabhakar wrote:
bq. >
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/RecoverableMemoryChannel.java,
line 123
bq. > <https://reviews.apache.org/r/4713/diff/3/?file=101674#file101674line123>
bq. >
bq. > nit:tab
done
bq. On 2012-04-30 19:22:02, Arvind Prabhakar wrote:
bq. >
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/RecoverableMemoryChannel.java,
line 82
bq. > <https://reviews.apache.org/r/4713/diff/3/?file=101674#file101674line82>
bq. >
bq. > nit: this seems to be duplicating part of the functionality offered
by the LifecycleState flag maintained by the abstract base class. It will be
good to see if that can be reused here.
I agree, I have created FLUME-1169 to implement this checking in the superclass.
bq. On 2012-04-30 19:22:02, Arvind Prabhakar wrote:
bq. >
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/RecoverableMemoryChannel.java,
lines 144-148
bq. > <https://reviews.apache.org/r/4713/diff/3/?file=101674#file101674line144>
bq. >
bq. > This call needs to follow the transaction idiom with the appropriate
try/catch. Also, since this is initialization time code, it may be easier to do
a bulk put in a single transaction.
Done
bq. On 2012-04-30 19:22:02, Arvind Prabhakar wrote:
bq. >
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/wal/WAL.java,
lines 92-93
bq. > <https://reviews.apache.org/r/4713/diff/3/?file=101676#file101676line92>
bq. >
bq. > Since these are getting accessed by worker thread and no longer
propagated, they should be marked volatile? (or propagated).
I figured eventual consistency for this parameters was fine, however I have
marked them volatile.
- Brock
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4713/#review7380
-----------------------------------------------------------
On 2012-04-13 20:12:25, Brock Noland wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4713/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-13 20:12:25)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Channel.start() is not being called. This is fixed in
DefaultLogicalNodeManager.
bq.
bq. Additionally RecoverableMemoryChannel now tracks it's own capacity due to
the MemoryChannel semantics being completely different. Basically, if we rely
on MemoryChannel capacity, then an error will be thrown when we commit the
MemoryChannelTransaction. However, we will have already committed this data to
disk. If we commit to MemoryChannel first (there by checking capacity) we could
fail to write to disk resulting in data which is only in memory.
bq.
bq. Also, I ran cleanup on modules touched. This removes whitespace, unused
imports, and adds @Override tags where needed. This is one time cleanup which
allows automated cleanup in the future.
bq.
bq.
bq. This addresses bug FLUME-1121.
bq. https://issues.apache.org/jira/browse/FLUME-1121
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelUtils.java
1421449
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannelConcurrency.java
8dad0b2
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannelTransaction.java
bc81f26
bq.
flume-ng-node/src/main/java/org/apache/flume/conf/properties/FlumeConfiguration.java
d66f6d1
bq.
flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java
1f0e8c6
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/TestBasicChannelSemantics.java
80020fc
bq.
flume-ng-core/src/test/java/org/apache/flume/channel/AbstractBasicChannelSemanticsTest.java
6e71e46
bq. flume-ng-core/src/main/java/org/apache/flume/source/SyslogUDPSource.java
732cce5
bq. flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java ca5212e
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkType.java 6b08c09
bq. flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java
b0485b1
bq.
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/wal/WALDataFile.java
9d4a1fd
bq.
flume-ng-channels/flume-recoverable-memory-channel/src/test/java/org/apache/flume/channel/recoverable/memory/TestRecoverableMemoryChannel.java
edd8a8b
bq. flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java
be1891b
bq. flume-ng-core/src/main/java/org/apache/flume/conf/Configurables.java
84492e5
bq.
flume-ng-core/src/main/java/org/apache/flume/formatter/output/BucketPath.java
4722819
bq.
flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleAware.java
f179de0
bq. flume-ng-core/src/main/java/org/apache/flume/sink/AbstractSink.java
2334059
bq.
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/wal/WAL.java
97ef796
bq.
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/wal/SequenceIDBuffer.java
fa63b73
bq.
flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/RecoverableMemoryChannel.java
0622f27
bq.
flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java
07c3d0b
bq.
flume-ng-sdk/src/test/java/org/apache/flume/api/TestFailoverRpcClient.java
225cd34
bq.
flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java
0b8a2c0
bq.
bq. Diff: https://reviews.apache.org/r/4713/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. All unit tests pass and manual testing passes as well.
bq.
bq.
bq. Thanks,
bq.
bq. Brock
bq.
bq.
> Recoverable Memory Channel cannot recover data
> -----------------------------------------------
>
> Key: FLUME-1121
> URL: https://issues.apache.org/jira/browse/FLUME-1121
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.1.0, v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Brock Noland
> Attachments: FLUME-1121-2.patch
>
>
> Run an agent with a source, like seq with a recoverable memory channel and no
> sink. kill the process with ctrl-C. Run an agent now with no source, but
> logger sink and the same recoverable memory channel. I expected the events
> generated by the source earlier should be written to the sink, but no events
> with picked up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
