GitHub user jfenc91 opened a pull request:
https://github.com/apache/storm/pull/1679
storm-kafka-client tests: tuples not being properly replayed
I have been working with kafka 10 and storm 1.0.2 using the
storm-kafka-client in master and am having issues with topologies that fail
tuples. The kafka offsets seemed to get stuck and the kafka spout eventually
halted even though the last committed offset was no where near the end of the
queue.
Here are a few unit tests that I believe replicate my situation. The last 2
are currently failing which is reflective of what I am seeing in my topologies.
Let me know if I missed anything! This seems like a pretty big oversight, so I
am getting the feeling that something in the test is wrong. Thanks!
I added tests for the following cases:
- All tuples being acked.
- A tuple being failed in order it was emitted
- A tuple being failed out of the order it was emitted
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jfenc91/storm stormKafkaClientTests
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1679.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1679
----
commit 94da00023dceb3a9281c536f92c052e4e969835b
Author: Jeff Fenchel <[email protected]>
Date: 2016-09-11T01:44:35Z
Added unit tests to storm-kafka-client to cover reading from a single
topic with:
- All tuples being acked.
- A tuple being failed in order it was emitted
- A tuple being failed out of the order it was emitted
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
