GitHub user srdo opened a pull request:
https://github.com/apache/storm/pull/2766
STORM-2972: Replace storm-kafka with storm-kafka-client in storm-sql-â¦
â¦kafka
https://issues.apache.org/jira/browse/STORM-2972
The new kafka scheme syntax is stolen from [Apache
Camel](https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc),
since I figure we are doing something very similar to them.
The primary limitation here compared to the storm-kafka version is that you
can't include metadata (e.g. partition) in the emitted tuple, because I had to
adapt the Scheme API to the RecordTranslator API used in storm-kafka-client.
Emitting metadata with a Scheme in the storm-kafka spout is handled by [special
case instanceof'ing the
scheme](https://github.com/apache/storm/blob/af42f434f4a4c3d9087c6058b359033736d3b5e8/external/storm-kafka/src/jvm/org/apache/storm/kafka/KafkaUtils.java#L134)
to see if it fits one that takes metadata, and I'd really prefer not to do
that.
If we need to be able to emit metadata as well, we can just allow people to
override the RecordTranslatorSchemeAdapter. I'll leave that as a task for
later, unless someone objects.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srdo/storm STORM-2972
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2766.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2766
----
commit 7f992a6b7474267691e48fd2e5a64230cda41390
Author: Stig Rohde Døssing <srdo@...>
Date: 2018-07-15T15:30:41Z
STORM-2972: Replace storm-kafka with storm-kafka-client in storm-sql-kafka
----
---
