[
https://issues.apache.org/jira/browse/KAFKA-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320605#comment-16320605
]
Scott Davis edited comment on KAFKA-3625 at 1/10/18 4:45 PM:
-------------------------------------------------------------
FYI I just setup some unit tests using the (not documented) internal test
classes in Kafka Streams, and I thought I'd share some thoughts on my
experience with it.
First, it's super-helpful! The high-level DSL style of Kafka Streams
applications doesn't fit well into standard unit testing frameworks, and this
solves that problem. It's also helpful compared with integration testing in the
sense that it isn't necessary to produce source messages on a broker, wait for
timeouts, etc.
The internal test classes in 1.0.0 require the unit tests to provide
serializers. This can create some extra boilerplate work to configure the
serializers (especially the Confluent AVRO Serde, which requires a schema
registry). However, I only need to unit test my application logic (i.e. the
contents of the "filter" and "map" methods, etc). I can see how testing
serializers is a requirement for testing Kafka Streams internally, but it isn't
a requirement for testing the logic of Kafka Streams applications. Note: To
work around this, I created a "JavaObjectSerde", which uses
java.io.Object(In|Out)putStream, for use by the unit tests.
I needed about 10 lines of boilerplate code in an @Before method to setup the
ProcessorTopologyTestDriver, which seemed slightly excessive but not
burdensome. Most of it was to create the serializers and the StreamsConfig,
which aren't part of my application logic but were required to create the test
driver.
I was able to get it to work by adding
{{org.apache.kafka:kafka-streams:1.0.0:test}} to my build.gradle, so it seems
there already is an artifact. However, there is no documentation, which seems
to me like the biggest drawback. I learned it by reading the Kafka Streams
source code.
was (Author: scott.davis):
FYI I just setup some unit tests using the (not documented) internal test
classes in Kafka Streams, and I thought I'd share some thoughts on my
experience with it.
First, it's super-helpful! The high-level DSL style of Kafka Streams
applications doesn't fit well into standard unit testing frameworks, and this
solves that problem. It's also helpful compared with integration testing in the
sense that it isn't necessary to produce source messages on a broker, wait for
timeouts, etc.
The internal test classes in 1.0.0 require the unit tests to provide
serializers. This can create some extra boilerplate work to configure the
serializers (especially the Confluent AVRO Serde, which requires a schema
registry). However, I only need to unit test my application logic (i.e. the
contents of the "filter" and "map" methods, etc). I can see how testing
serializers is a requirement for testing Kafka Streams internally, but it isn't
a requirement for testing the logic of Kafka Streams applications. Note: To
work around this, I created a "JavaObjectSerde", which uses
java.io.Object(In|Out)putStream, for use by the unit tests.
I needed about 10 lines of boilerplate code in an @Before method to setup the
ProcessorTopologyTestDriver, which seemed slightly excessive but not
burdensome. Most of it was to create the serializers and the StreamsConfig,
which aren't part of my application logic but were required to create the test
driver.
> Move kafka-streams test fixtures into a published package
> ---------------------------------------------------------
>
> Key: KAFKA-3625
> URL: https://issues.apache.org/jira/browse/KAFKA-3625
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Jeff Klukas
> Assignee: Matthias J. Sax
> Priority: Minor
> Labels: needs-kip, user-experience
>
> The KStreamTestDriver and related fixtures defined in
> streams/src/test/java/org/apache/kafka/test would be useful to developers
> building applications on top of Kafka Streams, but they are not currently
> exposed in a package.
> I propose moving this directory to live under streams/fixtures/src/main and
> creating a new 'streams:fixtures' project in the gradle configuration to
> publish these as a separate package.
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-247%3A+Add+public+test+utils+for+Kafka+Streams
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
