About moving some streams text utils into a separate package: I think this
has been requested before with a filed JIRA
https://issues.apache.org/jira/browse/KAFKA-3625
Guozhang
On Tue, Aug 16, 2016 at 10:18 AM, Michael Noll wrote:
> Addendum:
>
> > Unfortunately, Apache Kafka does not publish
Addendum:
> Unfortunately, Apache Kafka does not publish these testing facilities as
maven artifacts -- that's why everyone is rolling their own.
Some testing facilities (like kafka.utils.TestUtils) are published via
maven, but other helpful testing facilities are not.
Since Radek provided a sni
Mathieu,
FWIW here are some pointers to run embedded Kafka/ZK instances for
integration testing. The second block of references below uses Curator's
TestingServer for running embedded ZK instances. See also the relevant
pom.xml for how the integration tests are being run (e.g. disabled JVM
reusa
Mathieu,
The TestUtils class is available with the following artifact:
[org.apache.kafka/kafka_2.xx "0.10.0.0" :classifier “test”]
where XX is your desired scala version.
Usage (roughly translated from clojure to scala, quotes need redioing, copy
/ paste won’t work):
val brokerCfg = TestUtils.
Hi Guozhang,
Thanks for the feedback. What would you think about
including ProcessorTopologyTestDriver in a released artifact from kafka
streams in a future release? Or alternatively, what other approach would
you recommend to incorporating it into another project's tests? I can copy
it wholesa
Hi Radek,
No, I'm not familiar with these tools. I see that Curator's TestingServer
looks pretty straight-forward, but, I'm not really sure what
kafka.util.TestUtils
is. I can't find any documentation referring to this, and it doesn't seem
to be a part of any published maven artifacts in the Kaf
Mathieu,
Your composition of Per-module Unit Tests + ProcessorTopologyTestDriver +
System Tests looks good to me, and I agree with you that since this is part
of your pre-commit process, which could be triggered concurrently from
different developers / teams, EmbeddedSingleNodeKafkaCluster +
Embed
Out of curiosity, are you aware of kafka.util.TestUtils and Apache Curator
TestingServer?
I’m using this successfully to test publis / consume scenarios with things
like Flink, Spark and custom apps.
What would stop you from taking the same approach?
–
Best regards,
Radek Gruchalski
ra...@gruchals
Hi Michael,
It would definitely be an option. I am not currently doing any testing
like that; it could replace the ProcessorTopologyTestDriver-style testing
that I'd like to do, but there are some trade-offs to consider:
- I can't do an isolated test of just the TopologyBuilder; I'd be
bri
Mathieu,
follow-up question: Are you also doing or considering integration testing
by spawning a local Kafka cluster and then reading/writing to that cluster
(often called embedded or in-memory cluster)? This approach would be in
the middle between ProcessorTopologyTestDriver (that does not spaw
10 matches
Mail list logo
