It's the org.apache.kafka.streams.integration.utils.EmbeddedKafkaCluster (start ZK and calls org.apache.kafka.streams.integration.utils.KafkaEmbedded to start Kafka). So these are embedded in the sense that it's not another process, just threads within the main streams test process.
Thanks Eno > On 11 Nov 2016, at 16:26, Ali Akhtar <ali.rac...@gmail.com> wrote: > > Hey Eno, > > Thanks for the quick reply. > > In the meantime, is it possible to just send a sigterm / kill -9 which just > kills the zookeeper + kafka? I can figure out how to do it if you can point > out which class / method creates the processes / threads. > > Thanks. > > On Fri, Nov 11, 2016 at 9:24 PM, Eno Thereska <eno.there...@gmail.com> > wrote: > >> Hi Ali, >> >> You're right, shutting down the broker and ZK is expensive. We kept the >> number of integration tests relatively small (and pushed some more tests as >> system tests, while doing as many as possible as unit tests). It's not just >> the shutdown that's expensive, it's also the starting up unfortunately. >> It's on our todo list to do something about this, but we haven't gotten >> there yet. If someone from the community wants to have a look and help out, >> that'd be great (with a JIRA and PR). >> >> About the second problem with ZK logs, this is being worked on as part of >> removing the ZK dependency from streams and should be merged shortly: >> https://github.com/apache/kafka/pull/1884 <https://github.com/apache/ >> kafka/pull/1884>. The msg you see does not affect correctness, it's just >> annoying and it will go away. >> >> Thanks, >> Eno >> >> >>> On 11 Nov 2016, at 14:28, Ali Akhtar <ali.rac...@gmail.com> wrote: >>> >>> I have some unit tests in which I create an embedded single broker kafka >>> cluster, using : >>> >>> EmbeddedSingleNodeKafkaCluster.java from >>> https://github.com/confluentinc/examples/blob/ >> master/kafka-streams/src/test/java/io/confluent/examples/streams/kafka/ >> EmbeddedSingleNodeKafkaCluster.java >>> >>> That class also creates an embedded zookeeper cluster / instance. >>> >>> The problem is, while the tests run pretty fast and pass, they then stay >>> stuck in the 'teardown / clean up' stage for a really long time, often >> upto >>> 10-20 >>> seconds per test. >>> >>> As I have a lot of test classes, each class creating its own embedded >> kafka >>> cluster, this time can really add up during compiles. >>> >>> Is it possible to get these test classes to not do any clean up / safety >>> stuff, because the instances are just throwaway. Just have them kill -9 >> the >>> kafka / zookeeper and exit? >>> >>> It doesn't make any sense that tests pass within seconds, but can't move >> on >>> to the next test class because its cleaning up. >>> >>> I also have an embedded cassandra instance in these tests, but I don't >>> think that one is the problem, as i see a lot of zookeeper logs such as >>> these after the test runs: >>> >>> 133764 [main-SendThread(127.0.0.1:38846)] WARN >>> org.apache.zookeeper.ClientCnxn - Session 0x15853c3497f0001 for server >>> null, unexpected error, closing socket connection and attempting >> reconnect >>> java.net.ConnectException: Connection refused >>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >>> at sun.nio.ch.SocketChannelImpl.finishConnect( >> SocketChannelImpl.java:717) >>> at >>> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport( >> ClientCnxnSocketNIO.java:361) >>> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) >>> >>> >>> Could it be that zookeeper doesn't exit and keeps retrying to connect? >> >>
