Michael, Would there be any advantage to using the kafka connect method? Seems like it'd just add an extra step of overhead?
On Thu, Oct 20, 2016 at 12:35 PM, Michael Noll <mich...@confluent.io> wrote: > Ali, > > my main feedback is similar to what Eno and Dave have already said. In > your situation, options like these are what you'd currently need to do > since you are writing directly from your Kafka Stream app to Cassandra, > rather than writing from your app to Kafka and then using Kafka Connect to > ingest into Cassandra. > > > > On Wed, Oct 19, 2016 at 11:03 PM, Ali Akhtar <ali.rac...@gmail.com> wrote: > > > Yeah, I did think to use that method, but as you said, it writes to a > dummy > > output topic, which means I'd have to put in magic code just for the > tests > > to pass (the actual code writes to cassandra and not to a dummy topic). > > > > > > On Thu, Oct 20, 2016 at 2:00 AM, Tauzell, Dave < > > dave.tauz...@surescripts.com > > > wrote: > > > > > For similar queue related tests we put the check in a loop. Check > every > > > second until either the result is found or a timeout happens. > > > > > > -Dave > > > > > > -----Original Message----- > > > From: Ali Akhtar [mailto:ali.rac...@gmail.com] > > > Sent: Wednesday, October 19, 2016 3:38 PM > > > To: users@kafka.apache.org > > > Subject: How to block tests of Kafka Streams until messages processed? > > > > > > I'm using Kafka Streams, and I'm attempting to write integration tests > > for > > > a stream processor. > > > > > > The processor listens to a topic, processes incoming messages, and > writes > > > some data to Cassandra tables. > > > > > > I'm attempting to write a test which produces some test data, and then > > > checks whether or not the expected data was written to Cassandra. > > > > > > It looks like this: > > > > > > - Step 1: Produce data in the test > > > - Step 2: Kafka stream gets triggered > > > - Step 3: Test checks whether cassandra got populated > > > > > > The problem is, Step 3 is occurring before Step 2, and as a result, the > > > test fails as it doesn't find the data in the table. > > > > > > I've resolved this by adding a Thread.sleep(2000) call after Step 1, > > which > > > ensures that Step 2 gets triggered before Step 3. > > > > > > However, I'm wondering if there's a more reliable way of blocking the > > test > > > until Kafka stream processor gets triggered? > > > > > > At the moment, I'm using 1 thread for the processor. If I increase that > > to > > > 2 threads, will that achieve what I want? > > > This e-mail and any files transmitted with it are confidential, may > > > contain sensitive information, and are intended solely for the use of > the > > > individual or entity to whom they are addressed. If you have received > > this > > > e-mail in error, please notify the sender by reply e-mail immediately > and > > > destroy all copies of the e-mail and any attachments. > > > > > >
