Hi Mads, Great question and yes your use case here is an excellent fit for Kafka Streams and Kafka Connect.
For step 2 you could use a KStram#flatMap operation to split it up into multiple rows. Regarding a Cassandra connector, there is an existing one: 1. For some background try https://www.confluent.io/blog/kafka-connect-cassandra-sink-the-perfect-match/ 2. To download the connector go to https://www.confluent.io/product/connectors/ As for writing directly to Cassandra, you *could*, in theory, do so with a KStream.process or KStream.foreach call. But you'd have to code that yourself which includes error handling, retry logic etc. Additionally, it's usually not recommended to write out directly to external systems from Kafka Streams. I'd say it's better to leverage Kafka Connect for that. HTH, Bill On Thu, Dec 21, 2017 at 5:49 AM, Mads Tandrup < mads.tand...@schneider-electric.com> wrote: > Hi > > Sorry for the simple question. I’m just starting to learn about Kafka > streams and connect and I’m struggling to understand the exact difference > and which one to use. I’m coming from Apache Storm so forgive me if I make > false assumptions. > > I have a use case where I have a Kafka topic with some messages. What I > need to do: > 1. Read the messages > 2. Split and map the message into a number of rows > 3. Write the rows to Cassandra > > It seems the first 2 steps are a natural fit for Kafka Streams. But it > seems the way to write to Cassandra is to use Kafka Connect. > Is that correctly understood? > > Is there any way to connect Kafka Streams and Kafka Connect without > writing it to a new kafka topic? Since the transformation in step 2 is so > simple it seems a waste to write it to disk. > > Is there any other way I should consider? > > Best regards, > Mads > >
