Streams is not designed to be run inside Connect, and this won't work. What you can do is, to import the data via connect into a "staging topic" and then read this "staging topic" with a Kafka Streams application and apply the transformations etc to write the data into the actual target topics.
-Matthias On 10/25/18 2:34 PM, RK Sing wrote: > Thank you Ryanne for the answer, My question is can I run Streams app > inside Connect, what I mean is since Streams is a library, can I build a > custom Source connector and use Streams in there . or using Streams in > transforms?? > We want to containerize Kafka connect as in wrap it in a docker, and since > all the Kafka-connect configs are stored in Kafka itself, so the container > can be stateless, we can spin up containers as and when needed, we want > this approach to fetch data from microservices and golden gate oracle. etc. > > > > > On Thu, Oct 25, 2018 at 2:24 PM Ryanne Dolan <ryannedo...@gmail.com> wrote: > >> Dhurandar, definitely! Connect and Streams are both agnostic to how their >> workers are run. They aren't really platforms per se. You just need to spin >> up one or more workers and they do their thing. So a Streams app doesn't >> run "inside" Connect, but you can certainly have Connect and Streams >> workers talk to each other through Kafka topics. >> >> Ryanne >> >> On Thu, Oct 25, 2018 at 3:34 PM RK Sing <dhurandarg...@gmail.com> wrote: >> >>> We have a requirement to do Single row transformations, basic joins, >>> deduping and routing from the source to the destination Kafka topics. >>> >>> We want to use Kafka-connect as the platform which is running Kafka >> stream >>> inside. Has anyone used Kafkastreams inside Kafka connect ?? >>> >>> Is this pattern ok for Kafka-connect? >>> >>> regards >>> dhurandar >>> >> >
signature.asc
Description: OpenPGP digital signature
