Beside state and DSL Kafka Stream also allow for fully fault-tolerant, salable, and elastic deployment.
-Matthias On 3/13/18 10:55 AM, Hans Jespersen wrote: > "If your system is stateless and the transformations are not interdependent" > then I would just look at using Kafka Connect's Single Message Transform > (SMT) feature. > > -hans > > /** > * Hans Jespersen, Director Systems Engineering, Confluent Inc. > * h...@confluent.io (650)924-2670 > */ > > On Tue, Mar 13, 2018 at 9:18 AM, Jacob Sheck <shec0...@gmail.com> wrote: > >> If you are augmenting streaming data with dimensional data, or if you can >> transform your data with a map, filter or join operation Streams will be a >> good option. If your system is stateless and the transformations are not >> interdependent, you may want to look into using one of the queue >> technologies like AcitveMQ. >> >> On Tue, Mar 13, 2018 at 10:00 AM Sameer Rahmani <lxsame...@gmail.com> >> wrote: >> >>> Thanks Jacob. I'm using kafka as a distributed queue and my data pipeline >>> is a several components which connected together via a stream >> abstraction. >>> each component has a input and output stream. Basically the source of >>> this pipeline can be anything and the output can be anything to. >>> >>> On Tue, Mar 13, 2018 at 1:27 PM, Jacob Sheck <shec0...@gmail.com> wrote: >>> >>>> Sameer when you say that you need to "consume from and produce to a >>> topic" >>>> to me that seems like a good fit for Kafka Streams. Streaming your >> data >>>> out of Kafka for a transform and back in has some fundamental costs and >>>> operational challenges involved. Are the events in your stream >>> stateless? >>>> If it isn't stateless streams will ensure a consistent playback of >> events >>>> if needed. Without knowing more about your pipeline it is hard to make >>>> recommendations. Are you possibly using Kafka as a distributed queue? >>>> >>>> On Tue, Mar 13, 2018 at 6:29 AM Sameer Rahmani <lxsame...@gmail.com> >>>> wrote: >>>> >>>>> Hi folks, >>>>> I need to consume from and produce to a topic. I have my own data >>>> pipeline >>>>> to process the data. >>>>> So I was wondering beside the stores and StreamDSL what does Kafka >>>> Streams >>>>> brings to the table >>>>> that might be useful to me ? >>>>> >>>> >>> >> >
signature.asc
Description: OpenPGP digital signature
