RE: Messages streaming from topic to topic
Thanks for instant reply Yes, I did I have checked till now the below: Apache Storm Apache Heron Apache Samza Apache Spark Apache Flink Apache NiFi Confluent replicator StreamSets Data collector Envoy Filters -Original Message- From: megh vidani Sent: Tuesday, November 21, 2023 12:46 To: us...@kafka.apache.org Cc: dev ; kafka-clients Subject: Re: Messages streaming from topic to topic [You don't often get email from vidanimeg...@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] CAUTION: This email is from an external source. Please don't open any unknown links or attachments. Hi Alexander, Have you explored Spark or Flink? Thanks, Megh On Tue, Nov 21, 2023, 15:50 Alexander Shapiro (ashapiro) wrote: > Hi team! > > > I am looking for an open-source Tool with capability to stream Kafka > message from topic A to topic B with bellow conditions: > a. A and B can be on same cluster. > b. A and B can be on different clusters. > c. Aggregation of messages > d. Filtration of messages > e. Customization options for payload manipulation > before sending to topic B - preferable java. > f. Optional - streaming message to other technologies > > For example, > such capabilities can be implemented with "StreamSets Data Collector", > but this tool looks like overkill for my needs This email and the > information contained herein is proprietary and confidential and > subject to the Amdocs Email Terms of Service, which you may review at > https://www.amdocs.com/about/email-terms-of-service < > https://www.amdocs.com/about/email-terms-of-service> > > This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service <https://www.amdocs.com/about/email-terms-of-service>
Re: Messages streaming from topic to topic
Hi Alexander, Have you explored Spark or Flink? Thanks, Megh On Tue, Nov 21, 2023, 15:50 Alexander Shapiro (ashapiro) wrote: > Hi team! > > > I am looking for an open-source Tool with capability to stream Kafka > message from topic A to topic B with bellow conditions: > a. A and B can be on same cluster. > b. A and B can be on different clusters. > c. Aggregation of messages > d. Filtration of messages > e. Customization options for payload manipulation before > sending to topic B - preferable java. > f. Optional - streaming message to other technologies > > For example, > such capabilities can be implemented with "StreamSets Data Collector", but > this tool looks like overkill for my needs > This email and the information contained herein is proprietary and > confidential and subject to the Amdocs Email Terms of Service, which you > may review at https://www.amdocs.com/about/email-terms-of-service < > https://www.amdocs.com/about/email-terms-of-service> > >
Re: Messages streaming from topic to topic
Okay. For us both of them are working well for the use case mentioned by you. Thanks, Megh On Tue, Nov 21, 2023, 16:25 Alexander Shapiro (ashapiro) wrote: > Thanks for instant reply > > Yes, I did > > I have checked till now the below: > Apache Storm > Apache Heron > Apache Samza > Apache Spark > Apache Flink > Apache NiFi > Confluent replicator > StreamSets Data collector > Envoy Filters > > > > > > -Original Message- > From: megh vidani > Sent: Tuesday, November 21, 2023 12:46 > To: us...@kafka.apache.org > Cc: dev ; kafka-clients < > kafka-clie...@googlegroups.com> > Subject: Re: Messages streaming from topic to topic > > [You don't often get email from vidanimeg...@gmail.com. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > CAUTION: This email is from an external source. Please don't open any > unknown links or attachments. > > Hi Alexander, > > Have you explored Spark or Flink? > > Thanks, > Megh > > On Tue, Nov 21, 2023, 15:50 Alexander Shapiro (ashapiro) < > alexander.shap...@amdocs.com.invalid> wrote: > > > Hi team! > > > > > > I am looking for an open-source Tool with capability to stream Kafka > > message from topic A to topic B with bellow conditions: > > a. A and B can be on same cluster. > > b. A and B can be on different clusters. > > c. Aggregation of messages > > d. Filtration of messages > > e. Customization options for payload manipulation > > before sending to topic B - preferable java. > > f. Optional - streaming message to other technologies > > > > For example, > > such capabilities can be implemented with "StreamSets Data Collector", > > but this tool looks like overkill for my needs This email and the > > information contained herein is proprietary and confidential and > > subject to the Amdocs Email Terms of Service, which you may review at > > https://www.amdocs.com/about/email-terms-of-service < > > https://www.amdocs.com/about/email-terms-of-service> > > > > > This email and the information contained herein is proprietary and > confidential and subject to the Amdocs Email Terms of Service, which you > may review at https://www.amdocs.com/about/email-terms-of-service < > https://www.amdocs.com/about/email-terms-of-service> > >
