Re: GlobalKTable with RocksDB - queries before state RUNNING?

2023-11-21 Thread Sophie Blee-Goldman
Just to make sure I understand the logs, you're saying the "new file processed" lines represent store queries, and presumably the com.osr.serKafkaStreamsService is your service that's issuing these queries? You need to wait for the app to finish restoring state before querying it. Based on this me

GlobalKTable with RocksDB - queries before state RUNNING?

2023-11-21 Thread Christian Zuegner
Hi, we have the following problem - a Kafka Topic ~20Megabytes is made available as GlobalKTable for queries. With using RocksDB the GKTable is ready for queries instantly even without having reading the data complete - all get() requests return null. After a few seconds the data is querieable

Change super.users at runtime

2023-11-21 Thread Artem Timchenko
Hi, I'm trying to change super.users config at runtime, without broker restart. The next command is used (I use the same approach for all brokers, but posting output only from one broker here not to overload): > /opt/confluent/bin/kafka-configs --bootstrap-server localhost:9093 > --command-config

RE: Messages streaming from topic to topic

2023-11-21 Thread Alexander Shapiro (ashapiro)
As I know, MM2 cannot sync topics on same cluster as topics on source and target supposed to have same name or to come with prefix.sameName plz correct me, there is no topicA -> topicB replication -Original Message- From: Anders Engström Sent: Tuesday, November 21, 2023 12:58 To: users@

Re: Messages streaming from topic to topic

2023-11-21 Thread megh vidani
Okay. For us both of them are working well for the use case mentioned by you. Thanks, Megh On Tue, Nov 21, 2023, 16:25 Alexander Shapiro (ashapiro) wrote: > Thanks for instant reply > > Yes, I did > > I have checked till now the below: > Apache Storm > Apache Heron > Apache Samza > Apache Spark

Re: Messages streaming from topic to topic

2023-11-21 Thread Anders Engström
Mirror Maker (2) might be a good solution.. It's "mostly" Kafka Connect, so it should be possible to do filtering etc. https://developers.redhat.com/articles/2023/11/13/demystifying-kafka-mirrormaker-2-use-cases-and-architecture#mirrormaker_2_internal_topics /Anders On Tue, Nov 21, 2023 at 11:55 A

RE: Messages streaming from topic to topic

2023-11-21 Thread Alexander Shapiro (ashapiro)
Thanks for instant reply Yes, I did I have checked till now the below: Apache Storm Apache Heron Apache Samza Apache Spark Apache Flink Apache NiFi Confluent replicator StreamSets Data collector Envoy Filters -Original Message- From: megh vidani Sent: Tuesday, November 21, 2023 12:

Re: Messages streaming from topic to topic

2023-11-21 Thread megh vidani
Hi Alexander, Have you explored Spark or Flink? Thanks, Megh On Tue, Nov 21, 2023, 15:50 Alexander Shapiro (ashapiro) wrote: > Hi team! > > > I am looking for an open-source Tool with capability to stream Kafka > message from topic A to topic B with bellow conditions: > a. A an

Messages streaming from topic to topic

2023-11-21 Thread Alexander Shapiro (ashapiro)
Hi team! I am looking for an open-source Tool with capability to stream Kafka message from topic A to topic B with bellow conditions: a. A and B can be on same cluster. b. A and B can be on different clusters. c. Aggregation of messages

[VOTE] 3.5.2 RC1

2023-11-21 Thread Luke Chen
Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.5.2. This is a bugfix release with several fixes since the release of 3.5.1, including dependency version bumps for CVEs. Release notes for the 3.5.2 release: https://home.apache.org/~s