Re: Re-key by multiple properties without composite key

2024-02-07 Thread Karsten Stöckmann
Matthias, thank you for getting back on this. - I'll just let the application run for now. We are trying to build a CDC system leveraging Kafka in order to feed aggregated data into an indexing solution like Elasticsearch or Opensearch. Maybe after all the Kafka Streams application will be

Re: Re-key by multiple properties without composite key

2024-02-07 Thread Matthias J. Sax
Using the DSL, this sounds about right. I am not worried about the complexity -- KS can handle it, and it's not uncommon to end up with such topologies. You might be able to cut down on complexity by not using the DSL, but the Processor API. It gives you more control, and thus you might be

Re: Re-key by multiple properties without composite key

2024-02-07 Thread Karsten Stöckmann
Sorry for being late with the response - I've been quite busy working on our Streams application lately. That leads me back to my initial question. The Folder class contains multiple fields with FK pointing to the Person table, all of them with different semantics (customer, billing address,

Re: [DISCUSS] KIP-1010: Topic Partition Quota

2024-02-07 Thread Viktor Somogyi-Vass
Hi Afshin, We keep KIP discussions on d...@kafka.apache.org so please post this over there too. I'll go over this later this week but devs usually monitor that list more frequently and you'll have better chances of getting a reply there. Regards, Viktor On Wed, Jan 17, 2024 at 12:03 AM Afshin

Regarding the solution implemented is correct or wrong ?

2024-02-07 Thread Vinay Reddy Pannala
Hi Team, I am working on a story where I have to send the records from the aggregation processor along with uuid, I have a deque topic, and adding the records to the deque topic and forwarding it from the Main Aggregate class. But when we do perf testing the TPS is coming around 1200/1300 but

Regarding the solution implemented is correct or wrong ?

2024-02-07 Thread Vinay Reddy Pannala
Hi Team, I am working on a story where I have to send the records from the aggregation processor along with uuid, I have a deque topic, and adding the records to the deque topic and forwarding it from the Main Aggregate class. But when we do perf testing the TPS is coming around 1200/1300 but