Re: does Kafka exactly-once guarantee also apply to kafka state stores?

2021-01-04 Thread Ning Zhang
If there is a "change-log" topic to back up the state store, then it may not lose data. Also, if the third party store is not "kafka community certified" (or not well-maintained), it may have chances to lose data (in different ways). On 2021/01/05 04:56:12, Pushkar Deole wrote: > In case w

Re: does Kafka exactly-once guarantee also apply to kafka state stores?

2021-01-04 Thread Pushkar Deole
In case we opt to choose some third party store instead of kafka's stores for storing state (e.g. Redis cache or Ignite), then will we lose the exactly-once guarantee provided by kafka and the state stores can be in an inconsistent state ? On Sat, Jan 2, 2021 at 4:56 AM Ning Zhang wrote: > The p

Offset commit consistently timing out in transaction

2021-01-04 Thread Kindernay Oliver
Hello, we are experiencing problems with offset commit timing out on brokers. This started to happen when we started using transactions. send_offsets_to_transactions periodically (every few minutes) times out. We have cluster of three brokers, and topics with 15 partitions. We use one transact