How is decompress called in DefaultRecordBatch#iterator

Hi all， There are huge latency gap between arm and x86 when running kafka-producer-perf-test.sh with compression set, especially when the codec is zstd. So I looked into the source code. It seems the response from broker server is too slow for arm. And the

Re: Broker always out of ISRs

I’ve had the same experience as Liam with this symptom (all followers on a single broker of a given leader getting stuck). It sounds likely that either the replica fetcher thread is getting stuck or dying with an unhandled exception. The the former case, jstack output can be helpful to understand

Re: Get after put in stateStore returns null

You can try to check the file which rocksdb creates for the state store to see if value is stored for the key. What I suspect is that get may be getting called before put in your topology. Try retrieving the value from state store further downstream. Thanks Sachin On Thu, 2 Apr 2020, 02:35

Re: Broker always out of ISRs

Hi Zach, If you check the cluster's controller's controller.log, do you see broker 2 bouncing in and out of ISRs? There'll be logs to that effect. Or is it just never getting in-sync in the first place? Whenever I've had this issue in the past, it's been because the replica fetcher has died.

Re: Get after put in stateStore returns null

That is expected behavior. And yes, there is a `Transformer` instance per partition with it's own store that holds one shard of the overall state. The reason is, that you could run one KafkaStreams instance per partition on different hosts/servers and thus, we need to have a `Transformer` and

Re: Get after put in stateStore returns null

Ok, Matthias, thanks for the hint: *Even if any upstream operation was key-changing, no auto-repartition is triggered. If repartitioning is required, a call to through() should be

Re: Broker always out of ISRs

Hi Liam, > Any issues with partitions broker 2 is leader of? > Earlier today, broker 2 was not leader of any partitions. At that time, 2 appeared to be in ISRs of all partitions where 1 was leader, but 2 was not in any ISRs of partitions where 0 was leader. Currently, broker 2 is leader of 55

Re: Statestore restoration - Error while range compacting during restoring

Added some nasty code in kafka 2.4.1. Seems to work fine for now... From my understanding, the compaction process when restoring a store is only done to speed up things. So I guess this kind of "hack" isn't such a big deal ? [image: image.png] Le mer. 1 avr. 2020 à 10:44, Nicolas Carlot a écrit

Re: Statestore restoration - Error while range compacting during restoring

Here is the full configuration of Rocks. [image: image.png] Le mer. 1 avr. 2020 à 10:41, Nicolas Carlot a écrit : > It's not that I cannot turn on compaction. > Compaction works fine. > The issue is with the restoration process of the state store, which tries > to compact the store to a single

Re: Statestore restoration - Error while range compacting during restoring

It's not that I cannot turn on compaction. Compaction works fine. The issue is with the restoration process of the state store, which tries to compact the store to a single level: db.compactRange(columnFamily, true, 1, 0) before bulk loading the data. It automatically fails when I'm using FIFO

Re: Statestore restoration - Error while range compacting during restoring

My current workaround is to completely delete the state store and rebuild it from scratch. Le mar. 31 mars 2020 à 21:39, Boyang Chen a écrit : > Thanks Nicolas for the report, so are you suggesting that you couldn't turn > on compactions for the state store? Is there a workaround? > > On Tue,