Hi Chris We are using them like you described. Performance is very good compared to the database used before. Beware that until https://issues.apache.org/jira/browse/KAFKA-7380 is done the startup will be blocked until all global stores are restored (sequentially). This can take a little for larger topic and/or multiple global stores.
We are blocking access until they are available although this is not ideal in terms of timeout tuning. Any ideas are welcome. Best regards Patrik > Am 14.11.2018 um 00:17 schrieb Chris Toomey <ctoo...@gmail.com>: > > We're considering using GlobalKTables / KeyValueStores for locally caching > topic content in services. The topics would be compacted such that only the > latest key/value pair would exist for a given key. > > One question that's come up is how to determine, when bootstrapping the > app, when the cache has been populated with the latest content from the > topic (so we start with a "warm" cache). ReadOnlyKeyValueStore has > an approximateNumEntries() method that we could use to see how much we've > got, but trying to figure out how much there is in the topic looks much > more difficult -- the only way I can see via the APIs / code is to use an > AdminClient to get the topic partitions and then the KafkaConsumer to get > the end offsets for those. > > Does anyone have experience doing this kind of caching? How did you handle > the bootstrapping issue? > > Any thoughts on easier or better ways to determine when the cache is warm? > > thx, > Chris
