Thanks Liam! On Tue, May 12, 2020, 15:12 Liam Clarke-Hutchinson < liam.cla...@adscale.co.nz> wrote:
> Hi Pushkar, > > GlobalKTables and KTables can have whatever data structure you like, if you > provide the appropriate deserializers - for example, an Kafka Streams app I > maintain stores model data (exported to a topic per entity from Postgres > via Kafka Connect's JDBC Source) as a GlobalKTable of Jackson ObjectNode's > keyed by entity id > > If you're worried about efficiency, just treat KTables/GlobalKTables as a > HashMap<K, V> to and you're pretty much there. In terms of efficiency, > we're joining model data to about 7 - 10 TB of transactional data a day, > and on average, run about 5 - 10 instances of our enrichment app with about > 2GB max heap. > > Kind regards, > > Liam "Not a part of the Confluent team, but happy to help" > Clarke-Hutchinson > > On Tue, May 12, 2020 at 9:35 PM Pushkar Deole <pdeole2...@gmail.com> > wrote: > > > Hello confluent team, > > > > Could you provide some information on what data structures are used > > internally by GlobalKTable and KTables. The application that I am working > > on has a requirement to read cached data from GlobalKTable on every > > incoming event, so the reads from GlobalKTable need to be efficient. > > >
