> Offset and Transaction indexes are probably the only ones that make sense
to cache as are used on every fetch.
I do not think (correct me if I am wrong) that the transaction index is
used on every fetch. It is only used when consumers want to include aborted
transactions [1] i.e. when they use
Divij, thanks for your prompt feedback!
1. Agree, caching at the plugin level was my initial idea as well; though,
keeping two caches for the same data both at the broker and at the plugin
seems wasteful. (added this as a rejected alternative in the meantime)
2. Not necessarially. The API allows
Hi Jorge
1. I don't think we need a new API here because alternatives solutions
exist even with the current API. As an example, when the first index is
fetched, the RSM plugin can choose to download all indexes and cache it
locally. On the next call to fetch an index from the remote tier, we will
Hello everyone,
I would like to start the discussion on a KIP for Tiered Storage. It's
about improving cross-segment latencies by reducing calls to fetch indexes
individually.
Have a look:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1002%3A+Fetch+remote+segment+indexes+at+once
Cheers,
