I am assuming that you are referring to the RSM.fetchLogSegment() API call. The RemoteLogSegmentMetadata object passed to it contains RemoteLogSegmentId which contains information about Topic and the Partition for this segment. Isn't that information sufficient for your use case? If not, RemoteLogSegmentMetadata also contains a map, CustomMetadata which is opaque to the broker and is populated by RemoteLogMetadataManager. You can choose to feed any attributes you require into this custom metadata and read it in RSM.
Does this answer your question? -- Divij Vaidya On Wed, Nov 8, 2023 at 8:55 PM philipp lehmann < philipp.lehm...@medionmail.com> wrote: > Hello, > > If I understand tiered storage correctly, the RemoteStorageManager doesn't > know to which topic the data it receives belongs. In some environments that > offer access-pattern-based storage, this is disadvantageous. If the > corresponding topics were part of the storage request, the > RemoteStorageManager could use this to predict the access pattern. Which in > turn enables the selection of the best matching storage. Let's say that > there's a topic that gets rarely consumed. In this case, the > RemoteStorageManager could use colder storage, which results in lower > storage costs. So my question is, is this within the scope of tiered > storage? If it is, what is needed to get it changed? > > Regards, > Philipp >
