Hi Kamal, Thanks for the feedback. It’s helpful to understand the overlapping parts between local and remote logs.
Regarding `onlyLocalLogSize`, does it refer to local log size minus overlapping retention part? If so, I prefer to set the field when IncludeRemoteInfo is true. Renamed the `RemotePartitionSize` to `RemoteLogSize`. Updated `PartitionSize` description. --- Hi Ally, The log dirs can also include directories in remote storage. If we introduce `describeRemoteLog`, it will be a new RPC. However, for retrieving the remote log size, we don’t need a new RPC. We can include this information in the existing `describeLogDirs` RPC. If we want to provide more details about remote logs in the future, we can consider introducing a new RPC at that time. Thank you, PoAn > On Jun 27, 2025, at 3:46 PM, Kamal Chandraprakash > <kamal.chandraprak...@gmail.com> wrote: > > Hi PoAn, > > Thanks for the KIP! Having the remoteLogSize as part of the DescribeLogDirs > response will be useful to determine > the exact cost of the topic. > > Kafka uploads the segments eagerly. Assume a topic is configured with 48 > hrs of retention time and 12 hrs of > local-retention time. Then, the remote storage might contain ~47 hrs > (excluding the active segment) of data and > local storage might contain 12 hrs of data. When you combine the > (PartitionSize + RemotePartitionSize) sizes, > then it might be ~60 hrs. > > The current attributes in DescribeLogDirsResult: > > 1. `PartitionSize` - provides the size of the local-log and > 2. `RemotePartitionSize` - provides the size of the remote-log. > > We may need another attribute to provide the size of the segments that > exist only in the local-log: `onlyLocalLogSize`. > The user can determine the cost of the topic as per their requirements. > > nit: > 1. Shall we rename the `RemotePartitionSize` to `RemoteLogSize` since all > the replicas send the values in the response? > 2. Shall we update the about/description of the `PartitionSize` attribute > to mention that the size can be of local-log when remote storage is enabled > on the topic? > > Thanks, > Kamal > > On Mon, Jun 16, 2025 at 8:03 PM PoAn Yang <yangp...@gmail.com> wrote: > >> Hi all, >> >> I would like to start a discussion thread about KIP-1187. >> >> Please take a look and feel free to share any thought. >> >> https://cwiki.apache.org/confluence/x/sYkhFg >> >> Thanks, >> PoAn
