siddhantsangwan commented on PR #8707: URL: https://github.com/apache/ozone/pull/8707#issuecomment-3087146615
> Our current mechanism is based on the YAML file. If we generate cache directly from ContainerData, this mechanism will change. such as: KeyValueContainer#updateDataScanTimestamp, if the updateContainerFile throw a StorageContainerException in the updateContainerData, only acontainerData.setState(oldState); will be called to restore the Container state in the memory, the dataScanTimestamp may inconsistent between memory with Yaml. I see. I think it's worth ensuring that the in-memory object stays in sync with the yaml file and making this logic more robust first. We probably missed adding exception handling for this case earlier and only have handling for container state. Yes, the current mechanism will change. But it should be fine if the two are always consistent. So we have two options: 1. Compress the yaml and keep it in memory. 2. Ensure ContainerData is always consistent with on-disk yaml and use it instead. What do you think? I'm leaning toward the second option to keep memory usage low. Have you tried out the first option to see how much size reduction is possible because of compression? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org For additional commands, e-mail: issues-h...@ozone.apache.org
