jiacai2050 commented on issue #1600: URL: https://github.com/apache/horaedb/issues/1600#issuecomment-2506900473
Those are all great questions, 1. We use `version` field to deal with schema evolution, if we want to add some fields to manifest, a new version could be added, and when merge, to convert old manifest to new one. 2. Object store don't have the append interface, so append here means download the old one into memory, then merge with new delta manifests, then upload it again to overwrite the old one. 3. There shouldn't many manifest files, we have a hard limit on how many delta files a manifest can have, if there are more than that, the write process will fail, only after the merge process is finished, we can allow creating new manifest deltas. As for the third question, that why we need to keep metadata of each sst small, so we can hold millions of sst files in one manifest snapshot, whose size is less than 1GB. ``` shell 1024*1024*1024 / 28 (size of each sst's metadata) = 38347922 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
