jiacai2050 opened a new issue, #1600: URL: https://github.com/apache/horaedb/issues/1600
### Describe This Problem Currently our manifest is defined using protobuf, that's: https://github.com/apache/horaedb/blob/9e81c4ed5df1998cbd210dc48fc67b6b7405a553/horaedb/pb_types/protos/sst.proto#L32 Protobuf is useful for schema evolution, but not very efficient in our case: - For `Vec<struct>` field, protobuf will serialize metadata of every struct, which is a waste of space. - Serialize manifest body using protobuf in a whole make a hard to update incrementally. ### Proposal We can serialize manifest all by ourselves, a proposed format: ``` | version(u8) | length(u32)| Record(N) ... | # Record is a self-descriptive message | id(u64) | time_range(i64*2)| size(u32) | ``` ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
