drse opened a new issue, #2273: URL: https://github.com/apache/fury/issues/2273
### Question I decided to use Fury as the serialization mechanism for both data in motion and data at rest across the entire application, and I'm wondering what would be the safest approach to future proof the data and prevent deserialization issues. So far, the approach is for every serialized payload, add a header consisting of the following: - byte: fury major version - byte: for fury minor version - byte: for fury patch version then a 32bit int with feature bit flags that correspond to the builder params as follows: - 0: language xlang - 1: language java - 2: language python - 3: language cpp - 4: language go - 5: language javascript - 6: language rust - 7: is row format - 8: timeRefIgnored - 9: compressInt - 10: compressLong - 11: compressString - 12: compatible mode COMPATIBLE - 13: compatible mode CONSISTENT - 14: checkClassVersion - 15: checkJdkClassSerializable - 16: registerGuavaTypes - 17: requireClassRegistration - 18: suppressClassRegistrationWarnings - 19: metaShareEnabled - 20: scopedMetaShareEnabled - 21: deserializeNonexistentClass - 22: codeGenEnabled - 23: asyncCompilationEnabled - 24: scalaOptimizationEnabled - 25: copyRef - 26: serializeEnumByName - 27: reserved - 28: reserved - 29: reserved - 30: reserved - 31: reserved My rational is that in cases of future errors, the builder can be constructed exactly as it was during the time of original serialization to recover. The question: is this overkill and unnecessary (Some of this data is already baked into the fury serialization protocol)? Also, is there any other metadata that should be included? I'm expecting to wind up with multiple languages and configurations across multiple fury versions. Is there a more compact way to represent the essential metadata? Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
