chaokunyang commented on code in PR #1413: URL: https://github.com/apache/incubator-fury/pull/1413#discussion_r1541372807
########## docs/protocols/xlang_object_graph_spec.md: ########## @@ -0,0 +1,612 @@ +# Cross language object graph serialization + +Fury xlang serialization is an automatic object serialization framework that supports reference and polymorphism. +Fury will convert an object from/to fury xlang serialization binary format. +Fury has two core concepts for xlang serialization: + +- **Fury xlang binary format** +- **Framework implemented in different languages to convert object to/from Fury xlang binary format** + +The serialization format is a dynamic binary format. The dynamics and reference/polymorphism support make Fury flexible, +much more easy to use, but +also introduce more complexities compared to static serialization frameworks. So the format will be more complex. + +## Type Systems + +### Data Types + +- bool: A boolean value (true or false). +- byte: An 8-bit signed integer. +- i16: A 16-bit signed integer. +- i32: A 32-bit signed integer. +- i64: A 64-bit signed integer. +- half-float: A 16-bit floating point number. +- float: A 32-bit floating point number. +- double: A 64-bit floating point number including NaN and Infinity. +- string: A text string encoded using Latin1/UTF16/UTF-8 encoding. +- enum: a data type consisting of a set of named values. Rust enum with non-predefined field values are not supported as + an enum +- list: A sequence of objects. +- set: An unordered set of unique elements. +- map: A map of key-value pairs. +- time types: + - Duration: an absolute length of time independent of any calendar/timezone, as a count of seconds and + fractions of seconds at nanosecond resolution. + - Timestamp: a point in time independent of any calendar/timezone, as a count of seconds and fractions of + seconds at nanosecond resolution. The count is relative to an epoch at UTC midnight on January 1, 1970. +- decimal: exact decimal value represented as an integer value in two's complement. Review Comment: The decimal will be serialized like: ```java buffer.writeVarUint32(value.scale()); buffer.writeVarUint32(value.precision()); buffer.writeVarUint32(bytes.length); buffer.writeBytes(value.unscaledValue().toByteArray()); ``` Could you elaborate how decimal128/decimal256 save space or provide better performance? One thing I can see is that we can skip write `buffer.writeVarUint32(bytes.length)`, we can take binary took up 128/256 bytes. but other data are still unavoidable. And if the unscaledValue is small, take `new BigDecimal(BigInteger.valueOf(100), 200, MathContext.DECIMAL128)` for an example, it took only 6 bytes, but when using decimal128 format, it will take 131 bytes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
