bjornbouetsmith opened a new issue, #1142: URL: https://github.com/apache/pulsar-site/issues/1142
I have looked at a lot of pages, but I have been unable to find a definite answer as to how implementers of the clients should ensure that primitives that are serialized can be deserialized by any other client. https://pulsar.apache.org/docs/next/schema-understand/#primitive-type e.g. INT64 - takes up 8 bytes in memory and would be serialized to 8 bytes if you do not use variable length serialization. But since the endianness of the computer serializing and deseralizing the bytes controls the order of the actual bytes - then it would make sense that this project defines explicitly how numeric values must be serialized - so implementers does not have to guess. I am aware that within the protocol of pulsar itself it is stated to use BIG endian for the payload commands size: https://pulsar.apache.org/docs/next/developing-binary-protocol/ But imagine someone creates a producer that simply sends raw INT64 values onto a topic - then if that happens on a little endian computer, the bytes would be in a different order, and if a big endian computer deserializes the value - it will be incorrect. Here I am specifically talking about the scenario where producer and consumer are not running the same type of client, where there are no guarantee that the two different libraries have decided to take endianness into account and decide on the same order. This might just be me being unable to find it - but if not - then I think it would make sense if this project explicitly states how the primitive schema types must be serialized. I am aware that most would probably be serializing values in either proto or something else where the serialization format would already be defined. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
