mosche commented on PR #24992: URL: https://github.com/apache/beam/pull/24992#issuecomment-1411492609
> We can keep it "as it is" for now but it will be anyway a breaking change once Avro will be dropped from "core". Do you see any other options? @aromanenko-dev You don't need Avro to produce Avro compatible bytes. It's a well defined format with a detailed Spec. Of course this could become a hassle quickly, but luckily this case is trivial. Avro uses [varint zigzag encoding for longs](https://avro.apache.org/docs/1.8.2/spec.html#binary_encoding), that's just the same Protobuf is using for [signed ints](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-ints). And Avro records are just a concatenation of their fields without any further additions: Here's an example using Protobufs `CodedOutputStream` ```Java //import org.apache.beam.vendor.grpc.v1p48p1.com.google.protobuf.CodedOutputStream CodedOutputStream cos = CodedOutputStream.newInstance(outputStream); cos.writeSInt64NoTag(mark.getLastEmitted()); // signed int64 with varint zigzag encoding cos.writeSInt64NoTag(mark.getStartTime().getMillis()); // signed int64 with varint zigzag encoding cos.flush(); ``` Of course, that would require some additional tests to verify compatibility... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
