vbhanuchander-lang commented on PR #28996: URL: https://github.com/apache/flink/pull/28996#issuecomment-5518549850
@dianfu could you take a look, or point me at the right reviewer? You have merged essentially all recent `flink-python` changes, and FLINK-40370 is still `Open` and unassigned — I asked for the assignment on the JIRA on 20 Aug and have not had a reply, so I suspect this is simply sitting unseen rather than being held. Status: Azure is green on the head commit (build 78229), and the branch still applies to `master`. The change is two files in `pyflink/fn_execution/formats/avro.py` plus its test. `FlinkAvroEncoder` and `FlinkAvroDecoder` override the primitive reads and writes for JVM compatibility, but not `write_decimal_bytes` / `read_decimal_from_bytes`, which `avro>=1.12` (adopted in FLINK-37192) routes bytes-backed decimals through. Those size the payload with `write_long`, which in this encoder is a fixed 8-byte long, where the JVM frames a bytes field with a 4-byte int: ``` expected (JVM) : 0000000204d200000007 actual : 000000000000000204d200000007 ``` The JVM reader consumes the oversized prefix as the whole field, so every following field shifts and nothing is raised — silent corruption rather than a decode error. Flink's own Python reader is wrong in the same direction, which is why a pure-Python round trip looks fine and only the JVM/Python boundary shows it. Standard Avro interoperability is unaffected — `avro`, `fastavro` and ordinary Kafka Avro payloads are unchanged; this is specific to Flink's JVM-compatible serializer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
