rahil-c opened a new pull request, #19366:
URL: https://github.com/apache/hudi/pull/19366
### Change Logs
`ProtoConversionUtil` maps a proto `uint64` field to an Avro **FIXED**
`unsigned_long` of size 9. When it synthesizes the field default it returned a
raw `byte[]`, which Avro renders in the schema JSON as a base64 string
(`"AAAAAAAAAAAA"`, 12 chars). Avro 1.11/1.12 tightened `Schema.validateDefault`
for FIXED and rejects this because the default length (12) does not equal the
fixed size (9):
```
org.apache.avro.AvroTypeException: Invalid default for field <uint64 field>:
"AAAAAAAAAAAA" not a {"type":"fixed","name":"unsigned_long", ... "size":9,
...}
at org.apache.avro.Schema.validateDefault(Schema.java)
at org.apache.avro.Schema$Field.<init>(Schema.java)
at
org.apache.hudi.utilities.sources.helpers.ProtoConversionUtil$AvroSupport.getMessageSchema(...)
```
Any proto schema with a `uint64` field therefore fails schema generation on
Avro 1.11+ (e.g. via `ProtoSchemaToAvroSchemaConverter` / a proto schema
registry).
Fix: render the fixed bytes as their equivalent ISO-8859-1 String (a 1:1
byte-to-char mapping), so the default is a valid FIXED default of length 9. The
default value itself is unchanged. Added a regression test that generates the
schema for a proto with a `uint64` field and re-parses it with default
validation enabled (the existing tests compare `Schema` objects, and
`Schema.equals` ignores field defaults, so the invalid default was not caught).
### Impact
Fixes proto-to-Avro schema generation for any proto containing a `uint64`
field when running on Avro 1.11/1.12. No behavior change for existing data
(default value unchanged).
### Risk level: low
Localized change to the default synthesized for `uint64` fields, covered by
a new regression test.
### Documentation Update
N/A
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Change Logs and Impact were stated clearly
- [x] Adequate tests were added if applicable
- [x] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
