buraksenn opened a new pull request, #24945:
URL: https://github.com/apache/datafusion/pull/24945
## Which issue does this PR close?
- Closes #24621.
## Rationale for this change
Protobuf hooks that access fields individually can silently omit newly added
state. For JSON sinks, this caused an explicitly configured compression level
to revert to the default after a physical-plan protobuf roundtrip.
Exhaustive destructuring makes newly added source, sink, and wire fields
compile
errors until their serialization behavior is explicitly considered.
## What changes are included in this PR?
- Exhaustively destructure `JsonSource` and `JsonSink` in their encoders.
- Exhaustively destructure their protobuf nodes in the decoders.
- Keep the active `JsonSink` hook exhaustive while centralizing field
mapping in
its public `TryFrom<&JsonSink>` conversion.
- Add `compression_level` to the `JsonWriterOptions` protobuf message and
preserve it in both conversion directions.
- Strengthen the JSON sink roundtrip test to inspect non-default sink
options,
file configuration, and sort order directly.
The protobuf change is additive and backward compatible.
## Are these changes tested?
Yes. The focused JSON source and sink roundtrip tests pass:
```bash
cargo test -p datafusion-proto --test proto_integration roundtrip_json
```
## Are there any user-facing changes?
Physical plans containing JSON sinks now preserve an explicitly configured
compression level across protobuf roundtrips. Older payloads without the new
field continue to decode with no explicit compression level.
There are no breaking API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
