Marco Lugo created AVRO-4031:
--------------------------------
Summary: Writer::append_to_with_codec cannot take block_size so it
falls back to default even if original Writer had a different size
Key: AVRO-4031
URL: https://issues.apache.org/jira/browse/AVRO-4031
Project: Apache Avro
Issue Type: Improvement
Components: rust
Reporter: Marco Lugo
Writer::append_to_with_codec is great to continue a writer but it can be
inconsistent with the original writer in terms of block_size. If the original
writer used a custom block_size then this new writer will fall back to
DEFAULT_BLOCK_SIZE. This can be important when considering compression among
other things.
I am not sure what the proper API should be here:
- Change append_to_with_codec's signature ro reflect this, which would be a
breaking change;
- Add a new method, something like append_to_with_codec_and_block_size, but
this does not feel very future proof;
- Allow the Writer builder to set has_header to true. If I'm understanding this
correctly, this would be future proof and would not break backwards
compatibility;
- Other?
>From the little context I have, I would be inclined to choose the 3rd option.
What would be the preferred option, if any? I'd be happy to open a PR once
there is consensus on the way forward.
Thank you,
Marco
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
