scovich commented on code in PR #8006:
URL: https://github.com/apache/arrow-rs/pull/8006#discussion_r2252763116
##########
arrow-avro/src/reader/mod.rs:
##########
@@ -124,23 +132,26 @@ fn read_header<R: BufRead>(mut reader: R) ->
Result<Header, ArrowError> {
/// A low-level interface for decoding Avro-encoded bytes into Arrow
`RecordBatch`.
#[derive(Debug)]
pub struct Decoder {
- record_decoder: RecordDecoder,
+ active_decoder: RecordDecoder,
+ active_fingerprint: Option<Fingerprint>,
batch_size: usize,
- decoded_rows: usize,
+ remaining_capacity: usize,
+ #[cfg(feature = "lru")]
+ cache: LruCache<Fingerprint, RecordDecoder>,
+ #[cfg(not(feature = "lru"))]
+ cache: IndexMap<Fingerprint, RecordDecoder>,
+ max_cache_size: usize,
+ reader_schema: Option<AvroSchema<'static>>,
+ writer_schema_store: Option<SchemaStore<'static>>,
Review Comment:
I will admit I don't fully understand `'static` in rust. It doesn't
_necessarily_ imply a memory leak, but it does seem difficult (albeit
apparently not impossible) to apply `'static` lifetime to anything that will be
freed before the program terminates.
One way would be to never create a schema that actually uses `TypeName::Ref`
nor any `Attributes`, for example. But that's kind of cheating -- may as well
not support them at that point.
So I guess the question would be: Can one create a `Decoder` with Some
`reader_schema`, where the memory allocated to create that reader schema
actually gets freed when the decoder gets dropped?
A refinement of that question would be: Can one create a `Decoder` with Some
`reader_schema` that does use `Attributes` or `TypeName::Ref`, with dynamically
allocated strings that can be freed after the decoder is dropped?
Or, as I suspect, would one be forced to leak them (or use compile-time
constant strings instead), in order to get the code to compile?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
