jecsand838 commented on PR #7834:
URL: https://github.com/apache/arrow-rs/pull/7834#issuecomment-3059322972

   @scovich 
   
   > > I don't think the decoder should support implicitly inferring a schema 
to be clear. It should be the responsibility of the caller to identify when the 
writer schema has changed and to call a method that updates the decoder
   > 
   > How would the caller do that, out of curiosity? A quick skim of the wire 
protocol didn't turn up anything obvious. Does the sender embed a new 
header+schema that the caller should notice and respond to as they see fit?
   
   The caller can do this by being signaled to read a new .avsc file, 
retrieving an updated schema definition from a schema registry, or attempting 
to decode an avro header from the sender (which I think happens) using the 
`Header` decoder in `arrow-avro/src/reader/header.rs` upon detection (maybe by 
detecting the magic byte / something defined in the specifications) or after 
getting a `ParseError` or `SchemaError` returned from the `Decoder`. I'm 
personally most familiar with schema registry (Kafka) use cases.
   
   We can definitely add utility functions to assist with this as needed and/or 
make the `Reader` and `Decoder` configurable to attempt to detect a new header 
definition record, but imo the responsibility should be on the caller since 
it's very use-case specific.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to