alamb commented on PR #7741:
URL: https://github.com/apache/arrow-rs/pull/7741#issuecomment-3001892724

   > I wonder if we can add a gc() method to VariantObject. Similar to a 
StringView, users can call VariantObject::gc to undergo this expensive 
operation of pruning stale references, etc...
   
   I think maybe we could just show how to do this with an example (rather than 
having to make a function) prior to anyone actually encountering the error. I 
think it could be done by recursively rewriting  the entire variant
   
   
   > The above would certainly work, in the sense of producing a valid variant 
object. My only concern would be that the scenario almost certainly arises due 
to user error (which is quite different from a generic map or set), and 
silently tolerating that error isn't necessarily doing the user any favors in 
the long run. They'll just discover at read time that they lost data, instead 
of fast-failing at write time. We can probably get away with either approach -- 
silently replacing or loudly complaining -- I just want to be sure we make the 
choice intentionally.
   
   Maybe we could have some flag that controls the validation behavior? 
Something like
   ```rust
   let mut builder = VariantBuilder::new();
   let mut obj = builder.new_object()
     // specify that an error should be thrown on repeated fields
     .with_validate_unique_fields()
   ...
   obj.finish()?; // this throws error if there were repeated fields
   ```
   
   That way people could check for errors programatically if they wanted to and 
could disable the checking if they didn't care 🤔  
   
   
   This is all for a follow on PR I think
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to