alamb commented on PR #7741: URL: https://github.com/apache/arrow-rs/pull/7741#issuecomment-3001892724
> I wonder if we can add a gc() method to VariantObject. Similar to a StringView, users can call VariantObject::gc to undergo this expensive operation of pruning stale references, etc... I think maybe we could just show how to do this with an example (rather than having to make a function) prior to anyone actually encountering the error. I think it could be done by recursively rewriting the entire variant > The above would certainly work, in the sense of producing a valid variant object. My only concern would be that the scenario almost certainly arises due to user error (which is quite different from a generic map or set), and silently tolerating that error isn't necessarily doing the user any favors in the long run. They'll just discover at read time that they lost data, instead of fast-failing at write time. We can probably get away with either approach -- silently replacing or loudly complaining -- I just want to be sure we make the choice intentionally. Maybe we could have some flag that controls the validation behavior? Something like ```rust let mut builder = VariantBuilder::new(); let mut obj = builder.new_object() // specify that an error should be thrown on repeated fields .with_validate_unique_fields() ... obj.finish()?; // this throws error if there were repeated fields ``` That way people could check for errors programatically if they wanted to and could disable the checking if they didn't care 🤔 This is all for a follow on PR I think -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org