pepijnve commented on issue #1637: URL: https://github.com/apache/arrow-rs/issues/1637#issuecomment-4826213497
I was going through old issues and this one piqued my interest. I had a look at the current state of `UnionBuilder` and it seems like there is quite some room for improvement. The `BTreeMap<String, FieldData>` and the way `FieldData` seems to be removed and reinserted on each append in particular seems rather problematic from a performance point of view. An API where the possible fields are specified up front seems more sensible. I can have a go at implementing this, but integrating that in the existing implementation seems like it's more hassle than it's worth. In particular, being forced to pass the field name on each append (and as a consequence being forced to verify it in the implementation) doesn't seem desirable. What would the best solution be here? Create a new parallel 'statically defined union type' implementation or break compatibility of the existing implementation? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
