alamb commented on issue #4840: URL: https://github.com/apache/arrow-rs/issues/4840#issuecomment-1964673511
> This might be a little off-topic. I'd like to propose a new array similar to `RunArray` but only accepts one value in construction, like `SingularArray`. The difference to `RunArray` is logic as `value()` can be simplified, and can have different behavior on null-related APIs (e.g., override the default impl of `is_valid()`/`is_null()`). And it can express both `(a)` & `(b)` (though I think it's not necessary to distinguish these two types...). This sounds very much like what datafusion `ScalarValue` and [`Datum`](https://docs.rs/arrow/latest/arrow/array/trait.Datum.html) are designed to do 🤔 -- I bet there would be interest on the arrow mailing list as well (they may have been prior discussion about it too) > I don't know if it's still an option to not have "logical null" and "physical null". Maybe overriding `is_valid()` and `is_null()` can have a slight help toward it? Adding a new array means lots of work, I'm not sure if this is viable, please let me know your thought. I think the `logical` and `physical` nulls refer to how the nulls are encoded in the Arrow arrays themselves. Given that this library is designed as a low level API for Arrow arrays, I believe the rationale is that exposing the null buffers directly as arrow encodes them provides the most control I made https://github.com/apache/arrow-rs/pull/5434 to try and clarify this even more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
