alamb commented on issue #1071: URL: https://github.com/apache/arrow-rs/issues/1071#issuecomment-998314855
I am not 100% sure if supporting non monotonic offsets is something we want to fix to be honest. My rationale is if the offsets are monotonic, all arrays with offsets can calculate length as `offsets[i] - offsets[i-1]` which has better failure properties than if we ended up doing stuff like `0 - 32` or something because the offset had been left unset Thus Also for fun, I checked @jorgecarleitao 's implementation in arrow2 which also ensures that offsets are always monotonic. https://github.com/jorgecarleitao/arrow2/blob/37a9c758826a92d98dc91e992b2a49ce9724095d/src/array/specification.rs#L102-L119 I am pretty sure the C++ implementation also ensures the offsets are monotonic without first checking it is null: https://github.com/apache/arrow/blob/master/cpp/src/arrow/array/validate.cc#L568-L592 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
