nevi-me commented on pull request #9492: URL: https://github.com/apache/arrow/pull/9492#issuecomment-778737657
@jorgecarleitao instead of marking it unsafe as the first option, I'd prefer that we fully validate that we create spec-compliant arrays. If the performance hit on the full validation is significant, then we could expose an `unsafe make_array_unchecked`. I'm currently tackling a slice change that works with deeply-nested structs, and having stronger validations could make that effort less painful than it currently is. Some suggestions (I'm happy to work on them with you, as I've wanted to do this for a while): - validate field nullability against arrays (this is more for when creating RecordBatch, so perhaps not `make_array` related) - Check that the length of buffers equals the stipulated length in `ArrayData`. This would extend to the physical bit width because 2 x i16 should = 4 u8s in the buffer (if there's no offset) - Check that list offsets are sequential. I don't think we validate such a case in offsets: `[0, 0, 5, 3, 6, 6]` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
