felipecrv commented on PR #41320: URL: https://github.com/apache/arrow/pull/41320#issuecomment-2075450368
> I would argue that libarrow/libparquet should be robust against hostile/corrupted datasets even on Release builds, as those kind of crashes are undesirable, and may potentially have security implications. I don't disagree, but we can't and don't want to validate everywhere to ensure safety, some classes need to assume pre-conditions. These pre-conditions should be documented. For instance, every array type has a complex `Validate` function that we can't afford to call on every compute kernel, but that doesn't mean we trust files or IPC stream to contain valid arrays. Validations should happen as close as possible to communication of the untrusted external world (e.g. reading Parquet files). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
