Github user moustaki commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-141298174
@marmbrus On your earlier documentation point, I was thinking that there
might be genuine cases where the actual location of a partition might not be
matching the expected partition structure, and where this check needs to be
disabled.
The case I just encountered was a unit test which was adding as a partition
a randomly generated temp directory using ALTER TABLE / ADD PARTITION, which
leads to an NPE in
https://github.com/apache/spark/pull/5059/files#diff-8887a877bd52611df9aea06ccfe3a2d7R166
when running a SELECT against the corresponding table.
Disabling the check using that configuration flag gets rid of that
exception. Really not sure how common that use-case is though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
