Sean Owen created SPARK-17193: --------------------------------- Summary: HadoopRDD NPE at DEBUG log level when getLocationInfo == null Key: SPARK-17193 URL: https://issues.apache.org/jira/browse/SPARK-17193 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 2.0.0 Reporter: Sean Owen Assignee: Sean Owen Priority: Trivial
When I set the log level to "DEBUG" in one of my apps that reads from Parquet, I notice several NullPointerExceptions logged from HadoopRDD.getPreferredLocations. It doesn't affect executions as it just results in "no preferred locations". It happens when InputSplitWithLocationInfo.getLocationInfo produces null, which it may. The code just dereferences it however. It's cleaner to check this directly (and maybe tighten up the code slightly) and avoid polluting the log, though, it's just at debug level. No big deal, but enough of an annoyance when I was debugging something that it's probably worth zapping. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org