[
https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328928#comment-16328928
]
Steve Loughran commented on SPARK-21697:
----------------------------------------
No, it's spark's ability to have hdfs:// URLs on the classpath.
The classpath is being scanned for commons logging properties, which is forcing
in HDFS which is then NPEing as the logging code is being called before
commons-logging is fully set up. Kind of a recursive class init problem
triggered by a scan for commons-logging.properties.
> NPE & ExceptionInInitializerError trying to load UTF from HDFS
> --------------------------------------------------------------
>
> Key: SPARK-21697
> URL: https://issues.apache.org/jira/browse/SPARK-21697
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.1
> Environment: Spark Client mode, Hadoop 2.6.0
> Reporter: Steve Loughran
> Priority: Minor
>
> Reported on [the
> PR|https://github.com/apache/spark/pull/17342#issuecomment-321438157] for
> SPARK-12868: trying to load a UDF of HDFS is triggering an
> {{ExceptionInInitializerError}}, caused by an NPE which should only happen if
> the commons-logging {{LOG}} log is null.
> Hypothesis: the commons logging scan for {{commons-logging.properties}} is
> happening in the classpath with the HDFS JAR; this is triggering a D/L of the
> JAR, which needs to force in commons-logging, and, as that's not inited yet,
> NPEs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
