Zongheng Yang created SPARK-2498:
------------------------------------
Summary: Concurrent initialization of various DataType objects
causes exceptions
Key: SPARK-2498
URL: https://issues.apache.org/jira/browse/SPARK-2498
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.0.1
Reporter: Zongheng Yang
Reported by Keith Simmons and gorenuru
[here|http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-Spark-SQL-join-when-using-Spark-1-0-1-td9776.html].
This is probably due to the fact that Scala Reflection API is not thread-safe
for 2.10.* versions; see [SI-6240|https://issues.scala-lang.org/browse/SI-6240]
for more details.
The purposed fix is to add a lock object that is synchronized on when various
data type objects make use of the reflection API (during their initializations
or other places in program execution). (Longer-term, we could consider evolving
this to a SparkSQL-global lock, and all calls to the reflection APi from Spark
SQL should probably synchronize on it.)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
