[
https://issues.apache.org/jira/browse/SPARK-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Lian updated SPARK-13540:
-------------------------------
Description:
Add the following test case to {{DatasetSuite}} to reproduce this issue:
{code}
object OuterObject {
case class InnerClass(a: String)
}
test("foo") {
OuterScopes.addOuterScope(OuterObject)
Seq(OuterObject.InnerClass("foo")).toDS()
}
{code}
Exception thrown:
{noformat}
org.apache.spark.sql.AnalysisException: Unable to generate an encoder for inner
class `org.apache.spark.sql.OuterObject$InnerClass` without access to the scope
that this class was defined in.
{noformat}
The reason is that an inner class within a Scala object is translated into a
static nested class, and no outer scope is needed. But the analyzer still
thinks there should be an outer scope.
was:
Add the following test case to {{DatasetSuite}} to reproduce this issue:
{code}
object OuterObject {
case class InnerClass(a: String)
}
test("foo") {
OuterScopes.addOuterScope(OuterObject)
Seq(OuterObject.InnerClass("foo")).toDS()
}
{code}
Exception thrown:
{noformat}
org.apache.spark.sql.AnalysisException: Unable to generate an encoder for inner
class `org.apache.spark.sql.OuterObject$InnerClass` without access to the scope
that this class was defined in.
{noformat}
The reason is that the class name added to {{OuterScope}} is the the one with
{{$}} suffix (class name of the object), while the Java declaring class of
{{InnerClass}} is the one without {{$}} suffix.
> Scala object can't be added to OuterScopes
> ------------------------------------------
>
> Key: SPARK-13540
> URL: https://issues.apache.org/jira/browse/SPARK-13540
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Cheng Lian
> Assignee: Cheng Lian
>
> Add the following test case to {{DatasetSuite}} to reproduce this issue:
> {code}
> object OuterObject {
> case class InnerClass(a: String)
> }
> test("foo") {
> OuterScopes.addOuterScope(OuterObject)
> Seq(OuterObject.InnerClass("foo")).toDS()
> }
> {code}
> Exception thrown:
> {noformat}
> org.apache.spark.sql.AnalysisException: Unable to generate an encoder for
> inner class `org.apache.spark.sql.OuterObject$InnerClass` without access to
> the scope that this class was defined in.
> {noformat}
> The reason is that an inner class within a Scala object is translated into a
> static nested class, and no outer scope is needed. But the analyzer still
> thinks there should be an outer scope.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
