[
https://issues.apache.org/jira/browse/SPARK-15308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takuya Ueshin updated SPARK-15308:
----------------------------------
Description:
The following code generates wrong schema:
{code}
val schema = new StructType().add(
"struct",
new StructType()
.add("i", IntegerType, nullable = false)
.add(
"s",
new StructType().add("int", IntegerType, nullable = false),
nullable = false),
nullable = false)
val ds = sqlContext.range(10).map(l => Row(l, Row(l)))(RowEncoder(schema))
ds.printSchema()
{code}
This should print as follows:
{code}
root
|-- struct: struct (nullable = false)
| |-- i: integer (nullable = false)
| |-- s: struct (nullable = false)
| | |-- int: integer (nullable = false)
{code}
but the result is:
{code}
root
|-- struct: struct (nullable = false)
| |-- col1: integer (nullable = false)
| |-- col2: struct (nullable = false)
| | |-- col1: integer (nullable = false)
{code}
was:
The following code generates wrong schema:
{code}
val schema = new StructType().add(
"struct",
new StructType()
.add("i", IntegerType, nullable = false)
.add(
"s",
new StructType().add("int", IntegerType, nullable = false),
nullable = false),
nullable = false)
val ds = sqlContext.range(10).map(l => Row(l, Row(l)))(RowEncoder(schema))
ds.printSchema()
{code}
This should print as follows:
{code}
root
|-- struct: struct (nullable = false)
| |-- i: integer (nullable = false)
| |-- s: struct (nullable = false)
| | |-- int: integer (nullable = false)
{code}
but the result is:
{code}
|-- struct: struct (nullable = false)
| |-- col1: integer (nullable = false)
| |-- col2: struct (nullable = false)
| | |-- col1: integer (nullable = false)
{code}
> RowEncoder should preserve nested column name.
> ----------------------------------------------
>
> Key: SPARK-15308
> URL: https://issues.apache.org/jira/browse/SPARK-15308
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Takuya Ueshin
>
> The following code generates wrong schema:
> {code}
> val schema = new StructType().add(
> "struct",
> new StructType()
> .add("i", IntegerType, nullable = false)
> .add(
> "s",
> new StructType().add("int", IntegerType, nullable = false),
> nullable = false),
> nullable = false)
> val ds = sqlContext.range(10).map(l => Row(l, Row(l)))(RowEncoder(schema))
> ds.printSchema()
> {code}
> This should print as follows:
> {code}
> root
> |-- struct: struct (nullable = false)
> | |-- i: integer (nullable = false)
> | |-- s: struct (nullable = false)
> | | |-- int: integer (nullable = false)
> {code}
> but the result is:
> {code}
> root
> |-- struct: struct (nullable = false)
> | |-- col1: integer (nullable = false)
> | |-- col2: struct (nullable = false)
> | | |-- col1: integer (nullable = false)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
