[
https://issues.apache.org/jira/browse/SPARK-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453858#comment-16453858
]
Peter Simon commented on SPARK-11748:
-------------------------------------
Possible workaround can be:
{code:java}
scala> spark.sql("set parquet.column.index.access=true").show
scala> spark.sql("set spark.sql.hive.convertMetastoreParquet=false").show
scala> spark.sql ("select * from test_parq").show
+----+------+
|a_ch| b|
+----+------+
| 1| a|
| 2|test 2|
+----+------+
{code}
> Result is null after alter column name of table stored as Parquet
> ------------------------------------------------------------------
>
> Key: SPARK-11748
> URL: https://issues.apache.org/jira/browse/SPARK-11748
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.1
> Reporter: pin_zhang
> Priority: Major
>
> 1. Test with the following code
> hctx.sql(" create table " + table + " (id int, str string) STORED AS
> PARQUET ")
> val df = hctx.jsonFile("g:/vip.json")
> df.write.format("parquet").mode(SaveMode.Append).saveAsTable(table)
> hctx.sql(" select * from " + table).show()
> // alter table
> val alter = "alter table " + table + " CHANGE id i_d int "
> hctx.sql(alter)
>
> hctx.sql(" select * from " + table).show()
> 2. Result
> after change table column name, data in null for the changed column
> Result before alter table
> +---+---+
> | id|str|
> +---+---+
> | 1| s1|
> | 2| s2|
> +---+---+
> Result after alter table
> +----+---+
> | i_d|str|
> +----+---+
> |null| s1|
> |null| s2|
> +----+---+
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
