With Spark 1.4, you may use data source option "mergeSchema" to control it:
sqlContext.read.option("mergeSchema", "false").parquet("some/path")
or
CREATE TABLE t USING parquet OPTIONS (mergeSchema false, path
"some/path")
We're considering to disable schema merging by default in 1.5.0 since it
brings unnecessary performance cost when schema evolution is not a problem.
Cheng
On 6/23/15 2:20 AM, Rex Xiong wrote:
I remember in a previous PR, schema merging can be disabled by
setting spark.sql.hive.convertMetastoreParquet.mergeSchema to false.
But in 1.4 release, I don't see this config anymore, is there a new
way to do it?
Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
