Sim did you find anything? :) On Sun, Jul 26, 2015 at 9:31 AM, sim <s...@swoop.com> wrote:
> The schema merging > <http://spark.apache.org/docs/latest/sql-programming-guide.html#schema-merging> > section of the Spark SQL documentation shows an example of schema evolution > in a partitioned table. > > Is this functionality only available when creating a Spark SQL table? > > dataFrameWithEvolvedSchema.saveAsTable("my_table", SaveMode.Append) fails > with > > java.lang.RuntimeException: Relation[ ... ] > org.apache.spark.sql.parquet.ParquetRelation2@83a73a05 > requires that the query in the SELECT clause of the INSERT INTO/OVERWRITE > statement generates the same number of columns as its schema. > > What is the Spark SQL idiom for appending data to a table while managing > schema evolution? > ------------------------------ > View this message in context: Schema evolution in tables > <http://apache-spark-user-list.1001560.n3.nabble.com/Schema-evolution-in-tables-tp23999.html> > Sent from the Apache Spark User List mailing list archive > <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. > >
