Hi Justin, It is expected. We do not check if the provided schema matches rows since all rows need to be scanned to give a correct answer.
Thanks, Yin On Fri, Feb 13, 2015 at 1:33 PM, Justin Pihony <justin.pih...@gmail.com> wrote: > Per the documentation: > > It is important to make sure that the structure of every Row of the > provided RDD matches the provided schema. Otherwise, there will be runtime > exception. > > However, it appears that this is not being enforced. > > import org.apache.spark.sql._ > val sqlContext = new SqlContext(sc) > val struct = StructType(List(StructField("test", BooleanType, true))) > val myData = sc.parallelize(List(Row(0), Row(true), Row("stuff"))) > val schemaData = sqlContext.applySchema(myData, struct) //No error > schemaData.collect()(0).getBoolean(0) //Only now will I receive an error > > Is this expected or a bug? > > Thanks, > Justin > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/SQLContext-applySchema-strictness-tp21650.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >