Applying schema is a pretty low-level operation, and I would expect most users would use the type safe interfaces. If you are unsure you can always run:
import org.apache.spark.sql.execution.debug._ schemaRDD.typeCheck() and it will tell you if you have made any mistakes. Michael On Sat, Feb 14, 2015 at 1:05 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > Would it make sense to add an optional validate parameter to applySchema() > which defaults to False, both to give users the option to check the schema > immediately and to make the default behavior clearer? > > > On Sat Feb 14 2015 at 9:18:59 AM Michael Armbrust <mich...@databricks.com> > wrote: > >> Doing runtime type checking is very expensive, so we only do it when >> necessary (i.e. you perform an operation like adding two columns together) >> >> On Sat, Feb 14, 2015 at 2:19 AM, nitin <nitin2go...@gmail.com> wrote: >> >>> AFAIK, this is the expected behavior. You have to make sure that the >>> schema >>> matches the row. It won't give any error when you apply the schema as it >>> doesn't validate the nature of data. >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/SQLContext-applySchema-strictness-tp21650p21653.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >>> >>
