My understanding is one of the biggest advantages of DF¹s is that schema information allows a lot of optimization. For example assume frame had many column but your computation only uses 2 columns. No need to load all the data.
Andy From: "email2...@gmail.com" <email2...@gmail.com> Date: Monday, December 14, 2015 at 7:35 PM To: "user @spark" <user@spark.apache.org> Subject: what are the cons/drawbacks of a Spark DataFrames > Hello All - I've started using the Spark DataFrames and looks like it > provides rich column level operations and functions. > > In the same time, I would like to understand are there any drawbacks / cons > of using a DataFrames?. If so please share your experience on that. > > Thanks, > Gokul > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/what-are-the-cons-drawback > s-of-a-Spark-DataFrames-tp25703.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
