Re: what are the cons/drawbacks of a Spark DataFrames

My understanding is one of the biggest advantages of DF¹s is that schema information allows a lot of optimization. For example assume frame had many column but your computation only uses 2 columns. No need to load all the data. Andy From: "email2...@gmail.com" Date:

Re: what are the cons/drawbacks of a Spark DataFrames

With DataFrames you loose type-safety. Depending on the language you are using this can also be considered a drawback. On 15 December 2015 at 15:08, Jakob Odersky wrote: > By using DataFrames you will not need to specify RDD operations explicity, > instead the operations are

Re: what are the cons/drawbacks of a Spark DataFrames

By using DataFrames you will not need to specify RDD operations explicity, instead the operations are built and optimized for by using the information available in the DataFrame's schema. The only draw-back I can think of is some loss of generality: given a dataframe containing types A, you will

what are the cons/drawbacks of a Spark DataFrames

Hello All - I've started using the Spark DataFrames and looks like it provides rich column level operations and functions. In the same time, I would like to understand are there any drawbacks / cons of using a DataFrames?. If so please share your experience on that. Thanks, Gokul -- View