Now, I am not able to directly use my RDD object and have it implicitly
become a DataFrame. It can be used as a DataFrameHolder, of which I could
write:
rdd.toDF.registerTempTable(foo)
The rational here was that we added a lot of methods to DataFrame and made
the implicits more
It appears that the metastore_db problem is related to
https://issues.apache.org/jira/browse/SPARK-4758. I had another shell open
that was stuck. This is probably a bug, though?
import sqlContext.implicits
case class Foo(x: Int)
val rdd = sc.parallelize(List(Foo(1)))
rdd.toDF
To answer your first question - yes 1.3.0 did break backward compatibility for
the change from SchemaRDD - DataFrame. SparkSQL was an alpha component so api
breaking changes could happen. It is no longer an alpha component as of 1.3.0
so this will not be the case in future.
Adding toDF
