Good idea. https://github.com/apache/spark/pull/14252
On Mon, Jul 18, 2016 at 12:16 PM, Michael Armbrust <mich...@databricks.com> wrote: > + dev, reynold > > Yeah, thats a good point. I wonder if SparkSession.sqlContext should be > public/deprecated? > > On Mon, Jul 18, 2016 at 8:37 AM, Koert Kuipers <ko...@tresata.com> wrote: > >> in my codebase i would like to gradually transition to SparkSession, so >> while i start using SparkSession i also want a SQLContext to be available >> as before (but with a deprecated warning when i use it). this should be >> easy since SQLContext is now a wrapper for SparkSession. >> >> so basically: >> val session = SparkSession.builder.set(..., ...).getOrCreate() >> val sqlc = new SQLContext(session) >> >> however this doesnt work, the SQLContext constructor i am trying to use >> is private. SparkSession.sqlContext is also private. >> >> am i missing something? >> >> a non-gradual switch is not very realistic in any significant codebase, >> and i do not want to create SparkSession and SQLContext independendly (both >> from same SparkContext) since that can only lead to confusion and >> inconsistent settings. >> > >
