Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/22295#discussion_r225666954 --- Diff: python/pyspark/sql/session.py --- @@ -231,6 +231,7 @@ def __init__(self, sparkContext, jsparkSession=None): or SparkSession._instantiatedSession._sc._jsc is None: SparkSession._instantiatedSession = self self._jvm.SparkSession.setDefaultSession(self._jsparkSession) + self._jvm.SparkSession.setActiveSession(self._jsparkSession) --- End diff -- @holdenk @HyukjinKwon Thanks for the comments. I looked the scala code, it ```setActiveSession``` in ```createDataFrame```. ``` def createDataFrame[A <: Product : TypeTag](rdd: RDD[A]): DataFrame = { SparkSession.setActiveSession(this) ... } ``` I will do the same for python. ``` def createDataFrame(self, data, schema=None, samplingRatio=None, verifySchema=True): SparkSession._activeSession = self self._jvm.SparkSession.setActiveSession(self._jsparkSession) ``` Will also add a test
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org