Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/22295#discussion_r222699236 --- Diff: python/pyspark/sql/session.py --- @@ -231,6 +231,7 @@ def __init__(self, sparkContext, jsparkSession=None): or SparkSession._instantiatedSession._sc._jsc is None: SparkSession._instantiatedSession = self self._jvm.SparkSession.setDefaultSession(self._jsparkSession) + self._jvm.SparkSession.setActiveSession(self._jsparkSession) --- End diff -- So @HyukjinKwon in this code session1 and session2 are already equal: > Welcome to > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /__ / .__/\_,_/_/ /_/\_\ version 2.3.1 > /_/ > > Using Python version 3.6.5 (default, Apr 29 2018 16:14:56) > SparkSession available as 'spark'. > >>> session1 = SparkSession.builder.config("key1", "value1").getOrCreate() > >>> session2 = SparkSession.builder.config("key2", "value2").getOrCreate() > >>> session1 > <pyspark.sql.session.SparkSession object at 0x7ff6d4843b00> > >>> session2 > <pyspark.sql.session.SparkSession object at 0x7ff6d4843b00> > >>> session1 == session2 > True > >>> > > > > > That being said the possibility of having multiple Spark session in Python is doable you manually have to call the init e.g.: > >>> session3 = SparkSession(sc) > >>> session3 > <pyspark.sql.session.SparkSession object at 0x7ff6d3dbd160> > >>> > And supporting that is reasonable.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org