GitHub user ericl opened a pull request:

    https://github.com/apache/spark/pull/20927

    [SPARK-23809][SQL] Active SparkSession should be set by getOrCreate

    ## What changes were proposed in this pull request?
    
    Currently, the active spark session is set inconsistently (e.g., in 
createDataFrame, prior to query execution). Many places in spark also 
incorrectly query active session when they should be calling 
activeSession.getOrElse(defaultSession).
    
    The semantics here can be cleaned up if we also set the active session when 
the default session is set.
    
    Related: https://github.com/apache/spark/pull/20926/files
    
    ## How was this patch tested?
    
    Unit test

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ericl/spark active-session-cleanup

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/20927.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #20927
    
----
commit 1b192d2542884f378a0ccc5c1936ff72c20403ee
Author: Eric Liang <ekl@...>
Date:   2018-03-28T22:19:39Z

    set active

commit 7a1b0f60ac9b6828223b1fce7efc79cb41b6d198
Author: Eric Liang <ekl@...>
Date:   2018-03-28T22:25:54Z

    test

commit 8f3cbf3399420a14f5ebe74b99b2739437fe3647
Author: Eric Liang <ekl@...>
Date:   2018-03-28T22:29:26Z

    mirror spark-23808

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to