GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20830
[SPARK-23691][PYTHON] Use sql_conf util in PySpark tests where possible ## What changes were proposed in this pull request? https://github.com/apache/spark/commit/d6632d185e147fcbe6724545488ad80dce20277e added an useful util ```python @contextmanager def sql_conf(self, pairs): ... ``` to allow configuration set/unset within a block: ```python with self.sql_conf({"spark.blah.blah.blah", "blah"}) # test codes ``` This PR proposes to use this util where possible in PySpark tests. Note that there look already few places affecting tests without restoring the original value back in unittest classes. ## How was this patch tested? Manually tested via: ``` ./run-tests --modules=pyspark-sql --python-executables=python2 ./run-tests --modules=pyspark-sql --python-executables=python3 ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark cleanup-sql-conf Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20830.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20830 ---- commit 89cf69be7ae00f571c51de402067928b663c5a45 Author: hyukjinkwon <gurwls223@...> Date: 2018-03-15T04:16:18Z Use sql_conf util in PySpark tests where possible ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org