GitHub user juliuszsompolski opened a pull request:
https://github.com/apache/spark/pull/20136
[SPARK-22938] Assert that SQLConf.get is accessed only on the driver.
## What changes were proposed in this pull request?
Assert if code tries to access SQLConf.get on executor.
This can lead to hard to detect bugs, where the executor will read
fallbackConf, falling back to default config values, ignoring potentially
changed non-default configs.
If a config is to be passed to executor code, it needs to be read on the
driver, and passed explicitly.
## How was this patch tested?
Check in existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/juliuszsompolski/apache-spark SPARK-22938
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20136.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20136
----
commit d2b3bc6374cde4f8d4ebeedd7612f51d18f13806
Author: Juliusz Sompolski <julek@...>
Date: 2017-12-14T17:50:23Z
[SPARK-22938] Assert that SQLConf.get is accessed only on the driver.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
