stczwd commented on a change in pull request #28048: [SPARK-31142][PYSPARK]Remove useless conf set in pyspark context URL: https://github.com/apache/spark/pull/28048#discussion_r399972619
########## File path: python/pyspark/context.py ########## @@ -181,11 +181,6 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize, self.appName = self._conf.get("spark.app.name") self.sparkHome = self._conf.get("spark.home", None) - for (k, v) in self._conf.getAll(): Review comment: In linux, we can use export LD_LIBRARY_PATH to inherit or overwrite environment. We want to support this in spark as well. ```bash # we got same result in linux export LD_LIBRARY_PATH=$PYTHONHOME/lib:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH=./python/lib:/usr/native_lib; # we can overwrite environment in linux like this export LD_LIBRARY_PATH=./python/lib ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org