This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 02f45229171 [SPARK-39133][PYTHON][DOC] Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED 02f45229171 is described below commit 02f452291712c32f167d7d9389b1fc375448a6de Author: Xinrong Meng <xinrong.m...@databricks.com> AuthorDate: Tue May 10 17:59:39 2022 +0900 [SPARK-39133][PYTHON][DOC] Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED ### What changes were proposed in this pull request? Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED. ### Why are the changes needed? Even if `spark.sql.pyspark.jvmStacktrace.enabled` is set False, we should set log level to FATAL to see Python exceptions only. The PR is proposed to document that. ### Does this PR introduce _any_ user-facing change? No. Documents change only. ### How was this patch tested? Manual tests. Closes #36490 from xinrong-databricks/doc_jvm_stacktrace. Lead-authored-by: Xinrong Meng <xinrong.m...@databricks.com> Co-authored-by: Xinrong Meng <47337188+xinrong-databri...@users.noreply.github.com> Co-authored-by: Hyukjin Kwon <gurwls...@gmail.com> Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> --- .../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala index 4c0eccbf35d..43d342fd06b 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala @@ -2577,8 +2577,9 @@ object SQLConf { val PYSPARK_JVM_STACKTRACE_ENABLED = buildConf("spark.sql.pyspark.jvmStacktrace.enabled") .doc("When true, it shows the JVM stacktrace in the user-facing PySpark exception " + - "together with Python stacktrace. By default, it is disabled and hides JVM stacktrace " + - "and shows a Python-friendly exception only.") + "together with Python stacktrace. By default, it is disabled to hide JVM stacktrace " + + "and shows a Python-friendly exception only. Note that this is independent from log " + + "level settings.") .version("3.0.0") .booleanConf // show full stacktrace in tests but hide in production by default. --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org