[
https://issues.apache.org/jira/browse/SPARK-43782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727784#comment-17727784
]
Snoot.io commented on SPARK-43782:
----------------------------------
User 'jwang0306' has created a pull request for this issue:
https://github.com/apache/spark/pull/41302
> Support log level configuration with static Spark conf
> ------------------------------------------------------
>
> Key: SPARK-43782
> URL: https://issues.apache.org/jira/browse/SPARK-43782
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.4.0
> Reporter: Yan-Lin (Jared) Wang
> Priority: Major
>
> Today users are using {{sc.setLogLevel}} to configure the current log level
> using {{SparkContext}} accessible from within the {{{}SparkSession{}}}. It is
> a convenient alternative to directly depending on the Log4j mechanism like
> statically changing the {{log4j.xml}} or dynamically invoke the Log4j API.
> It would be great to allow the configuration of the log level using a static
> Spark conf that has the same kind of simplicity, which comes in handy when we
> don't want to manually change the {{{}log4j.xml{}}}. This is particularly
> helpful in the Spark Connect scenario where there’s no way for the client to
> call {{setLogLevel}} because {{SparkContext}} is not yet supported in its
> API, and when it connects to a platform where it can't easily change the
> Log4j config file.
> ./bin/pyspark --conf spark.log.level ERROR
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
