[
https://issues.apache.org/jira/browse/SPARK-13626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-13626:
------------------------------------
Assignee: (was: Apache Spark)
> SparkConf deprecation log messages are printed multiple times
> -------------------------------------------------------------
>
> Key: SPARK-13626
> URL: https://issues.apache.org/jira/browse/SPARK-13626
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, SQL
> Affects Versions: 2.0.0
> Reporter: Marcelo Vanzin
> Priority: Minor
>
> I noticed that if I have a deprecated config in my spark-defaults.conf, I'll
> see multiple warnings when running, for example, spark-shell. I collected the
> backtrace from when the messages are printed, and here's a few instances. The
> first one is the only one I expect to be printed.
> {noformat}
> java.lang.Exception:
> ...
> at org.apache.spark.SparkConf.<init>(SparkConf.scala:53)
> at org.apache.spark.repl.Main$.<init>(Main.scala:30)
> {noformat}
> The following ones are causing duplicate log messages and we should clean
> those up:
> {noformat}
> java.lang.Exception:
> at
> org.apache.spark.SparkConf$$anonfun$logDeprecationWarning$2.apply(SparkConf.scala:682)
> ...
> at org.apache.spark.SparkConf.<init>(SparkConf.scala:53)
> at org.apache.spark.repl.Main$.createSparkContext(Main.scala:82)
> {noformat}
> {noformat}
> java.lang.Exception:
> at
> org.apache.spark.SparkConf$$anonfun$logDeprecationWarning$2.apply(SparkConf.scala:682)
> ...
> at org.apache.spark.SparkConf.setAll(SparkConf.scala:139)
> at org.apache.spark.SparkConf.clone(SparkConf.scala:358)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:368)
> at org.apache.spark.repl.Main$.createSparkContext(Main.scala:98)
> {noformat}
> There are also a few more caused by the use of {{SparkConf.clone()}}.
> {noformat}
> java.lang.Exception:
> at
> org.apache.spark.SparkConf$$anonfun$logDeprecationWarning$2.apply(SparkConf.scala:682)
> ...
> at org.apache.spark.SparkConf.<init>(SparkConf.scala:59)
> at org.apache.spark.SparkConf.<init>(SparkConf.scala:53)
> at
> org.apache.spark.deploy.SparkHadoopUtil.<init>(SparkHadoopUtil.scala:48)
> {noformat}
> {noformat}
> java.lang.Exception:
> at
> org.apache.spark.SparkConf$$anonfun$logDeprecationWarning$2.apply(SparkConf.scala:682)
> ...
> at org.apache.spark.SparkConf.<init>(SparkConf.scala:53)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:93)
> at
> org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
> {noformat}
> {noformat}
> java.lang.Exception:
> at
> org.apache.spark.SparkConf$$anonfun$logDeprecationWarning$2.apply(SparkConf.scala:682)
> ...
> at org.apache.spark.SparkConf.<init>(SparkConf.scala:53)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:93)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
