Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/17725#discussion_r113091235
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -252,11 +252,17 @@ private[spark] class EventLoggingListener(
private[spark] def redactEvent(
event: SparkListenerEnvironmentUpdate):
SparkListenerEnvironmentUpdate = {
- // "Spark Properties" entry will always exist because the map is
always populated with it.
- val redactedProps = Utils.redact(sparkConf,
event.environmentDetails("Spark Properties"))
- val redactedEnvironmentDetails = event.environmentDetails +
- ("Spark Properties" -> redactedProps)
- SparkListenerEnvironmentUpdate(redactedEnvironmentDetails)
+ // environmentDetails maps a string descriptor to a set of properties
+ // Similar to:
+ // "JVM Information" -> jvmInformation,
+ // "Spark Properties" -> sparkProperties,
+ // ...
+ // where jvmInformation, sparkProperties, etc. are sequence of tuples.
+ // We go through the various of properties and redact sensitive
information from them.
+ val redactedProps = event.environmentDetails.map{
--- End diff --
`.map { case (name, props) =>`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
