[
https://issues.apache.org/jira/browse/SPARK-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383733#comment-16383733
]
Thomas Graves commented on SPARK-22479:
---------------------------------------
Also the example above shows the password, but the password should have been
already redacted, this pr excluded url, user, and username. Was the password
not being redacted for some reason?
> SaveIntoDataSourceCommand logs jdbc credentials
> -----------------------------------------------
>
> Key: SPARK-22479
> URL: https://issues.apache.org/jira/browse/SPARK-22479
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Onur Satici
> Assignee: Onur Satici
> Priority: Major
> Fix For: 2.2.1, 2.3.0
>
>
> JDBC credentials are not redacted in plans including a
> 'SaveIntoDataSourceCommand'.
> Steps to reproduce:
> {code}
> spark-shell --packages org.postgresql:postgresql:42.1.1
> {code}
> {code}
> import org.apache.spark.sql.execution.QueryExecution
> import org.apache.spark.sql.util.QueryExecutionListener
> val listener = new QueryExecutionListener {
> override def onFailure(funcName: String, qe: QueryExecution, exception:
> Exception): Unit = {}
> override def onSuccess(funcName: String, qe: QueryExecution, duration:
> Long): Unit = {
> System.out.println(qe.toString())
> }
> }
> spark.listenerManager.register(listener)
> spark.range(100).write.format("jdbc").option("url",
> "jdbc:postgresql:sparkdb").option("password", "pass").option("driver",
> "org.postgresql.Driver").option("dbtable", "test").save()
> {code}
> The above will yield the following plan:
> {code}
> == Parsed Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver ->
> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass),
> ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> == Analyzed Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver ->
> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass),
> ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> == Optimized Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver ->
> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass),
> ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> == Physical Plan ==
> ExecutedCommand
> +- SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver ->
> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass),
> ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
