[ https://issues.apache.org/jira/browse/SPARK-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-22479: ------------------------------------ Assignee: Apache Spark > SaveIntoDataSourceCommand logs jdbc credentials > ----------------------------------------------- > > Key: SPARK-22479 > URL: https://issues.apache.org/jira/browse/SPARK-22479 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.2.0 > Reporter: Onur Satici > Assignee: Apache Spark > > JDBC credentials are not redacted in plans including a > 'SaveIntoDataSourceCommand'. > Steps to reproduce: > {code} > spark-shell --packages org.postgresql:postgresql:42.1.1 > {code} > {code} > import org.apache.spark.sql.execution.QueryExecution > import org.apache.spark.sql.util.QueryExecutionListener > val listener = new QueryExecutionListener { > override def onFailure(funcName: String, qe: QueryExecution, exception: > Exception): Unit = {} > override def onSuccess(funcName: String, qe: QueryExecution, duration: > Long): Unit = { > System.out.println(qe.toString()) > } > } > spark.listenerManager.register(listener) > spark.range(100).write.format("jdbc").option("url", > "jdbc:postgresql:sparkdb").option("password", "pass").option("driver", > "org.postgresql.Driver").option("dbtable", "test").save() > {code} > The above will yield the following plan: > {code} > == Parsed Logical Plan == > SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> > org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), > ErrorIfExists > +- Range (0, 100, step=1, splits=Some(8)) > == Analyzed Logical Plan == > SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> > org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), > ErrorIfExists > +- Range (0, 100, step=1, splits=Some(8)) > == Optimized Logical Plan == > SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> > org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), > ErrorIfExists > +- Range (0, 100, step=1, splits=Some(8)) > == Physical Plan == > ExecutedCommand > +- SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> > org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), > ErrorIfExists > +- Range (0, 100, step=1, splits=Some(8)) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org