[ https://issues.apache.org/jira/browse/SPARK-48946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mikó Szilárd updated SPARK-48946: --------------------------------- Target Version/s: (was: 4.0.0) > NPE in DataSourceV2ScanExecBase redact method when session is null > ------------------------------------------------------------------ > > Key: SPARK-48946 > URL: https://issues.apache.org/jira/browse/SPARK-48946 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 4.0.0 > Reporter: Mikó Szilárd > Priority: Major > > If we call redact method from a thread which was created before we had a > Spark session we get an NPE. > [https://github.com/apache/spark/blob/3f6e2d6ef354c034ce0b39a092d23dd99431a16c/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExecBase.scala#L70C3-L75C4] > > Changing > {code:java} > session.sessionState.conf.stringRedactionPattern {code} > to > {code:java} > conf.stringRedactionPattern {code} > could prevent this problem as conf checks if session is null or not. > [https://github.com/apache/spark/blob/af5eb085b0ac01eec2bb5db206b7d499d462abc4/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala#L69] > In case of DataSourceScanExec we already use conf.stringRedactionPattern: > https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala#L93-L95 -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org