Johan Lasperas created SPARK-48308:
--------------------------------------
Summary: Unify getting data schema without partition columns in
FileSourceStrategy
Key: SPARK-48308
URL: https://issues.apache.org/jira/browse/SPARK-48308
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.5.1
Reporter: Johan Lasperas
In
[FileSourceStrategy,|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala#L191]
the schema of the data excluding partition columns is computed 2 times in a
slightly different way:
```
val dataColumnsWithoutPartitionCols =
dataColumns.filterNot(partitionSet.contains)
```
vs
```
val readDataColumns = dataColumns
.filterNot(partitionColumns.contains)
```
This should be unified
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
