felipepessoto commented on code in PR #13136:
URL: https://github.com/apache/gluten/pull/13136#discussion_r4354352026
##########
gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/datasources/GlutenFileMetadataStructSuite.scala:
##########
@@ -45,33 +55,64 @@ class GlutenFileMetadataStructSuite extends
FileMetadataStructSuite with GlutenS
private val METADATA_FILE_NAME = "_metadata.file_name"
private val METADATA_FILE_SIZE = "_metadata.file_size"
private val METADATA_FILE_MODIFICATION_TIME =
"_metadata.file_modification_time"
+ private val METADATA_FILE_BLOCK_START = "_metadata.file_block_start"
+ private val METADATA_FILE_BLOCK_LENGTH = "_metadata.file_block_length"
+
+ private val metadataColumns = Seq(
+ METADATA_FILE_PATH,
+ METADATA_FILE_NAME,
+ METADATA_FILE_SIZE,
+ METADATA_FILE_MODIFICATION_TIME,
+ METADATA_FILE_BLOCK_START,
+ METADATA_FILE_BLOCK_LENGTH)
+
+ private val schemaWithMetadataCollisions = StructType(
+ metadataColumns.zipWithIndex.map {
+ case (name, index) =>
+ StructField(name.stripPrefix("_metadata."), if (index < 2) LongType
else StringType)
Review Comment:
Fixed in 8e0881901. The fixture now uses an explicit `Map[String, DataType]`
keyed by metadata column name instead of `index < 2`. Field order and row data
are unchanged. All six focused collision regression cases passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
