LuciferYang commented on code in PR #12981:
URL: https://github.com/apache/gluten/pull/12981#discussion_r3969479545
##########
backends-velox/src/test/scala/org/apache/spark/sql/execution/VeloxParquetWriteForHiveSuite.scala:
##########
@@ -298,7 +291,7 @@ class VeloxParquetWriteForHiveSuite extends GlutenQueryTest
with SQLTestUtils wi
reader =>
val column =
reader.getFooter.getBlocks.get(0).getColumns.get(0)
// native writer and vanilla spark hive writer should be
consistent
- "zstd".equalsIgnoreCase(column.getCodec.toString)
+ assert("zstd".equalsIgnoreCase(column.getCodec.toString))
Review Comment:
Added the message, and it paid off immediately. CI now reports:
```
expected zstd but got GZIP in
file:.../VeloxParquetWriteForHiveSuite/t/part-00000-...,
nativeWriteEnabled=true
```
So the native writer takes `spark.sql.parquet.compression.codec` and ignores
the `parquet.compression` table property. That is a real behaviour gap, not a
bad assertion: the case is named `native writer should respect table
properties` and it does not. It passed before only because the comparison was a
discarded Boolean.
Filed as #12987 and the case is `ignore`d here with the reason and the issue
in a comment. The assertion stays inside the ignored case so it is ready when
someone fixes the writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
