dongjoon-hyun edited a comment on pull request #31284:
URL: https://github.com/apache/spark/pull/31284#issuecomment-768858103
I confirmed that this bug still exists in the vectorized reader while MR
reader can read this correctly like the test Parquet file in this PR.
```scala
scala> sql("set spark.sql.parquet.enableVectorizedReader=false")
res4: org.apache.spark.sql.DataFrame = [key: string, value: string]
scala>
spark.read.parquet("/tmp/decimal32-written-as-64-bit.snappy.parquet").show
21/01/27 23:27:24 WARN Types$BasePrimitiveBuilder: Decimal with 9 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
21/01/27 23:27:24 WARN Types$BasePrimitiveBuilder: Decimal with 9 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
21/01/27 23:27:24 WARN Types$BasePrimitiveBuilder: Decimal with 9 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
+-----------+
| _c0|
+-----------+
| 79205949.2|
| 98684298.7|
| 54024799.8|
| null|
| 35799107.8|
| 49413105.9|
| 9253639.6|
| 42684715.7|
|-99999999.9|
| 20448609.4|
+-----------+
scala>
spark.read.parquet("/tmp/decimal32-written-as-64-bit-dict.snappy.parquet").show
21/01/27 23:29:34 WARN Types$BasePrimitiveBuilder: Decimal with 3 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
21/01/27 23:29:34 WARN Types$BasePrimitiveBuilder: Decimal with 3 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
21/01/27 23:29:34 WARN Types$BasePrimitiveBuilder: Decimal with 3 digits is
stored in an INT64, but fits in an INT32. See
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md.
+-----+
| _c0|
+-----+
| 75.1|
| 93.7|
| 51.1|
| null|
| 33.7|
| 46.7|
| 8.4|
| 40.3|
|-99.9|
| 19.0|
| 4.1|
| 56.6|
| 9.6|
| 53.7|
| 30.8|
| null|
| 95.7|
| 1.8|
| 70.6|
| 0.2|
+-----+
only showing top 20 rows
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
