vamshikrishnakyatham opened a new issue, #14139:
URL: https://github.com/apache/hudi/issues/14139
### Bug Description
**What happened:**
When downgrading a Hudi table from version 9 to version 8, column stats V2
partitions are correctly deleted, but partition stats partitions remain in the
metadata table. This leaves the table in an inconsistent state with:
- partition_stats directory still present in .hoodie/metadata/
- partition_stats still listed in hoodie.properties under
hoodie.table.metadata.partitions
- Stale index definitions remaining in .hoodie/.index_defs/index.json
**What you expected:**
During downgrade from version 9 to version 8:
1. Both column stats V2 and partition stats V2 should be deleted (they are
both incompatible with version 8)
2. Index definitions for both should be removed from index.json
3. Metadata partition entries should be removed from table config
**Steps to reproduce:**
1. Create a table with version 9, enabling column stats and partition
stats:
```
df.write.format("hudi")
.option("hoodie.table.version", "9")
.option("hoodie.metadata.index.column.stats.enable", "true")
.option("hoodie.metadata.index.partition.stats.enable", "true")
.save(basePath)
```
2. Verify both partitions exist in .hoodie/metadata/ and check index.json
3. Downgrade to version 8:
`spark.sql(s"call downgrade_table(table => '$tableName', to_version =>
'EIGHT')").show()`
4. Observe:
- column_stats is deleted
- partition_stats is NOT deleted
- index.json still contains entries for V2 stats
### Environment
**Hudi version:** 1.1
**Query engine:** (Spark/Flink/Trino etc)
**Relevant configs:**
### Logs and Stack Trace
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
