Fokko commented on PR #11842: URL: https://github.com/apache/iceberg/pull/11842#issuecomment-2560684459
@advancedxy Sorry for ignoring comment 1, I had to think about that one a bit: > upgrade the v1 table to v2 and then remove the void transform in the old spec and produces a new one. Call ExpireSnapshot with metadata clean when all the data files written by old spec are rewritten/cleaned after ttl. After that, the v1 specs are expired and the offending columns can be removed. So, we cannot alter existing partition specs. Even after upgrading to V2, the metadata is still in V1. The relevant part of the spec: <img width="783" alt="image" src="https://github.com/user-attachments/assets/044e5081-8e95-4ec2-a515-a78899c30633"; /> This means they are still sequential, and we cannot rely on field-IDs. Removing one field in the middle is dangerous since you alter the order. Best case it will throw an error when evaluating the data (if the types are compatible), worst case it will return faulty data. I think when it finds a partition field, that does not point to a field anymore because it is dropped, it should just be ignored when the partition-spec is evaluated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
