Re: [I] dropDeleteFilesOlderthan should be partition level instead of table level [iceberg]

2024-01-03 Thread via GitHub
manuzhang commented on issue #9383: URL: https://github.com/apache/iceberg/issues/9383#issuecomment-1876612899 @zinking I see. An extreme case is if there's one partition left not compacted, none of the other partitions can drop their delete files after compaction. -- This is an

Re: [I] dropDeleteFilesOlderthan should be partition level instead of table level [iceberg]

2024-01-03 Thread via GitHub
zinking commented on issue #9383: URL: https://github.com/apache/iceberg/issues/9383#issuecomment-1876294561 @manuzhang sounds different stuff. the issue pointed here is not POS delete specific. equality delete has same issue. the key here is `partition` delete files within a partition

Re: [I] dropDeleteFilesOlderthan should be partition level instead of table level [iceberg]

2024-01-03 Thread via GitHub
manuzhang commented on issue #9383: URL: https://github.com/apache/iceberg/issues/9383#issuecomment-1876206670 > I am seeing v2 tables (partitioned tables) having delete files retained in partitions but those delete files wont apply to any data files within that partition. This is

Re: [I] dropDeleteFilesOlderthan should be partition level instead of table level [iceberg]

2023-12-26 Thread via GitHub
zinking commented on issue #9383: URL: https://github.com/apache/iceberg/issues/9383#issuecomment-1869871515 @RussellSpitzer any comments ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[I] dropDeleteFilesOlderthan should be partition level instead of table level [iceberg]

2023-12-26 Thread via GitHub
zinking opened a new issue, #9383: URL: https://github.com/apache/iceberg/issues/9383 ### Apache Iceberg version 1.4.2 (latest release) ### Query engine Spark ### Please describe the bug  ``` public List apply(TableMetadata base, Snapshot snapshot) {