Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2024-07-02 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1654742829 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -481,6 +488,13 @@ public static TableMetadata fromJson(String metadataLocation, JsonNode

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2024-06-24 Thread via GitHub
advancedxy commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1650705615 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -481,6 +488,13 @@ public static TableMetadata fromJson(String metadataLocation, JsonNode no

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-18 Thread via GitHub
ajantha-bhat commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1861945339 @aokolnychyi: Thanks for the detailed review and merge. The next small PR to review will be https://github.com/apache/iceberg/pull/9284 (Spark 3.5: Ensure that partition stats

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-18 Thread via GitHub
aokolnychyi merged PR #8502: URL: https://github.com/apache/iceberg/pull/8502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-18 Thread via GitHub
aokolnychyi commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1430591144 ## core/src/main/java/org/apache/iceberg/SetPartitionStatistics.java: ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-15 Thread via GitHub
ajantha-bhat commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1858140592 Re-triggering build due to flaky test in Flink https://github.com/apache/iceberg/actions/runs/7222400549/job/19679307743?pr=8502 -- This is an automated message from the Apa

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-15 Thread via GitHub
ajantha-bhat closed pull request #8502: Api: Track partition statistics via TableMetadata URL: https://github.com/apache/iceberg/pull/8502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-15 Thread via GitHub
ajantha-bhat commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1857853254 @aokolnychyi: I have handled all the new comments. Thanks again for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-15 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427955084 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -946,15 +955,25 @@ private Builder(TableMetadata base) { this.previousFileLocation = base

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-15 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427954785 ## api/src/main/java/org/apache/iceberg/PartitionStatisticsFile.java: ## @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-15 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427926385 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1288,6 +1307,24 @@ public Builder suppressHistoricalSnapshots() { return this; }

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-15 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427920868 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -946,15 +955,25 @@ private Builder(TableMetadata base) { this.previousFileLocation = base

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-14 Thread via GitHub
aokolnychyi commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1856508439 @ajantha-bhat, I'll take a look at other PRs once this is in. I feel this one is almost ready to go. -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-14 Thread via GitHub
aokolnychyi commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427180249 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -946,15 +955,25 @@ private Builder(TableMetadata base) { this.previousFileLocation = base.

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-12 Thread via GitHub
ajantha-bhat commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1851876373 @aokolnychyi: Fixed all the comments and also opened a new Spark module PR (which is dependent on this) to ensure partition stats are considered for GC (expire snapshots and remove o

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-12 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r142387 ## core/src/main/java/org/apache/iceberg/SetPartitionStatistics.java: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-12 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1423873054 ## core/src/main/java/org/apache/iceberg/ReachableFileUtil.java: ## @@ -137,7 +137,9 @@ public static List manifestListLocations(Table table, Set snapshot *

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-11 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1422543077 ## .palantir/revapi.yml: ## @@ -873,6 +873,10 @@ acceptedBreaks: new: "method void org.apache.iceberg.encryption.Ciphers::()" justification: "Static

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-11 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1422541957 ## core/src/main/java/org/apache/iceberg/ReachableFileUtil.java: ## @@ -137,7 +137,9 @@ public static List manifestListLocations(Table table, Set snapshot *

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-07 Thread via GitHub
ajantha-bhat commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1845791782 @aokolnychyi: Thanks for the review. - I see that most of the questions or comments related to why puffin stats followed that style. We can fix in this PR for partition stats

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-06 Thread via GitHub
aokolnychyi commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1412672873 ## .palantir/revapi.yml: ## @@ -873,6 +873,10 @@ acceptedBreaks: new: "method void org.apache.iceberg.encryption.Ciphers::()" justification: "Static u

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-12-05 Thread via GitHub
aokolnychyi commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1841336236 I started looking on Friday but got distracted. I will try to finish by end of Wed. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-11-30 Thread via GitHub
ajantha-bhat commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1834166805 > Ack, will review this week. Thanks. Also, please include https://github.com/apache/iceberg/pull/9170 in the review -- This is an automated message from the Apache Git

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-11-28 Thread via GitHub
aokolnychyi commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1830900823 Ack, will review this week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

2023-11-28 Thread via GitHub
ajantha-bhat commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1407512055 ## api/src/main/java/org/apache/iceberg/PartitionStatisticsFile.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or