Re: [PR] Support changelog scan for table with delete files [iceberg]

2024-08-21 Thread via GitHub
wypoon commented on code in PR #10935: URL: https://github.com/apache/iceberg/pull/10935#discussion_r1726394464 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestChangelogReader.java: ## @@ -191,39 +214,361 @@ public void testMixDeleteAndInsert() throws IOExc

Re: [PR] Support changelog scan for table with delete files [iceberg]

2024-08-21 Thread via GitHub
wypoon commented on PR #10935: URL: https://github.com/apache/iceberg/pull/10935#issuecomment-2303850176 @pvary I have fixed the implementation so that existing deletes are applied before new deletes are emitted. I have fixed the test case accordingly. (I also renamed `testFlinkScenario1` a

Re: [PR] Support changelog scan for table with delete files [iceberg]

2024-08-21 Thread via GitHub
wypoon commented on code in PR #10935: URL: https://github.com/apache/iceberg/pull/10935#discussion_r1726387609 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/ChangelogRowReader.java: ## @@ -112,13 +149,62 @@ private CloseableIterable openChangelogScanTask(Ch

Re: [PR] Support changelog scan for table with delete files [iceberg]

2024-08-21 Thread via GitHub
wypoon commented on code in PR #10935: URL: https://github.com/apache/iceberg/pull/10935#discussion_r1726379602 ## data/src/main/java/org/apache/iceberg/data/DeleteFilter.java: ## @@ -197,31 +197,31 @@ record -> deleteSet.contains(projectRow.wrap(asStructLike(record))); }

Re: [PR] Support changelog scan for table with delete files [iceberg]

2024-08-21 Thread via GitHub
wypoon commented on code in PR #10935: URL: https://github.com/apache/iceberg/pull/10935#discussion_r1726371909 ## gradle.properties: ## @@ -32,4 +32,4 @@ org.gradle.parallel=true org.gradle.configureondemand=true # explicitly disable the configuration cache org.gradle.config

Re: [PR] Procedure to compute table stats [iceberg]

2024-08-21 Thread via GitHub
karuppayya commented on PR #10986: URL: https://github.com/apache/iceberg/pull/10986#issuecomment-2303795329 @aokolnychyi @szehon-ho Can you help review this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[PR] Procedure to compute table stats [iceberg]

2024-08-21 Thread via GitHub
karuppayya opened a new pull request, #10986: URL: https://github.com/apache/iceberg/pull/10986 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
karuppayya commented on PR #10288: URL: https://github.com/apache/iceberg/pull/10288#issuecomment-2303782938 Thanks @aokolnychyi and everyone for the reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Core,AWS: Fix NPE in ResolvingFileIO when HadoopConf is not set [iceberg]

2024-08-21 Thread via GitHub
munendrasn commented on PR #10872: URL: https://github.com/apache/iceberg/pull/10872#issuecomment-2303780773 Thank you @nastra and @amogh-jahagirdar for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

2024-08-21 Thread via GitHub
stevenzwu commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1726315077 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/JdbcLockFactory.java: ## @@ -0,0 +1,321 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

2024-08-21 Thread via GitHub
stevenzwu commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1726304565 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/JdbcLockFactory.java: ## @@ -0,0 +1,246 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

2024-08-21 Thread via GitHub
stevenzwu commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1726213077 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestLockFactoryBase.java: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

2024-08-21 Thread via GitHub
stevenzwu commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1726304565 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/JdbcLockFactory.java: ## @@ -0,0 +1,246 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

2024-08-21 Thread via GitHub
stevenzwu commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1726213077 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestLockFactoryBase.java: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Core,AWS: Fix NPE in ResolvingFileIO when HadoopConf is not set [iceberg]

2024-08-21 Thread via GitHub
amogh-jahagirdar merged PR #10872: URL: https://github.com/apache/iceberg/pull/10872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Core,AWS: Fix NPE in ResolvingFileIO when HadoopConf is not set [iceberg]

2024-08-21 Thread via GitHub
amogh-jahagirdar commented on code in PR #10872: URL: https://github.com/apache/iceberg/pull/10872#discussion_r1726207335 ## core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java: ## @@ -174,7 +176,7 @@ DelegateFileIO io(String location) { return ioInstances.compute

Re: [PR] Core,AWS: Fix NPE in ResolvingFileIO when HadoopConf is not set [iceberg]

2024-08-21 Thread via GitHub
amogh-jahagirdar commented on PR #10872: URL: https://github.com/apache/iceberg/pull/10872#issuecomment-2303628644 THanks @munendrasn and thanks @nastra for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Flink: backport PR #10777 from 1.19 to 1.18 for sink test refactoring. [iceberg]

2024-08-21 Thread via GitHub
stevenzwu commented on code in PR #10965: URL: https://github.com/apache/iceberg/pull/10965#discussion_r1726192495 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java: ## @@ -70,6 +70,7 @@ public class FlinkCatalogFactory implements CatalogFactor

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi commented on PR #10288: URL: https://github.com/apache/iceberg/pull/10288#issuecomment-2303599910 Great work, @karuppayya! Thanks everyone for reviewing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi merged PR #10288: URL: https://github.com/apache/iceberg/pull/10288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [I] test: Integration test using sparksql and datafusion. [iceberg-rust]

2024-08-21 Thread via GitHub
liurenjie1024 commented on issue #574: URL: https://github.com/apache/iceberg-rust/issues/574#issuecomment-2303568240 I'll take it to design a framework. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Spark 3.5: Add utility to load table state reliably [iceberg]

2024-08-21 Thread via GitHub
karuppayya commented on code in PR #10984: URL: https://github.com/apache/iceberg/pull/10984#discussion_r1726144179 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -851,18 +851,29 @@ private static void deleteManifests(FileIO io, List manif

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1726000959 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/NDVSketchUtil.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[PR] Spark 3.5: Add utility to load table state reliably [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi opened a new pull request, #10984: URL: https://github.com/apache/iceberg/pull/10984 While reviewing #10288, I released we don't have a reliable way to load Iceberg table state as `Dataset` in Spark. -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1725995212 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1725995212 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1725995212 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-21 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1725995060 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,177 @@ +/* + * Licensed to the Apache Software

Re: [PR] API, AWS: Add RetryableInputStream and use that in S3InputStream [iceberg]

2024-08-21 Thread via GitHub
amogh-jahagirdar commented on code in PR #10433: URL: https://github.com/apache/iceberg/pull/10433#discussion_r1725991357 ## aws/src/main/java/org/apache/iceberg/aws/s3/S3InputStream.java: ## @@ -139,7 +140,11 @@ private InputStream readRange(String range) { S3RequestUtil

Re: [PR] API, AWS: Add RetryableInputStream and use that in S3InputStream [iceberg]

2024-08-21 Thread via GitHub
amogh-jahagirdar commented on code in PR #10433: URL: https://github.com/apache/iceberg/pull/10433#discussion_r1725989680 ## aws/src/test/java/org/apache/iceberg/aws/s3/TestFuzzyS3InputStream.java: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] API, AWS: Add RetryableInputStream and use that in S3InputStream [iceberg]

2024-08-21 Thread via GitHub
amogh-jahagirdar commented on code in PR #10433: URL: https://github.com/apache/iceberg/pull/10433#discussion_r1725989680 ## aws/src/test/java/org/apache/iceberg/aws/s3/TestFuzzyS3InputStream.java: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [I] Delete files not eventually removed if RewriteDataFile run right after delete (when using 'use-starting-sequence-number' default) [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #4127: Delete files not eventually removed if RewriteDataFile run right after delete (when using 'use-starting-sequence-number' default) URL: https://github.com/apache/iceberg/issues/4127 -- This is an automated message from the Apache Git Service. To respond

Re: [I] Support S3 Batch Removal of objects as part of snapshot expiration and removing orphan files [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #4012: Support S3 Batch Removal of objects as part of snapshot expiration and removing orphan files URL: https://github.com/apache/iceberg/issues/4012 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] Delete files not eventually removed if RewriteDataFile run right after delete (when using 'use-starting-sequence-number' default) [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #4127: URL: https://github.com/apache/iceberg/issues/4127#issuecomment-2303330460 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support S3 Batch Removal of objects as part of snapshot expiration and removing orphan files [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #4012: URL: https://github.com/apache/iceberg/issues/4012#issuecomment-2303330427 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Allowing ZOrder Distribution for normal Spark Writes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3962: URL: https://github.com/apache/iceberg/issues/3962#issuecomment-2303330391 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support Iceberg Metadata storage in a variety of engines [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3997: URL: https://github.com/apache/iceberg/issues/3997#issuecomment-2303330405 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Allowing ZOrder Distribution for normal Spark Writes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3962: Allowing ZOrder Distribution for normal Spark Writes URL: https://github.com/apache/iceberg/issues/3962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] [Feature Request] Support for change data capture [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3941: URL: https://github.com/apache/iceberg/issues/3941#issuecomment-2303330330 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Linkage errors in published jars [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3958: Linkage errors in published jars URL: https://github.com/apache/iceberg/issues/3958 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Core, Spark: Fallback when snapshot does not have schema id [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3852: Core, Spark: Fallback when snapshot does not have schema id URL: https://github.com/apache/iceberg/pull/3852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Support Iceberg branching for all the snapshot producer operations [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3896: URL: https://github.com/apache/iceberg/issues/3896#issuecomment-2303330307 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support Iceberg branching for all the snapshot producer operations [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3896: Support Iceberg branching for all the snapshot producer operations URL: https://github.com/apache/iceberg/issues/3896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] ParquetWriter leaks memory [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3950: ParquetWriter leaks memory URL: https://github.com/apache/iceberg/issues/3950 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] HiveTableOperations: update HMS transient_lastDdlTime param on snapshot writes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3670: HiveTableOperations: update HMS transient_lastDdlTime param on snapshot writes URL: https://github.com/apache/iceberg/pull/3670 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [I] [Improvement] ParallelIterable#hasNext submit reading ManifestFile task slowly with DataTableScan#planTasks [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3741: URL: https://github.com/apache/iceberg/issues/3741#issuecomment-2303330257 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] retain flinkCheckpointInfo when rewrite datafiles to ensure restore flink job from checkpoint success [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3647: retain flinkCheckpointInfo when rewrite datafiles to ensure restore flink job from checkpoint success URL: https://github.com/apache/iceberg/pull/3647 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] AWS: Make the error message of GlueCatalog's isValidIdentifier more detailed [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3668: URL: https://github.com/apache/iceberg/pull/3668#issuecomment-2303330221 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] AWS: Make the error message of GlueCatalog's isValidIdentifier more detailed [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3668: AWS: Make the error message of GlueCatalog's isValidIdentifier more detailed URL: https://github.com/apache/iceberg/pull/3668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] Adding MergeInto into the Spark Scala API [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3665: Adding MergeInto into the Spark Scala API URL: https://github.com/apache/iceberg/issues/3665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [I] can not change nullable via Spark SQL sometimes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3617: can not change nullable via Spark SQL sometimes URL: https://github.com/apache/iceberg/issues/3617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] BaseDataReader is not properly closed during query by spark sql [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3636: BaseDataReader is not properly closed during query by spark sql URL: https://github.com/apache/iceberg/issues/3636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] SPARK[docs]:Add Warning information under the MERGE INTO syntax [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3644: URL: https://github.com/apache/iceberg/pull/3644#issuecomment-2303330176 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Implement a call procedure for DeleteReachableFiles action [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3588: Implement a call procedure for DeleteReachableFiles action URL: https://github.com/apache/iceberg/issues/3588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] iceberg0.12 support removing duplicate primary key all the partition or one partition? [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3566: iceberg0.12 support removing duplicate primary key all the partition or one partition? URL: https://github.com/apache/iceberg/issues/3566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] iceberg0.12 support removing duplicate primary key all the partition or one partition? [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3566: URL: https://github.com/apache/iceberg/issues/3566#issuecomment-2303330117 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Already closed files for partition error when sorting by date due to timezone [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3564: URL: https://github.com/apache/iceberg/issues/3564#issuecomment-2303330094 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] spark: Set whether the write order of the table supports case-sensitive synchronization with sparkSQL [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3545: spark: Set whether the write order of the table supports case-sensitive synchronization with sparkSQL URL: https://github.com/apache/iceberg/pull/3545 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [WIP] Build - Ensure all CI Actions Exit Early if One Fails [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3524: [WIP] Build - Ensure all CI Actions Exit Early if One Fails URL: https://github.com/apache/iceberg/pull/3524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [WIP] Build - Ensure all CI Actions Exit Early if One Fails [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3524: URL: https://github.com/apache/iceberg/pull/3524#issuecomment-2303329962 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] In Flink, when we set the PK, the properties cannot be changed [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3521: URL: https://github.com/apache/iceberg/issues/3521#issuecomment-2303329931 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] In Flink, when we set the PK, the properties cannot be changed [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3521: In Flink, when we set the PK, the properties cannot be changed URL: https://github.com/apache/iceberg/issues/3521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] java.lang.IllegalArgumentException: schemas must exist in format v2 [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3507: URL: https://github.com/apache/iceberg/issues/3507#issuecomment-2303329900 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink streaming read from mix snapshot table [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3486: URL: https://github.com/apache/iceberg/issues/3486#issuecomment-2303329880 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Infra: add Github template for PRs [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3484: URL: https://github.com/apache/iceberg/issues/3484#issuecomment-2303329847 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Website: spec page does not show subsections in sidebar [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3475: Website: spec page does not show subsections in sidebar URL: https://github.com/apache/iceberg/issues/3475 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Website: spec page does not show subsections in sidebar [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3475: URL: https://github.com/apache/iceberg/issues/3475#issuecomment-2303329828 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Website: Hive page does not show sections in sidebar [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3474: URL: https://github.com/apache/iceberg/issues/3474#issuecomment-2303329807 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Spark overwrite issue when partition changes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #4510: URL: https://github.com/apache/iceberg/issues/4510#issuecomment-2303330477 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Spark overwrite issue when partition changes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #4510: Spark overwrite issue when partition changes URL: https://github.com/apache/iceberg/issues/4510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] push down aggregation (min/max/count) to iceberg scan [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #4046: URL: https://github.com/apache/iceberg/issues/4046#issuecomment-2303330445 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] push down aggregation (min/max/count) to iceberg scan [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #4046: push down aggregation (min/max/count) to iceberg scan URL: https://github.com/apache/iceberg/issues/4046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] Support Iceberg Metadata storage in a variety of engines [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3997: Support Iceberg Metadata storage in a variety of engines URL: https://github.com/apache/iceberg/issues/3997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [I] Linkage errors in published jars [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3958: URL: https://github.com/apache/iceberg/issues/3958#issuecomment-2303330369 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] ParquetWriter leaks memory [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3950: URL: https://github.com/apache/iceberg/issues/3950#issuecomment-2303330355 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] [Feature Request] Support for change data capture [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3941: [Feature Request] Support for change data capture URL: https://github.com/apache/iceberg/issues/3941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Docs: Add `Streaming upsert write` in flink.md [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3827: Docs: Add `Streaming upsert write` in flink.md URL: https://github.com/apache/iceberg/pull/3827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Core, Spark: Fallback when snapshot does not have schema id [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3852: URL: https://github.com/apache/iceberg/pull/3852#issuecomment-2303330290 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Docs: Add `Streaming upsert write` in flink.md [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3827: URL: https://github.com/apache/iceberg/pull/3827#issuecomment-2303330277 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] [Improvement] ParallelIterable#hasNext submit reading ManifestFile task slowly with DataTableScan#planTasks [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3741: [Improvement] ParallelIterable#hasNext submit reading ManifestFile task slowly with DataTableScan#planTasks URL: https://github.com/apache/iceberg/issues/3741 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] HiveTableOperations: update HMS transient_lastDdlTime param on snapshot writes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3670: URL: https://github.com/apache/iceberg/pull/3670#issuecomment-2303330236 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Adding MergeInto into the Spark Scala API [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3665: URL: https://github.com/apache/iceberg/issues/3665#issuecomment-2303330208 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] retain flinkCheckpointInfo when rewrite datafiles to ensure restore flink job from checkpoint success [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3647: URL: https://github.com/apache/iceberg/pull/3647#issuecomment-2303330195 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] SPARK[docs]:Add Warning information under the MERGE INTO syntax [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3644: SPARK[docs]:Add Warning information under the MERGE INTO syntax URL: https://github.com/apache/iceberg/pull/3644 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Support for S3FileIO in hadoop catalog [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3546: Support for S3FileIO in hadoop catalog URL: https://github.com/apache/iceberg/issues/3546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] BaseDataReader is not properly closed during query by spark sql [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3636: URL: https://github.com/apache/iceberg/issues/3636#issuecomment-2303330160 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Implement a call procedure for DeleteReachableFiles action [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3588: URL: https://github.com/apache/iceberg/issues/3588#issuecomment-2303330133 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] can not change nullable via Spark SQL sometimes [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3617: URL: https://github.com/apache/iceberg/issues/3617#issuecomment-2303330149 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Already closed files for partition error when sorting by date due to timezone [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3564: Already closed files for partition error when sorting by date due to timezone URL: https://github.com/apache/iceberg/issues/3564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Spark: Rewrite if no commit succeeded throw no commit succeed exception [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed pull request #3563: Spark: Rewrite if no commit succeeded throw no commit succeed exception URL: https://github.com/apache/iceberg/pull/3563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] Support for S3FileIO in hadoop catalog [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3546: URL: https://github.com/apache/iceberg/issues/3546#issuecomment-2303330051 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Spark: Rewrite if no commit succeeded throw no commit succeed exception [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3563: URL: https://github.com/apache/iceberg/pull/3563#issuecomment-2303330075 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] spark: Set whether the write order of the table supports case-sensitive synchronization with sparkSQL [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on PR #3545: URL: https://github.com/apache/iceberg/pull/3545#issuecomment-2303330032 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Iceberg does not support drop namespace cascade [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3541: URL: https://github.com/apache/iceberg/issues/3541#issuecomment-2303330013 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Iceberg does not support drop namespace cascade [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3541: Iceberg does not support drop namespace cascade URL: https://github.com/apache/iceberg/issues/3541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] Spark: Pushdown Filters / Improve Performance when Importing File Based Tables [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3532: URL: https://github.com/apache/iceberg/issues/3532#issuecomment-2303329972 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Does iceberg plan to implement SQL to optimize the layout? [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3537: Does iceberg plan to implement SQL to optimize the layout? URL: https://github.com/apache/iceberg/issues/3537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Spark: Pushdown Filters / Improve Performance when Importing File Based Tables [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3532: Spark: Pushdown Filters / Improve Performance when Importing File Based Tables URL: https://github.com/apache/iceberg/issues/3532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Set the write order of the table to be case sensitive, use WRITE ORDERED BY? Should the default value be changed to false [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] commented on issue #3523: URL: https://github.com/apache/iceberg/issues/3523#issuecomment-2303329946 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Set the write order of the table to be case sensitive, use WRITE ORDERED BY? Should the default value be changed to false [iceberg]

2024-08-21 Thread via GitHub
github-actions[bot] closed issue #3523: Set the write order of the table to be case sensitive, use WRITE ORDERED BY? Should the default value be changed to false URL: https://github.com/apache/iceberg/issues/3523 -- This is an automated message from the Apache Git Service. To respond to the

  1   2   3   >