Re: [I] ClassNotFoundException when using the flinksql to write iceberg table [iceberg]

xzwDavid closed issue #8947: ClassNotFoundException when using the flinksql to write iceberg table URL: https://github.com/apache/iceberg/issues/8947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] Build: Bump actions/checkout from 3 to 4 [iceberg]

dependabot[bot] opened a new pull request, #9474: URL: https://github.com/apache/iceberg/pull/9474 Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. Release notes Sourced from https://github.com/actions/checkout/releases";>actions/checkout's releases.

[PR] Build: Bump actions/setup-python from 4 to 5 [iceberg]

dependabot[bot] opened a new pull request, #9473: URL: https://github.com/apache/iceberg/pull/9473 Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. Release notes Sourced from https://github.com/actions/setup-python/releases";>actions/setup-python's

Re: [PR] Apply Name mapping [iceberg-python]

syun64 commented on code in PR #219: URL: https://github.com/apache/iceberg-python/pull/219#discussion_r1451652305 ## pyiceberg/io/pyarrow.py: ## @@ -698,77 +708,143 @@ def before_field(self, field: pa.Field) -> None: def after_field(self, field: pa.Field) -> None:

Re: [PR] Add SqlCatalog _commit_table support [iceberg-python]

syun64 commented on code in PR #265: URL: https://github.com/apache/iceberg-python/pull/265#discussion_r1451651003 ## pyiceberg/catalog/sql.py: ## @@ -268,16 +269,32 @@ def drop_table(self, identifier: Union[str, Identifier]) -> None: identifier_tuple = self.identifier

Re: [I] Support MOR CDC view [iceberg]

coolderli commented on issue #8975: URL: https://github.com/apache/iceberg/issues/8975#issuecomment-1890840629 @flyrain @puchengy any update for supporting this? Is there any plan for supporting Flink streaming read? Thanks. -- This is an automated message from the Apache Git Service. To

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.10 [iceberg]

dependabot[bot] closed pull request #9392: Build: Bump slf4j from 1.7.36 to 2.0.10 URL: https://github.com/apache/iceberg/pull/9392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[PR] Build: Bump software.amazon.awssdk:bom from 2.22.12 to 2.23.2 [iceberg]

dependabot[bot] opened a new pull request, #9471: URL: https://github.com/apache/iceberg/pull/9471 Bumps software.amazon.awssdk:bom from 2.22.12 to 2.23.2. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softwar

[PR] Build: Bump org.springframework:spring-web from 5.3.30 to 6.1.3 [iceberg]

dependabot[bot] opened a new pull request, #9470: URL: https://github.com/apache/iceberg/pull/9470 Bumps [org.springframework:spring-web](https://github.com/spring-projects/spring-framework) from 5.3.30 to 6.1.3. Release notes Sourced from https://github.com/spring-projects/spring

Re: [PR] Build: Bump com.esotericsoftware:kryo from 4.0.2 to 5.5.0 [iceberg]

dependabot[bot] commented on PR #8793: URL: https://github.com/apache/iceberg/pull/8793#issuecomment-1890839948 Superseded by #9469. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump com.esotericsoftware:kryo from 4.0.2 to 5.5.0 [iceberg]

dependabot[bot] closed pull request #8793: Build: Bump com.esotericsoftware:kryo from 4.0.2 to 5.5.0 URL: https://github.com/apache/iceberg/pull/8793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] Build: Bump com.esotericsoftware:kryo from 4.0.2 to 5.6.0 [iceberg]

dependabot[bot] opened a new pull request, #9469: URL: https://github.com/apache/iceberg/pull/9469 Bumps [com.esotericsoftware:kryo](https://github.com/EsotericSoftware/kryo) from 4.0.2 to 5.6.0. Release notes Sourced from https://github.com/EsotericSoftware/kryo/releases";>com.eso

[PR] Build: Bump slf4j from 1.7.36 to 2.0.11 [iceberg]

dependabot[bot] opened a new pull request, #9472: URL: https://github.com/apache/iceberg/pull/9472 Bumps `slf4j` from 1.7.36 to 2.0.11. Updates `org.slf4j:slf4j-api` from 1.7.36 to 2.0.11 Updates `org.slf4j:slf4j-simple` from 1.7.36 to 2.0.11 Dependabot will resolve any c

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.10 [iceberg]

dependabot[bot] commented on PR #9392: URL: https://github.com/apache/iceberg/pull/9392#issuecomment-1890840016 Superseded by #9472. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.35.0 [iceberg]

dependabot[bot] opened a new pull request, #9468: URL: https://github.com/apache/iceberg/pull/9468 Bumps [com.palantir.baseline:gradle-baseline-java](https://github.com/palantir/gradle-baseline) from 4.42.0 to 5.35.0. Release notes Sourced from https://github.com/palantir/gradle-b

[PR] Build: Bump nessie from 0.76.0 to 0.76.2 [iceberg]

dependabot[bot] opened a new pull request, #9467: URL: https://github.com/apache/iceberg/pull/9467 Bumps `nessie` from 0.76.0 to 0.76.2. Updates `org.projectnessie.nessie:nessie-client` from 0.76.0 to 0.76.2 Updates `org.projectnessie.nessie:nessie-jaxrs-testextension` from 0.76.0 t

Re: [PR] Add SqlCatalog _commit_table support [iceberg-python]

syun64 commented on code in PR #265: URL: https://github.com/apache/iceberg-python/pull/265#discussion_r1451648309 ## tests/catalog/test_sql.py: ## @@ -87,6 +88,20 @@ def catalog_sqlite(warehouse: Path) -> Generator[SqlCatalog, None, None]: catalog.destroy_tables() +@p

Re: [PR] Apply Name mapping [iceberg-python]

HonahX commented on code in PR #219: URL: https://github.com/apache/iceberg-python/pull/219#discussion_r1451632412 ## pyiceberg/io/pyarrow.py: ## @@ -698,77 +708,143 @@ def before_field(self, field: pa.Field) -> None: def after_field(self, field: pa.Field) -> None:

Re: [I] Dynamic partition pruning filters should be applied before invoking Table::planTasks in IcebergInputFormat [iceberg]

github-actions[bot] commented on issue #6726: URL: https://github.com/apache/iceberg/issues/6726#issuecomment-1890802436 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Dynamic partition pruning filters should be applied before invoking Table::planTasks in IcebergInputFormat [iceberg]

github-actions[bot] closed issue #6726: Dynamic partition pruning filters should be applied before invoking Table::planTasks in IcebergInputFormat URL: https://github.com/apache/iceberg/issues/6726 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Mysql CDC -- Flink -- Iceberg(Minio) Data is not reaching to Minio bucket location [iceberg]

github-actions[bot] commented on issue #6734: URL: https://github.com/apache/iceberg/issues/6734#issuecomment-1890802430 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Merge-on-read vs copy-on-write behavior during merge into [iceberg]

github-actions[bot] closed issue #6928: Merge-on-read vs copy-on-write behavior during merge into URL: https://github.com/apache/iceberg/issues/6928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

javrasya commented on PR #9464: URL: https://github.com/apache/iceberg/pull/9464#issuecomment-1890799411 Another idea is to still use original `writeUTF` since that is covering the most, but write it as chunks. The biggest character can be 3 bytes according to the `writeUTF` function. If we

Re: [PR] Add SqlCatalog _commit_table support [iceberg-python]

HonahX commented on code in PR #265: URL: https://github.com/apache/iceberg-python/pull/265#discussion_r1451397678 ## tests/catalog/test_sql.py: ## @@ -87,6 +88,20 @@ def catalog_sqlite(warehouse: Path) -> Generator[SqlCatalog, None, None]: catalog.destroy_tables() +@p

Re: [PR] Add `iceberg-bom` artifact [iceberg]

danielcweeks commented on PR #8065: URL: https://github.com/apache/iceberg/pull/8065#issuecomment-1890769619 I just ran the target to build the pom and still see spark and flink included, which I believe should be excluded. I saw comments about adding Scala versions but I don't think that

Re: [I] Failed to assign splits due to the serialized split size [iceberg]

javrasya commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1890599978 Tried `[rewrite_data_files](https://iceberg.apache.org/docs/1.4.0/spark-procedures/#rewrite_data_files)` via Spark, not really sure if it would do the same with `RewriteDataFilesAc

Re: [PR] Spark 3.5: Migrate tests that depend on SparkDistributedDataScanTestBase to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9416: URL: https://github.com/apache/iceberg/pull/9416#discussion_r1451573134 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/TestSparkDistributedDataScanReporting.java: ## @@ -21,41 +21,38 @@ import static org.apache.iceberg.PlanningM

Re: [PR] Spark 3.5: Migrate tests that depend on SparkDistributedDataScanTestBase to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9416: URL: https://github.com/apache/iceberg/pull/9416#discussion_r1451573049 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/SparkDistributedDataScanTestBase.java: ## @@ -21,46 +21,41 @@ import static org.apache.iceberg.PlanningMode.D

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451570358 ## mr/src/test/java/org/apache/iceberg/mr/TestHelper.java: ## @@ -122,7 +144,7 @@ public DataFile writeFile(StructLike partition, List records) throws IOE }

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451568936 ## data/src/test/java/org/apache/iceberg/data/TestGenericReaderDeletes.java: ## @@ -18,23 +18,25 @@ */ package org.apache.iceberg.data; +import static org.ass

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451568840 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceReaderDeletes.java: ## @@ -44,27 +43,18 @@ import org.apache.iceberg.relocated

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451568721 ## mr/src/test/java/org/apache/iceberg/mr/TestInputFormatReaderDeletes.java: ## @@ -62,24 +67,20 @@ public static Object[][] parameters() { }; } - @Befo

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451568818 ## mr/src/test/java/org/apache/iceberg/mr/TestHelper.java: ## @@ -122,7 +144,7 @@ public DataFile writeFile(StructLike partition, List records) throws IOE }

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451568424 ## mr/src/test/java/org/apache/iceberg/mr/TestInputFormatReaderDeletes.java: ## @@ -62,24 +67,20 @@ public static Object[][] parameters() { }; } - @Befo

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

nastra commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451549557 ## data/src/test/java/org/apache/iceberg/data/TestGenericReaderDeletes.java: ## @@ -18,23 +18,25 @@ */ package org.apache.iceberg.data; +import static org.assertj.c

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

nastra commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451548608 ## data/src/test/java/org/apache/iceberg/data/TestGenericReaderDeletes.java: ## @@ -18,23 +18,25 @@ */ package org.apache.iceberg.data; +import static org.assertj.c

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

nastra commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451547470 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceReaderDeletes.java: ## @@ -44,27 +43,18 @@ import org.apache.iceberg.relocated.com.g

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

nastra commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451546107 ## mr/src/test/java/org/apache/iceberg/mr/TestHelper.java: ## @@ -122,7 +144,7 @@ public DataFile writeFile(StructLike partition, List records) throws IOE } pri

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

nastra commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451546217 ## mr/src/test/java/org/apache/iceberg/mr/TestHelper.java: ## @@ -122,7 +144,7 @@ public DataFile writeFile(StructLike partition, List records) throws IOE } pri

Re: [PR] Spark 3.5: Migrate DeleteReadTests and its subclasses to JUnit5 [iceberg]

nastra commented on code in PR #9382: URL: https://github.com/apache/iceberg/pull/9382#discussion_r1451545549 ## mr/src/test/java/org/apache/iceberg/mr/TestInputFormatReaderDeletes.java: ## @@ -62,24 +67,20 @@ public static Object[][] parameters() { }; } - @Before +

Re: [PR] Add `iceberg-bom` artifact [iceberg]

nastra commented on PR #8065: URL: https://github.com/apache/iceberg/pull/8065#issuecomment-1890477020 I agree that we should get this into 1.5.0. @snazy could you rebase this please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Flink: Added error handling and default logic for Flink version detection [iceberg]

nastra commented on code in PR #9452: URL: https://github.com/apache/iceberg/pull/9452#discussion_r1451533014 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/util/FlinkPackage.java: ## @@ -19,15 +19,31 @@ package org.apache.iceberg.flink.util; import org.apache.f

Re: [PR] Flink: Added error handling and default logic for Flink version detection [iceberg]

nastra commented on code in PR #9452: URL: https://github.com/apache/iceberg/pull/9452#discussion_r1451531936 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/util/FlinkPackage.java: ## @@ -19,15 +19,31 @@ package org.apache.iceberg.flink.util; import org.apache.f

Re: [I] Facing warning when starting spark-sql in EMR using Glue Catalog [iceberg]

nastra commented on issue #8544: URL: https://github.com/apache/iceberg/issues/8544#issuecomment-1890470170 @imonteroq I would suggest to talk to the EMR team as that is a vendor-specific issue -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

lisirrx commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1451531023 ## core/src/test/java/org/apache/iceberg/TestManifestReader.java: ## @@ -32,17 +32,15 @@ import org.apache.iceberg.types.Types; import org.assertj.core.api.Assertions

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

lisirrx closed pull request #9217: Core: Create JUnit5 version of TableTestBase URL: https://github.com/apache/iceberg/pull/9217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

nastra merged PR #9380: URL: https://github.com/apache/iceberg/pull/9380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Flink: Migrate subclasses of FlinkCatalogTestBase to JUnit5 [iceberg]

nastra merged PR #9381: URL: https://github.com/apache/iceberg/pull/9381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1451526875 ## core/src/test/java/org/apache/iceberg/TestManifestReader.java: ## @@ -32,17 +32,15 @@ import org.apache.iceberg.types.Types; import org.assertj.core.api.Assertions;

Re: [I] Discussion: Rethink `PrimitiveLiteral`. [iceberg-rust]

liurenjie1024 commented on issue #159: URL: https://github.com/apache/iceberg-rust/issues/159#issuecomment-1890430482 > I don't think the BoundExpression is something that a user would ever use as it is internal. > > let literal = BoundLiteral::builder.with_literal(...).build(); >

Re: [PR] Spec: add multi-arg transform support [iceberg]

advancedxy commented on code in PR #8579: URL: https://github.com/apache/iceberg/pull/8579#discussion_r1451452956 ## format/spec.md: ## @@ -329,19 +329,35 @@ The `void` transform may be used to replace the transform in an existing partiti Bucket Transform Details -Buc

Re: [PR] Spec: add multi-arg transform support [iceberg]

advancedxy commented on code in PR #8579: URL: https://github.com/apache/iceberg/pull/8579#discussion_r1451447151 ## format/spec.md: ## @@ -1060,6 +1076,14 @@ The types below are not currently valid for bucketing, and so are not hashed. Ho | **`float`**| `hashLong(doub

[I] access failed from host to iceberg container [iceberg]

vagetablechicken opened a new issue, #9465: URL: https://github.com/apache/iceberg/issues/9465 hi, I'm doing some tests, based on spark quickstart . I start up it, and try to connect iceberg from outside(host->iceberg_in_containers). And it work

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

javrasya commented on PR #9464: URL: https://github.com/apache/iceberg/pull/9464#issuecomment-1890390386 Good catches @pvary , thank you. What if we get full inspiration from writeUTF and have our own writer but supports longer JSON. Btw, the reason why it limits the size to be 65K max beca

Re: [PR] support python 3.12 [iceberg-python]

cclauss commented on PR #254: URL: https://github.com/apache/iceberg-python/pull/254#issuecomment-1890383240 Similar to #35 @Fokko Can you please approve the workflow run so we can see the results of the tests? -- This is an automated message from the Apache Git Service. To respond

Re: [PR] support python 3.12 [iceberg-python]

cclauss commented on code in PR #254: URL: https://github.com/apache/iceberg-python/pull/254#discussion_r1451394303 ## pyproject.toml: ## @@ -70,6 +71,9 @@ adlfs = { version = ">=2023.1.0,<2024.1.0", optional = true } gcsfs = { version = ">=2023.1.0,<2024.1.0", optional = true

Re: [PR] support python 3.12 [iceberg-python]

cclauss commented on code in PR #254: URL: https://github.com/apache/iceberg-python/pull/254#discussion_r1451394047 ## pyproject.toml: ## @@ -29,7 +29,8 @@ classifiers = [ "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Programming Lan