Re: [I] Failed to assign splits due to the serialized split size [iceberg]

pvary commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1879580902 @javrasya: Table with 14 columns should not cause any issues. The default stats also could not cause issues. I made a mistake reading the code, and combined splits also could not

Re: [PR] Flink: Watermark read options [iceberg]

pvary commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1443647987 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceSql.java: ## @@ -39,4 +62,76 @@ public void before() throws IOException { .ge

Re: [I] Add the homepage to the `Cargo.toml` [iceberg-rust]

Xuanwo commented on issue #154: URL: https://github.com/apache/iceberg-rust/issues/154#issuecomment-1879495868 Marked as good first issues for new contributors to get started 🥰 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] Consider allows catalog.warehouse for Spark Hive Catalogs [iceberg]

github-actions[bot] closed issue #6863: Consider allows catalog.warehouse for Spark Hive Catalogs URL: https://github.com/apache/iceberg/issues/6863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] timestamp-without-timezone related settings ignored when adding a new timestamp column in spark [iceberg]

github-actions[bot] closed issue #6860: timestamp-without-timezone related settings ignored when adding a new timestamp column in spark URL: https://github.com/apache/iceberg/issues/6860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] Schema of the Underlying data files [iceberg]

github-actions[bot] commented on issue #6858: URL: https://github.com/apache/iceberg/issues/6858#issuecomment-1879443828 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Schema of the Underlying data files [iceberg]

github-actions[bot] closed issue #6858: Schema of the Underlying data files URL: https://github.com/apache/iceberg/issues/6858 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [I] timestamp-without-timezone related settings ignored when adding a new timestamp column in spark [iceberg]

github-actions[bot] commented on issue #6860: URL: https://github.com/apache/iceberg/issues/6860#issuecomment-1879443764 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Consider allows catalog.warehouse for Spark Hive Catalogs [iceberg]

github-actions[bot] commented on issue #6863: URL: https://github.com/apache/iceberg/issues/6863#issuecomment-1879443678 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Table Corruption iceberg [iceberg]

github-actions[bot] closed issue #6989: Table Corruption iceberg URL: https://github.com/apache/iceberg/issues/6989 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [I] Table Corruption iceberg [iceberg]

github-actions[bot] commented on issue #6989: URL: https://github.com/apache/iceberg/issues/6989#issuecomment-1879443537 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support partitioned writes [iceberg-python]

jqin61 commented on issue #208: URL: https://github.com/apache/iceberg-python/issues/208#issuecomment-1879332455 Hi @Fokko and Iceberg community, I and @syun64 are continuing working on testing the write capability in [Write support pr](https://github.com/apache/iceberg-python/pull/41). We

Re: [I] [BUG] parquet col array with null value is read in as empty list but should be none [iceberg-python]

Fokko commented on issue #251: URL: https://github.com/apache/iceberg-python/issues/251#issuecomment-1879323375 I've found the issue. We don't respect the null count when fetching the array through the accessor: https://github.com/apache/iceberg-python/assets/1134248/ba5225cc-1102-408

Re: [PR] Fix lint tests failed in main [iceberg-python]

Fokko merged PR #253: URL: https://github.com/apache/iceberg-python/pull/253 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Replace black by Ruff Formatter [iceberg-python]

hussein-awala commented on PR #127: URL: https://github.com/apache/iceberg-python/pull/127#issuecomment-1879277861 We need to merge https://github.com/apache/iceberg-python/pull/253 to fix the failed tests in main. -- This is an automated message from the Apache Git Service. To respond to

[PR] Fix lint tests failed in main [iceberg-python]

hussein-awala opened a new pull request, #253: URL: https://github.com/apache/iceberg-python/pull/253 After merging #127, it looks like another PR merged today needs to be formatted by the new linter. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

bitsondatadev commented on PR #8919: URL: https://github.com/apache/iceberg/pull/8919#issuecomment-1879255453 @Fokko I updated after your comments! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] How can we build self-defined iceberg-spark-runtime jar package? [iceberg]

gaoshihang closed issue #9425: How can we build self-defined iceberg-spark-runtime jar package? URL: https://github.com/apache/iceberg/issues/9425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

bitsondatadev commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443361277 ## site/docs/community.md: ## @@ -40,13 +40,13 @@ Issues are tracked in GitHub: ## Slack -We use the [Apache Iceberg workspace](https://apache-iceberg.slack.

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

bitsondatadev commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443359408 ## site/nav.yml: ## @@ -0,0 +1,49 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

amogh-jahagirdar commented on PR #9414: URL: https://github.com/apache/iceberg/pull/9414#issuecomment-1879243149 > That line of reasoning makes me think that any change (even comments) should just produce a new version of the view. Yes, that's the approach in this PR currently. A new

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

bitsondatadev commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443356487 ## site/docs/community.md: ## @@ -40,13 +40,13 @@ Issues are tracked in GitHub: ## Slack -We use the [Apache Iceberg workspace](https://apache-iceberg.slack.

Re: [PR] Shift site build to use monorepo and gh-pages [iceberg]

Fokko commented on code in PR #8919: URL: https://github.com/apache/iceberg/pull/8919#discussion_r1443340509 ## site/docs/community.md: ## @@ -40,13 +40,13 @@ Issues are tracked in GitHub: ## Slack -We use the [Apache Iceberg workspace](https://apache-iceberg.slack.com/) on

Re: [PR] Replace black by Ruff Formatter [iceberg-python]

Fokko merged PR #127: URL: https://github.com/apache/iceberg-python/pull/127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[I] Add the homepage to the `Cargo.toml` [iceberg-rust]

Fokko opened a new issue, #154: URL: https://github.com/apache/iceberg-rust/issues/154 Looks like we can add the homepage as well: https://doc.rust-lang.org/cargo/reference/manifest.html#the-homepage-field To our newly created https://rust.iceberg.apache.org/ -- This is an auto

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

Fokko commented on code in PR #9063: URL: https://github.com/apache/iceberg/pull/9063#discussion_r1443316711 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -108,6 +110,113 @@ public ParquetValueReader struct( } } + private clas

Re: [PR] Adding Snowflake's public documentation [iceberg-docs]

scottteal commented on PR #297: URL: https://github.com/apache/iceberg-docs/pull/297#issuecomment-1879185914 @Fokko I think I've now resolved the conflicts. I tried to preserve the ordering to match the current website as much as possible, although it's still unclear to me what the ordering

[I] How can we build self-defined iceberg-spark-runtime jar package? [iceberg]

gaoshihang opened a new issue, #9425: URL: https://github.com/apache/iceberg/issues/9425 ### Query engine Spark ### Question How can we build self-defined iceberg-spark-runtime jar package? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Adding Snowflake's public documentation [iceberg-docs]

Fokko commented on PR #297: URL: https://github.com/apache/iceberg-docs/pull/297#issuecomment-1879159046 @scottteal sorry for the late reply here, could you fix the conflicts once more? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] API: Fix day partition transform result type [iceberg]

tdcmeehan commented on PR #9345: URL: https://github.com/apache/iceberg/pull/9345#issuecomment-1879158054 It seems the consensus is it's more straightforward to keep this as `int`, so I will close this. Thanks folks! -- This is an automated message from the Apache Git Service. To respond

Re: [PR] API: Fix day partition transform result type [iceberg]

tdcmeehan closed pull request #9345: API: Fix day partition transform result type URL: https://github.com/apache/iceberg/pull/9345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] docs: Add link for iceberg rust [iceberg-docs]

Fokko merged PR #300: URL: https://github.com/apache/iceberg-docs/pull/300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix PR link in 1.4.3 release notes [iceberg-docs]

Fokko merged PR #299: URL: https://github.com/apache/iceberg-docs/pull/299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec [iceberg-docs]

Fokko merged PR #305: URL: https://github.com/apache/iceberg-docs/pull/305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec in 1.4.2 [iceberg-docs]

Fokko merged PR #303: URL: https://github.com/apache/iceberg-docs/pull/303 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec in 1.4.3 [iceberg-docs]

Fokko merged PR #304: URL: https://github.com/apache/iceberg-docs/pull/304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec in 1.4.1 [iceberg-docs]

Fokko merged PR #302: URL: https://github.com/apache/iceberg-docs/pull/302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Fix parquet default compression codec in 1.4.0 [iceberg-docs]

Fokko merged PR #301: URL: https://github.com/apache/iceberg-docs/pull/301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Failed to assign splits due to the serialized split size [iceberg]

javrasya commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1879135064 @pvary I wasn't aware of `splitOpenFileCost`, thank you for sharing that. The way how I did it is that I introduced my own SplitAssignerFactory and SplitAssigner and pass tha

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443275368 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -45,14 +47,17 @@ case class ResolveViews(spark: SparkSessi

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443227383 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -635,6 +633,51 @@ private Catalog tableCatalog() { return Spa

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443226670 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -635,6 +633,51 @@ private Catalog tableCatalog() { return Spa

Re: [I] Spark: Session level Iceberg table config defaults [iceberg]

kuldeepsinghchauhan commented on issue #2270: URL: https://github.com/apache/iceberg/issues/2270#issuecomment-1879091855 This is a very critical feature, In an actual production scenario where we are planning to rollout the iceberg format for operations and analytics user. we are currently

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

amogh-jahagirdar commented on code in PR #9063: URL: https://github.com/apache/iceberg/pull/9063#discussion_r1443148385 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -108,6 +110,113 @@ public ParquetValueReader struct( } } + p

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443156571 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DropV2ViewExec.scala: ## @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443153917 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -122,7 +128,27 @@ class Iceber

Re: [PR] Spark: Support dropping views [iceberg]

rdblue commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1443152002 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -122,7 +128,27 @@ class Iceber

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

amogh-jahagirdar commented on code in PR #9063: URL: https://github.com/apache/iceberg/pull/9063#discussion_r1443148176 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -108,6 +110,113 @@ public ParquetValueReader struct( } } + p

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

rdblue commented on PR #9414: URL: https://github.com/apache/iceberg/pull/9414#issuecomment-1879017514 Oops, I didn't intend to close this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Flink: Watermark read options [iceberg]

stevenzwu commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1443084956 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkReadOptions.java: ## @@ -109,4 +110,13 @@ private FlinkReadOptions() {} public static final String

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

rdblue commented on PR #9414: URL: https://github.com/apache/iceberg/pull/9414#issuecomment-1878993066 @amogh-jahagirdar, I agree with not adding a `with` method to modify a schema. If we can avoid making that kind of API addition I would prefer it. I'm also not sure that we want to m

Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]

rdblue closed pull request #9414: API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs URL: https://github.com/apache/iceberg/pull/9414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443109201 ## core/src/test/java/org/apache/iceberg/TestManifestReader.java: ## @@ -32,17 +32,15 @@ import org.apache.iceberg.types.Types; import org.assertj.core.api.Assertions;

[PR] Core: Add JUnit5 version of TableTestBase [iceberg]

nastra opened a new pull request, #9424: URL: https://github.com/apache/iceberg/pull/9424 This fixes https://github.com/apache/iceberg/issues/9073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443105597 ## core/src/test/java/org/apache/iceberg/TestManifestReader.java: ## @@ -32,17 +32,15 @@ import org.apache.iceberg.types.Types; import org.assertj.core.api.Assertions;

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

manuzhang commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443085007 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

manuzhang commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443085007 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp

Re: [PR] Spark 3.4: Add support for reading Iceberg views [iceberg]

amogh-jahagirdar merged PR #9422: URL: https://github.com/apache/iceberg/pull/9422 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Spark 3.4: Add support for reading Iceberg views [iceberg]

amogh-jahagirdar commented on PR #9422: URL: https://github.com/apache/iceberg/pull/9422#issuecomment-1878930402 Since this was a clean backport. I'll go ahead and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra merged PR #9417: URL: https://github.com/apache/iceberg/pull/9417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Adding Junit 5 conversion and AssertJ style for TestFlinkCatalogTable… [iceberg]

vinitpatni commented on PR #9381: URL: https://github.com/apache/iceberg/pull/9381#issuecomment-1878891716 I have done changes in TestStreamScanSql locally but running testcases on my local machine is taking lot of time. It is running indefinitely for this class. @nastra Do you have any ide

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443033477 ## core/src/test/java/org/apache/iceberg/TestCreateSnapshotEvent.java: ## @@ -40,76 +41,76 @@ public TestCreateSnapshotEvent(int formatVersion) { Listeners.register

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

manuzhang commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443027055 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp

Re: [PR] Core: Create JUnit5 version of TableTestBase [iceberg]

nastra commented on code in PR #9217: URL: https://github.com/apache/iceberg/pull/9217#discussion_r1443020124 ## core/src/test/java/org/apache/iceberg/TestCreateSnapshotEvent.java: ## @@ -40,76 +41,76 @@ public TestCreateSnapshotEvent(int formatVersion) { Listeners.register

Re: [PR] Parquet: Support reading INT96 column in row group filter [iceberg]

nastra commented on code in PR #8988: URL: https://github.com/apache/iceberg/pull/8988#discussion_r1443018102 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -2181,20 +2181,28 @@ public void testTableWithInt96Timestamp()

Re: [PR] Flink 1.17: Create JUnit5 version of TestFlinkScan [iceberg]

cgpoh commented on PR #9185: URL: https://github.com/apache/iceberg/pull/9185#issuecomment-1878844200 > @cgpoh can you please port these changes to other Flink versions? Sure. Will try to port these changes next week. -- This is an automated message from the Apache Git Service. To r

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

chinmay-bhat commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1442989187 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/PartitionedWritesTestBase.java: ## @@ -18,53 +18,49 @@ */ package org.apache.iceberg.spark.sql;

Re: [I] Failed to assign splits due to the serialized split size [iceberg]

pvary commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1878817631 @javrasya: This is table should not be too wide, and the statistics should be limited as well (unless you did some specific tweaking there). My best guess is your first suggestion:

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

chinmay-bhat commented on PR #9417: URL: https://github.com/apache/iceberg/pull/9417#issuecomment-1878814101 Thanks for the review, I've updated the PR with the suggestions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] Fix parquet default compression codec [iceberg-docs]

manuzhang opened a new pull request, #305: URL: https://github.com/apache/iceberg-docs/pull/305 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Spark 3.5: Set log level to WARN for rewrite task failure with partial progress [iceberg]

manuzhang commented on code in PR #9400: URL: https://github.com/apache/iceberg/pull/9400#discussion_r1442954829 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java: ## @@ -345,7 +345,7 @@ private Result doExecuteWithPartialProgres

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1442885901 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/PartitionedWritesTestBase.java: ## @@ -18,53 +18,49 @@ */ package org.apache.iceberg.spark.sql; +impo

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on PR #9417: URL: https://github.com/apache/iceberg/pull/9417#issuecomment-1878660248 @chinmay-bhat this is almost ready to go, just a few minor things to address -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442872510 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestFunctionCatalog.java: ## @@ -27,107 +30,106 @@ import org.apache.spark.sql.connector.catalog.FunctionCata

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442867841 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestFunctionCatalog.java: ## @@ -27,107 +30,106 @@ import org.apache.spark.sql.connector.catalog.FunctionCata

Re: [I] Failed to assign splits due to the serialized split size [iceberg]

javrasya commented on issue #9410: URL: https://github.com/apache/iceberg/issues/9410#issuecomment-1878646665 No idea tbh since I run into this on production and there I don't have the ability to go deep and debug. I wouldn't say it is a wide table in terms of number of columns and I

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442861281 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/TestTableSerialization.java: ## @@ -36,29 +39,23 @@ import org.apache.iceberg.relocated.com.google.common.collect.L

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442860998 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/TestTableSerialization.java: ## @@ -36,29 +39,23 @@ import org.apache.iceberg.relocated.com.google.common.collect.L

[PR] Arrow: Use case instead of wrapping a map/list [iceberg-python]

Fokko opened a new pull request, #252: URL: https://github.com/apache/iceberg-python/pull/252 Wrapping the list seems to introduce an odd behavior where `null` values are converted to an empty list `[]`. Resolves #251 -- This is an automated message from the Apache Git Service. To

Re: [PR] Spark 3.5: Remaining tests migrated to Junit5 [iceberg]

nastra commented on code in PR #9417: URL: https://github.com/apache/iceberg/pull/9417#discussion_r1442849817 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestChangelogIterator.java: ## @@ -196,10 +196,10 @@ public void testUpdatedRowsWithDuplication() { Itera

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.31.0 [iceberg]

nastra closed pull request #9041: Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.31.0 URL: https://github.com/apache/iceberg/pull/9041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.31.0 [iceberg]

dependabot[bot] commented on PR #9041: URL: https://github.com/apache/iceberg/pull/9041#issuecomment-1878626952 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Adding Junit 5 conversion and AssertJ style for TestFlinkCatalogTable… [iceberg]

nastra commented on PR #9381: URL: https://github.com/apache/iceberg/pull/9381#issuecomment-1878626062 @vinitpatni the issue with `GenericAppenderHelper` has been addressed in https://github.com/apache/iceberg/pull/9185, so this class should now be usable with JUnit4 + JUni5 -- This is a

Re: [PR] Nessie: Strip trailing slash for warehouse location [iceberg]

nastra merged PR #9415: URL: https://github.com/apache/iceberg/pull/9415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Create JUnit5-version of TestFlinkScan [iceberg]

nastra closed issue #9077: Create JUnit5-version of TestFlinkScan URL: https://github.com/apache/iceberg/issues/9077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Flink 1.17: Create JUnit5 version of TestFlinkScan [iceberg]

nastra merged PR #9185: URL: https://github.com/apache/iceberg/pull/9185 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Flink: Create JUnit5 version of TestFlinkScan [iceberg]

nastra commented on PR #9185: URL: https://github.com/apache/iceberg/pull/9185#issuecomment-1878559904 @cgpoh can you please port these changes to other Flink versions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.2.1 [iceberg]

nastra closed pull request #9371: Build: Bump spring-boot from 2.5.4 to 3.2.1 URL: https://github.com/apache/iceberg/pull/9371 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.2.1 [iceberg]

nastra commented on PR #9371: URL: https://github.com/apache/iceberg/pull/9371#issuecomment-1878558251 Looks like `iceberg-aliyun` uses this for testing. I don't think we can upgrade to the newer version, since that requires JDK 11 -- This is an automated message from the Apache Git Servi

Re: [PR] Build: Bump spring-boot from 2.5.4 to 3.2.1 [iceberg]

dependabot[bot] commented on PR #9371: URL: https://github.com/apache/iceberg/pull/9371#issuecomment-1878558298 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.23.0 to 2.24.0 [iceberg]

nastra merged PR #9369: URL: https://github.com/apache/iceberg/pull/9369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Bump jetty from 9.4.53.v20231009 to 11.0.19 [iceberg]

dependabot[bot] commented on PR #9370: URL: https://github.com/apache/iceberg/pull/9370#issuecomment-1878554815 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump jetty from 9.4.53.v20231009 to 11.0.19 [iceberg]

nastra closed pull request #9370: Build: Bump jetty from 9.4.53.v20231009 to 11.0.19 URL: https://github.com/apache/iceberg/pull/9370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Spark: Support dropping views [iceberg]

nastra commented on code in PR #9421: URL: https://github.com/apache/iceberg/pull/9421#discussion_r1442789176 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveViews.scala: ## @@ -111,8 +116,8 @@ case class ResolveViews(spark: SparkSessi

[PR] AWS: Add Option to don't write non current columns in glue schema closes #7584 [iceberg]

Raphael-Vignes opened a new pull request, #9420: URL: https://github.com/apache/iceberg/pull/9420 This PR aims to close this [issue](https://github.com/apache/iceberg/issues/7584) (and would resolve this [issue](https://github.com/apache/iceberg/issues/6340) too. We want to provide a

Re: [I] Allow snapshotting iceberg table (create new table based on certain Iceberg snapshot) [iceberg]

YehorKrivokon commented on issue #2481: URL: https://github.com/apache/iceberg/issues/2481#issuecomment-1878533755 Hi all, I'm trying to use Iceberg with Spark and SparkCatalog and I've a repro of this issue. Is there any workaround to work with snapshots? Thank you. -- This

Re: [PR] open-api: Use openapi-generator-gradle-plugin for validating specification [iceberg]

Fokko commented on code in PR #9344: URL: https://github.com/apache/iceberg/pull/9344#discussion_r1442774264 ## open-api/rest-catalog-open-api.yaml: ## @@ -2568,26 +2568,26 @@ components: propertyName: type mapping: assert-view-uuid: '#/components/sc

Re: [PR] API, Core: Fix errorprone warnings [iceberg]

ajantha-bhat commented on code in PR #9419: URL: https://github.com/apache/iceberg/pull/9419#discussion_r1442771024 ## api/src/main/java/org/apache/iceberg/util/CharSequenceSet.java: ## @@ -166,6 +166,7 @@ public void clear() { wrapperSet.clear(); } + @SuppressWarning

Re: [PR] API, Core: Fix errorprone warnings [iceberg]

ajantha-bhat commented on PR #9419: URL: https://github.com/apache/iceberg/pull/9419#issuecomment-1878529981 cc: @aokolnychyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[PR] API, Core: Fix errorprone warnings [iceberg]

ajantha-bhat opened a new pull request, #9419: URL: https://github.com/apache/iceberg/pull/9419 `./gradlew clean build -x test` is reporting some warnings as seen below. https://github.com/apache/iceberg/assets/5889404/e64c76c4-29d3-42cf-b3e0-c1d9f847200b";> After this PR, `./g

  1   2   >