[PR] Implement table metadata update and table requirement validation [iceberg-python]

2023-11-11 Thread via GitHub
HonahX opened a new pull request, #139: URL: https://github.com/apache/iceberg-python/pull/139 Fixes: #22 This PR: - Implements `update_table_metadata` which applies a list of TableUpdate on the current metadata - Adds unit tests in `test_init.py` -- This is an automated

Re: [PR] Build: Bump Apache Spark to 3.3.3 [iceberg]

2023-11-11 Thread via GitHub
wangyum closed pull request #8230: Build: Bump Apache Spark to 3.3.3 URL: https://github.com/apache/iceberg/pull/8230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

[PR] Build: Bump calcite from 1.10.0 to 1.36.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9042: URL: https://github.com/apache/iceberg/pull/9042 Bumps `calcite` from 1.10.0 to 1.36.0. Updates `org.apache.calcite:calcite-core` from 1.10.0 to 1.36.0 Commits https://github.com/apache/calcite/commit/0be8eaebcf27afae9ecda8ab7

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.30.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] closed pull request #8944: Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.30.0 URL: https://github.com/apache/iceberg/pull/8944 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.100.Final [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] commented on PR #8783: URL: https://github.com/apache/iceberg/pull/8783#issuecomment-1806996575 Superseded by #9040. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.31.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9041: URL: https://github.com/apache/iceberg/pull/9041 Bumps [com.palantir.baseline:gradle-baseline-java](https://github.com/palantir/gradle-baseline) from 4.42.0 to 5.31.0. Release notes Sourced from https://github.com/palantir/gradle-b

Re: [PR] Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.1.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] commented on PR #8779: URL: https://github.com/apache/iceberg/pull/8779#issuecomment-1806996532 Superseded by #9038. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.1.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] closed pull request #8779: Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.1.0 URL: https://github.com/apache/iceberg/pull/8779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] Build: Bump net.snowflake:snowflake-jdbc from 3.14.2 to 3.14.3 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9039: URL: https://github.com/apache/iceberg/pull/9039 Bumps [net.snowflake:snowflake-jdbc](https://github.com/snowflakedb/snowflake-jdbc) from 3.14.2 to 3.14.3. Release notes Sourced from https://github.com/snowflakedb/snowflake-jdbc/re

[PR] Build: Bump junit from 5.10.0 to 5.10.1 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9037: URL: https://github.com/apache/iceberg/pull/9037 Bumps `junit` from 5.10.0 to 5.10.1. Updates `org.junit.jupiter:junit-jupiter` from 5.10.0 to 5.10.1 Release notes Sourced from https://github.com/junit-team/junit5/releases";>org.j

[PR] Build: Bump orc from 1.9.1 to 1.9.2 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9045: URL: https://github.com/apache/iceberg/pull/9045 Bumps `orc` from 1.9.1 to 1.9.2. Updates `org.apache.orc:orc-core` from 1.9.1 to 1.9.2 Updates `org.apache.orc:orc-tools` from 1.9.1 to 1.9.2 Dependabot will resolve any con

[PR] Build: Bump software.amazon.awssdk:bom from 2.21.15 to 2.21.21 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9044: URL: https://github.com/apache/iceberg/pull/9044 Bumps software.amazon.awssdk:bom from 2.21.15 to 2.21.21. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softwa

[PR] Build: Bump arrow from 14.0.0 to 14.0.1 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9043: URL: https://github.com/apache/iceberg/pull/9043 Bumps `arrow` from 14.0.0 to 14.0.1. Updates `org.apache.arrow:arrow-memory-netty` from 14.0.0 to 14.0.1 Updates `org.apache.arrow:arrow-vector` from 14.0.0 to 14.0.1 Commits

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.30.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] commented on PR #8944: URL: https://github.com/apache/iceberg/pull/8944#issuecomment-1806996584 Superseded by #9041. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.100.Final [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] closed pull request #8783: Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.100.Final URL: https://github.com/apache/iceberg/pull/8783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[PR] Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.101.Final [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9040: URL: https://github.com/apache/iceberg/pull/9040 Bumps [io.netty:netty-buffer](https://github.com/netty/netty) from 4.1.68.Final to 4.1.101.Final. Commits https://github.com/netty/netty/commit/c6a6aadaface1b2b66d2608dcdc6e4c04c1

[PR] Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.2.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9038: URL: https://github.com/apache/iceberg/pull/9038 Bumps com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.2.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=com.

[PR] Build: Bump com.google.cloud:libraries-bom from 26.26.0 to 26.27.0 [iceberg]

2023-11-11 Thread via GitHub
dependabot[bot] opened a new pull request, #9036: URL: https://github.com/apache/iceberg/pull/9036 Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.26.0 to 26.27.0. Release notes Sourced from https://github.com/googleapis/java-cloud-bom/

Re: [I] Support for query the historical schemas and options of the table [iceberg]

2023-11-11 Thread via GitHub
github-actions[bot] commented on issue #7611: URL: https://github.com/apache/iceberg/issues/7611#issuecomment-1806951629 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Override useCommitCoordinator to false in Spark3.4 [iceberg]

2023-11-11 Thread via GitHub
huaxingao commented on PR #9028: URL: https://github.com/apache/iceberg/pull/9028#issuecomment-1806935150 Thanks @aokolnychyi @singhpk234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Override useCommitCoordinator to false in BaseStreamingWriter [iceberg]

2023-11-11 Thread via GitHub
huaxingao commented on PR #9027: URL: https://github.com/apache/iceberg/pull/9027#issuecomment-1806935028 Thanks @aokolnychyi @singhpk234 @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Core: lazily create locationProvider in SerializableTable [iceberg]

2023-11-11 Thread via GitHub
aokolnychyi commented on PR #9029: URL: https://github.com/apache/iceberg/pull/9029#issuecomment-1806884352 @przemekd, could you elaborate a bit on the custom `LocationProvider`? All implementations must be `Serializable`. Are you seeing issues with Java or Kryo serialization? -- This is

Re: [PR] Override useCommitCoordinator to false in Spark3.4 [iceberg]

2023-11-11 Thread via GitHub
aokolnychyi commented on PR #9028: URL: https://github.com/apache/iceberg/pull/9028#issuecomment-1806883015 Thanks, @huaxingao! Thanks for reviewing, @singhpk234! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Override useCommitCoordinator to false in Spark3.4 [iceberg]

2023-11-11 Thread via GitHub
aokolnychyi merged PR #9028: URL: https://github.com/apache/iceberg/pull/9028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Override useCommitCoordinator to false in BaseStreamingWriter [iceberg]

2023-11-11 Thread via GitHub
aokolnychyi commented on PR #9027: URL: https://github.com/apache/iceberg/pull/9027#issuecomment-1806882541 Thanks, @huaxingao! Thanks for reviewing, @singhpk234 @Fokko! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Override useCommitCoordinator to false in BaseStreamingWriter [iceberg]

2023-11-11 Thread via GitHub
aokolnychyi merged PR #9027: URL: https://github.com/apache/iceberg/pull/9027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] Flink: Add support for Flink 1.18 [iceberg]

2023-11-11 Thread via GitHub
PrabhuJoseph commented on issue #8930: URL: https://github.com/apache/iceberg/issues/8930#issuecomment-1806858264 Flink1.17 used to allow conversion of collection (List) data types `ARRAY` and `ARRAY` into `Object[]` or `Integer[]`. But Flink1.18 fixed it in https://issues.apache.org/jira/b

[PR] Core: Adding Serializing TableMetadataLocation in TableMetadataParser [iceberg]

2023-11-11 Thread via GitHub
leewjae opened a new pull request, #9034: URL: https://github.com/apache/iceberg/pull/9034 Adding metadataFileLocation in toJson method in TableMetadataParser, and checking nullity of metadataLocation in fromJson method -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Core: Adding Serializing TableMetadataLocation in TableMetadataParser [iceberg]

2023-11-11 Thread via GitHub
leewjae commented on PR #9033: URL: https://github.com/apache/iceberg/pull/9033#issuecomment-1806844932 Reopening PR with single commit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Core: Adding Serializing TableMetadataLocation in TableMetadataParser [iceberg]

2023-11-11 Thread via GitHub
leewjae closed pull request #9033: Core: Adding Serializing TableMetadataLocation in TableMetadataParser URL: https://github.com/apache/iceberg/pull/9033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Core: Adding Serializing TableMetadataLocation in TableMetadataParser [iceberg]

2023-11-11 Thread via GitHub
leewjae opened a new pull request, #9033: URL: https://github.com/apache/iceberg/pull/9033 Adding TableMetadataLocation into TableMetadataParser in case tableMetadata is null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Parquet: Remove the row position since parquet row group has it natively [iceberg]

2023-11-11 Thread via GitHub
ricardopereira33 commented on PR #6056: URL: https://github.com/apache/iceberg/pull/6056#issuecomment-1806829411 Hi @flyrain ! I added all the details in this [Slack thread](https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1699608872098359). Can you see it? -- This is an automated

Re: [I] Does the Java API support primary keys for creating tables [iceberg]

2023-11-11 Thread via GitHub
Zhangg7723 commented on issue #8950: URL: https://github.com/apache/iceberg/issues/8950#issuecomment-1806785206 Schema schema = new Schema(columns, identifierFieldIds) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Replace i64 with DateTime [iceberg-rust]

2023-11-11 Thread via GitHub
Xuanwo commented on code in PR #94: URL: https://github.com/apache/iceberg-rust/pull/94#discussion_r1390182259 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -72,7 +73,7 @@ pub struct Snapshot { sequence_number: i64, /// A timestamp when the snapshot was created, used f

Re: [PR] Replace i64 with DateTime [iceberg-rust]

2023-11-11 Thread via GitHub
Xuanwo commented on code in PR #94: URL: https://github.com/apache/iceberg-rust/pull/94#discussion_r1390182259 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -72,7 +73,7 @@ pub struct Snapshot { sequence_number: i64, /// A timestamp when the snapshot was created, used f

Re: [PR] Replace i64 with DateTime [iceberg-rust]

2023-11-11 Thread via GitHub
my-vegetable-has-exploded commented on PR #94: URL: https://github.com/apache/iceberg-rust/pull/94#issuecomment-1806753936 Since there is [timestamptz](https://github.com/apache/iceberg-rust/blob/e26bda3fc2f6df7dde49a6cf7a5f3faafe49313b/crates/iceberg/src/spec/values.rs#L982C4-L982C4) which

Re: [I] Substitue in memory data struct's timestamp type for DataTime rather i64 to simplify usage. [iceberg-rust]

2023-11-11 Thread via GitHub
my-vegetable-has-exploded commented on issue #90: URL: https://github.com/apache/iceberg-rust/issues/90#issuecomment-1806749835 > Welp, I started looking at this ticket before the most recent comments :S I went with `chrono::DateTime` as well but had to wrap it in a NewType to implement som