Re: [I] [Potential Bug] insertedRowMap too large could cause flink application failure? [iceberg]

pvary commented on issue #10030: URL: https://github.com/apache/iceberg/issues/10030#issuecomment-2016379645 You would like to compact the table to get rid of the delete files. RewriteDataFilesAction could help, or Spark compaction -- This is an automated message from the Apache Git Servi

Re: [I] flink to iceberg commit fail (Datastream API, hiveCatalog) [iceberg]

SML0127 commented on issue #10019: URL: https://github.com/apache/iceberg/issues/10019#issuecomment-2016361743 @pvary There's something strange about hive catalog. When I check HiveCatalog.namespaceExists(namespace), it always return `false`. But when I create name space, it re

Re: [I] flink to iceberg commit fail (Datastream API, hiveCatalog) [iceberg]

SML0127 commented on issue #10019: URL: https://github.com/apache/iceberg/issues/10019#issuecomment-2016358035 HI @pvary ! - Hive, HMS version is `2.3.2` - Current checkpoint interval is 60s. should I decrease checkpoint interval further? - I checked table exist in HD

Re: [I] Bloom filter not properly leveraged when using an OR condition [iceberg]

huaxingao commented on issue #10029: URL: https://github.com/apache/iceberg/issues/10029#issuecomment-2016304838 @cccs-jc Thanks a lot for your thorough investigation and analysis! The problem you described will also occur without a bloom filter. Let's use the where clause `col1=1 OR

[I] Pyarrow type error [iceberg-python]

dev-goyal opened a new issue, #541: URL: https://github.com/apache/iceberg-python/issues/541 ### Apache Iceberg version 0.6.0 (latest release) ### Please describe the bug 🐞 Given a table like so: `In [36]: table Out[36]: matches( ... 14: player_last_

Re: [I] remove a list .metadata.json in HadoopCatalog [iceberg]

github-actions[bot] commented on issue #1934: URL: https://github.com/apache/iceberg/issues/1934#issuecomment-2016205454 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] remove a list .metadata.json in HadoopCatalog [iceberg]

github-actions[bot] closed issue #1934: remove a list .metadata.json in HadoopCatalog URL: https://github.com/apache/iceberg/issues/1934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] BaseMetastoreCatalog's Base Metadata Tables use Data Table's Operations and EncryptionManager [iceberg]

github-actions[bot] closed issue #1928: BaseMetastoreCatalog's Base Metadata Tables use Data Table's Operations and EncryptionManager URL: https://github.com/apache/iceberg/issues/1928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Hive: Schema conversion should convert NOT NULL/required fields [iceberg]

github-actions[bot] closed issue #1922: Hive: Schema conversion should convert NOT NULL/required fields URL: https://github.com/apache/iceberg/issues/1922 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] BaseMetastoreCatalog's Base Metadata Tables use Data Table's Operations and EncryptionManager [iceberg]

github-actions[bot] commented on issue #1928: URL: https://github.com/apache/iceberg/issues/1928#issuecomment-2016205418 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Spark Table With Timestamp Cannot Query By Flink [iceberg]

github-actions[bot] closed issue #1916: Spark Table With Timestamp Cannot Query By Flink URL: https://github.com/apache/iceberg/issues/1916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] Hive: Schema conversion should convert NOT NULL/required fields [iceberg]

github-actions[bot] commented on issue #1922: URL: https://github.com/apache/iceberg/issues/1922#issuecomment-2016205393 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink 1.11.1 Not Supoort timestamptz [iceberg]

github-actions[bot] closed issue #1915: Flink 1.11.1 Not Supoort timestamptz URL: https://github.com/apache/iceberg/issues/1915 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Flink 1.11.1 Not Supoort timestamptz [iceberg]

github-actions[bot] commented on issue #1915: URL: https://github.com/apache/iceberg/issues/1915#issuecomment-2016205309 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Spark Table With Timestamp Cannot Query By Flink [iceberg]

github-actions[bot] commented on issue #1916: URL: https://github.com/apache/iceberg/issues/1916#issuecomment-2016205349 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Fix pagination description based on new feedback [iceberg]

rdblue merged PR #9917: URL: https://github.com/apache/iceberg/pull/9917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Fix pagination description based on new feedback [iceberg]

rdblue commented on PR #9917: URL: https://github.com/apache/iceberg/pull/9917#issuecomment-2015993960 Look good. Thanks, @rahil-c! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Docs: Fix inconsistency in branching and tagging scenario [iceberg]

bitsondatadev commented on PR #9968: URL: https://github.com/apache/iceberg/pull/9968#issuecomment-2015939651 I'm currently AFK, I'll test this tonight. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [core] fix #9997 - Handle s3a file upload interrupt which results in table metadata pointing to files that doesn't exist [iceberg]

stevenzwu commented on code in PR #9998: URL: https://github.com/apache/iceberg/pull/9998#discussion_r1536231485 ## core/src/test/java/org/apache/iceberg/hadoop/HadoopStreamsTest.java: ## @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Docs: Fix inconsistency in branching and tagging scenario [iceberg]

lawofcycles commented on PR #9968: URL: https://github.com/apache/iceberg/pull/9968#issuecomment-2015937292 @Fokko @bitsondatadev I would greatly appreciate it if you could kindly review this pull request. -- This is an automated message from the Apache Git Service. To respond to the m

Re: [I] flink to iceberg (hiveCatalog) commit fail [iceberg]

pvary commented on issue #10019: URL: https://github.com/apache/iceberg/issues/10019#issuecomment-2015914905 Seems like an issue with the HiveClient. Could you check the server HMS version, the HMSClient version in the job. Or alternatively if there is a connection issue between the tasks a

[I] Bloom filter not properly leveraged when using an OR condition [iceberg]

cccs-jc opened a new issue, #10029: URL: https://github.com/apache/iceberg/issues/10029 ### Apache Iceberg version 1.4.3 ### Query engine Spark ### Please describe the bug 🐞 I'm testing a table of flow data with a schema of `SRC_IP long, DST_IP long`

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

pvary commented on code in PR #10016: URL: https://github.com/apache/iceberg/pull/10016#discussion_r1536177091 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -397,6 +399,36 @@ public void testLockExceptionUnknownSuccessCommit() throws TExcep

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

pvary commented on code in PR #10016: URL: https://github.com/apache/iceberg/pull/10016#discussion_r1536170865 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -273,11 +275,11 @@ public void testThriftExceptionConcurrentCommit() throws TExcept

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

pvary commented on code in PR #10016: URL: https://github.com/apache/iceberg/pull/10016#discussion_r1536170178 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCommits.java: ## @@ -64,7 +66,7 @@ public void testSuppressUnlockExceptions() { AtomicReference l

Re: [PR] Support merge manifests on writes (MergeAppend) [iceberg-python]

syun64 commented on code in PR #363: URL: https://github.com/apache/iceberg-python/pull/363#discussion_r1536160297 ## pyiceberg/table/__init__.py: ## @@ -1091,7 +,7 @@ def append(self, df: pa.Table) -> None: _check_schema(self.schema(), other_schema=df.schema)

Re: [PR] Core, Spark: Fix handling of null binary values when sorting with zorder [iceberg]

RussellSpitzer commented on code in PR #10026: URL: https://github.com/apache/iceberg/pull/10026#discussion_r1536007907 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewriteDataFilesProcedure.java: ## @@ -218,6 +218,44 @@ public void testRe

[PR] Fix/add dremio blogs [iceberg]

AlexMercedCoder opened a new pull request, #10028: URL: https://github.com/apache/iceberg/pull/10028 @Fokko reviewed and you were correct, it was the wrong link the url should be fixed now -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] Core: Calling rewrite_position_delete_files fails on tables with more than 1k columns [iceberg]

szehon-ho closed pull request #10020: Core: Calling rewrite_position_delete_files fails on tables with more than 1k columns URL: https://github.com/apache/iceberg/pull/10020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Add 13 Dremio Blogs + Fix a few incorrect dates [iceberg]

AlexMercedCoder commented on code in PR #9967: URL: https://github.com/apache/iceberg/pull/9967#discussion_r1535951836 ## site/docs/blogs.md: ## @@ -22,6 +22,41 @@ title: "Blogs" Here is a list of company blogs that talk about Iceberg. The blogs are ordered from most recent

[I] Metadata file is not getting created when Iceberg table is created using Hive with catalog as GlueCatalog [iceberg]

ganeshashree opened a new issue, #10025: URL: https://github.com/apache/iceberg/issues/10025 ### Apache Iceberg version 1.5.0 (latest release) ### Query engine Hive ### Please describe the bug 🐞 When Iceberg table is created using Hive with catalog as GlueCa

Re: [PR] feat: implement prune column for schema [iceberg-rust]

Dysprosium0626 commented on code in PR #261: URL: https://github.com/apache/iceberg-rust/pull/261#discussion_r1535803356 ## crates/iceberg/src/spec/schema.rs: ## @@ -642,6 +644,204 @@ impl SchemaVisitor for IndexByName { } } +struct PruneColumn { +selected: HashSet,

Re: [I] Bug: Flink data loss after failed to refresh table [iceberg]

maekchi commented on issue #9753: URL: https://github.com/apache/iceberg/issues/9753#issuecomment-2015318587 hello. I'm leaving a comment because I'm experiencing similar situations. I'm using iceberg 1.4.3 and flink 1.15, and I'm experiencing the same problem quite often. when ops

Re: [PR] [Draft] Spark: Add CopyTable spark action [iceberg]

laithalzyoud commented on PR #10024: URL: https://github.com/apache/iceberg/pull/10024#issuecomment-2015316957 > How about position delete files? @manuzhang They are covered in this PR, let me add that in the description as well 👌 -- This is an automated message from the Apache Gi

Re: [PR] [Draft] Spark: Add CopyTable spark action [iceberg]

manuzhang commented on PR #10024: URL: https://github.com/apache/iceberg/pull/10024#issuecomment-2015305635 How about position delete files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] zorder fails with null binary values [iceberg]

amogh-jahagirdar commented on issue #10022: URL: https://github.com/apache/iceberg/issues/10022#issuecomment-2015236807 Yeah I think you're right, just looks like a missing null check. If the byte array is null I think we should just return an empty byte buffer. I'll put up a PR -- This

Re: [PR] Core: Lazily compute & cache hashCode in CharSequenceWrapper [iceberg]

nastra commented on code in PR #10023: URL: https://github.com/apache/iceberg/pull/10023#discussion_r1535671891 ## api/src/test/java/org/apache/iceberg/util/TestCharSequenceWrapper.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Core: Lazily compute & cache hashCode in CharSequenceWrapper [iceberg]

nastra commented on code in PR #10023: URL: https://github.com/apache/iceberg/pull/10023#discussion_r1535670174 ## api/src/main/java/org/apache/iceberg/types/JavaHashes.java: ## @@ -26,6 +26,10 @@ public class JavaHashes { private JavaHashes() {} public static int hashCo

Re: [PR] Core: Lazily compute & cache hashCode in CharSequenceWrapper [iceberg]

nastra commented on code in PR #10023: URL: https://github.com/apache/iceberg/pull/10023#discussion_r1535663017 ## api/src/main/java/org/apache/iceberg/util/CharSequenceWrapper.java: ## @@ -67,7 +78,13 @@ public boolean equals(Object other) { @Override public int hashCod

[PR] Core: Lazily compute & cache hashCode in CharSequenceWrapper [iceberg]

nastra opened a new pull request, #10023: URL: https://github.com/apache/iceberg/pull/10023 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Rename function name to `add_manifests` [iceberg-rust]

viirya commented on PR #293: URL: https://github.com/apache/iceberg-rust/pull/293#issuecomment-2015167742 Thank you @liurenjie1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[I] zorder fails with null binary values [iceberg]

cccs-jc opened a new issue, #10022: URL: https://github.com/apache/iceberg/issues/10022 ### Apache Iceberg version 1.5.0 (latest release) ### Query engine Spark ### Please describe the bug 🐞 I have a table of IP flow data. The src_ipv4 and dst_ipv4 columns a

Re: [PR] [core] fix #9997 - Handle s3a file upload interrupt which results in table metadata pointing to files that doesn't exist [iceberg]

abmo-x commented on PR #9998: URL: https://github.com/apache/iceberg/pull/9998#issuecomment-2015074656 Thanks @nastra @stevenzwu @RussellSpitzer for the review. Addressed all comments, can we merge this? -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] [core] fix #9997 - Handle s3a file upload interrupt which results in table metadata pointing to files that doesn't exist [iceberg]

abmo-x commented on code in PR #9998: URL: https://github.com/apache/iceberg/pull/9998#discussion_r1535548232 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopStreams.java: ## @@ -187,6 +187,16 @@ public void flush() throws IOException { public void close() throws IOEx

Re: [I] Add hive metastore catalog support [iceberg-rust]

liurenjie1024 closed issue #113: Add hive metastore catalog support URL: https://github.com/apache/iceberg-rust/issues/113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Add hive metastore catalog support (part 2/2) [iceberg-rust]

liurenjie1024 merged PR #285: URL: https://github.com/apache/iceberg-rust/pull/285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Add hive metastore catalog support (part 2/2) [iceberg-rust]

liurenjie1024 commented on PR #285: URL: https://github.com/apache/iceberg-rust/pull/285#issuecomment-2015064278 I'll merge this first to move on. We can refine it later if others have comments, thanks @marvinlanhenke -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Add 13 Dremio Blogs + Fix a few incorrect dates [iceberg]

Fokko commented on code in PR #9967: URL: https://github.com/apache/iceberg/pull/9967#discussion_r1535475495 ## site/docs/blogs.md: ## @@ -22,6 +22,41 @@ title: "Blogs" Here is a list of company blogs that talk about Iceberg. The blogs are ordered from most recent to oldest.

Re: [PR] docs: Add links checker [iceberg]

Fokko commented on code in PR #9965: URL: https://github.com/apache/iceberg/pull/9965#discussion_r1535464835 ## .github/workflows/docs-check-links.yml: ## @@ -0,0 +1,39 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] docs: Add links checker [iceberg]

Fokko commented on code in PR #9965: URL: https://github.com/apache/iceberg/pull/9965#discussion_r1535464349 ## format/spec.md: ## @@ -57,6 +57,7 @@ In addition to row-level deletes, version 2 makes some requirements stricter for ## Overview + Review Comment: That's an

Re: [PR] feat: init iceberg writer [iceberg-rust]

ZENOTME commented on code in PR #275: URL: https://github.com/apache/iceberg-rust/pull/275#discussion_r1535458284 ## crates/iceberg/src/writer/base_writer/data_file_writer.rs: ## @@ -0,0 +1,310 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contrib

Re: [I] data insertion to iceberg table [iceberg]

nastra commented on issue #10021: URL: https://github.com/apache/iceberg/issues/10021#issuecomment-2014894306 Please see https://py.iceberg.apache.org/api/#write-support for how to perform writes using pyiceberg -- This is an automated message from the Apache Git Service. To respond to th

Re: [I] data insertion to iceberg table [iceberg]

nastra closed issue #10021: data insertion to iceberg table URL: https://github.com/apache/iceberg/issues/10021 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535437170 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1041,7 +1041,7 @@ public void testNotExposeTableProperties() { .doesNotCo

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535436900 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1041,7 +1041,7 @@ public void testNotExposeTableProperties() { .doesNotCo

Re: [PR] Spark: Bump Spark minor versions for 3.3 and 3.4 [iceberg]

RussellSpitzer commented on PR #9187: URL: https://github.com/apache/iceberg/pull/9187#issuecomment-2014892016 We have always in our internal builds defaulted to purge off for iceberg (even before there was an option) for safety. I prefer that behavior and don't really mind that it's differ

Re: [PR] Hive: Tests to do validation hive content and iceberg table with the same name [iceberg]

nastra commented on code in PR #9980: URL: https://github.com/apache/iceberg/pull/9980#discussion_r1535435047 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -349,8 +351,70 @@ public void testListTables() throws TException, IOException { H

Re: [PR] Modify `Bind` calls so that they don't consume `self` and instead return a new struct, leaving the original unmoved [iceberg-rust]

liurenjie1024 merged PR #290: URL: https://github.com/apache/iceberg-rust/pull/290 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] feat: add builder to TableMetadata interface [iceberg-rust]

liurenjie1024 closed pull request #62: feat: add builder to TableMetadata interface URL: https://github.com/apache/iceberg-rust/pull/62 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] feat: add builder to TableMetadata interface [iceberg-rust]

liurenjie1024 commented on PR #62: URL: https://github.com/apache/iceberg-rust/pull/62#issuecomment-2014886303 > cc @y0psolo Should we close this now? I think it's resolved by #262 I'll close this for now, feel free to reopen it if necessary. -- This is an automated message from the

Re: [PR] Hive: Tests to do validation hive content and iceberg table with the same name [iceberg]

nastra commented on code in PR #9980: URL: https://github.com/apache/iceberg/pull/9980#discussion_r1535431150 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -349,8 +351,70 @@ public void testListTables() throws TException, IOException { H

Re: [PR] Rename function name to `add_manifests` [iceberg-rust]

liurenjie1024 merged PR #293: URL: https://github.com/apache/iceberg-rust/pull/293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Implement Glue Catalog [iceberg-rust]

liurenjie1024 commented on issue #249: URL: https://github.com/apache/iceberg-rust/issues/249#issuecomment-2014882621 > > I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue? > > I looked at the pyiceberg tests which utilize `moto`. Moto

Re: [PR] feat: implement prune column for schema [iceberg-rust]

liurenjie1024 commented on code in PR #261: URL: https://github.com/apache/iceberg-rust/pull/261#discussion_r1535397546 ## crates/iceberg/src/spec/schema.rs: ## @@ -642,6 +644,204 @@ impl SchemaVisitor for IndexByName { } } +struct PruneColumn { +selected: HashSet, +

Re: [PR] Hive: Tests to do validation hive content and iceberg table with the same name [iceberg]

nastra commented on code in PR #9980: URL: https://github.com/apache/iceberg/pull/9980#discussion_r1535430072 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -349,8 +351,70 @@ public void testListTables() throws TException, IOException { H

Re: [PR] Hive: Tests to do validation hive content and iceberg table with the same name [iceberg]

nastra commented on code in PR #9980: URL: https://github.com/apache/iceberg/pull/9980#discussion_r1535428308 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -349,8 +351,70 @@ public void testListTables() throws TException, IOException { H

Re: [PR] Hive: Tests to do validation hive content and iceberg table with the same name [iceberg]

nastra commented on code in PR #9980: URL: https://github.com/apache/iceberg/pull/9980#discussion_r1535428044 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -349,8 +351,70 @@ public void testListTables() throws TException, IOException { H

Re: [PR] Hive: Tests to do validation hive content and iceberg table with the same name [iceberg]

nastra commented on code in PR #9980: URL: https://github.com/apache/iceberg/pull/9980#discussion_r1535425970 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -349,8 +351,70 @@ public void testListTables() throws TException, IOException { H

Re: [PR] Hive: Tests to do validation hive content and iceberg table with the same name [iceberg]

nastra commented on code in PR #9980: URL: https://github.com/apache/iceberg/pull/9980#discussion_r1535425250 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -349,8 +351,70 @@ public void testListTables() throws TException, IOException { H

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535422769 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java: ## @@ -457,6 +448,16 @@ public String table() { return tableName; } + @Ove

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535422346 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java: ## @@ -365,6 +353,10 @@ private void setHmsTableParameters( } // Set the

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535422346 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java: ## @@ -365,6 +353,10 @@ private void setHmsTableParameters( } // Set the

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535421842 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java: ## @@ -304,30 +305,16 @@ protected void doCommit(TableMetadata base, TableMetadata m

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535420615 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java: ## @@ -304,30 +305,16 @@ protected void doCommit(TableMetadata base, TableMetadata m

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535419303 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -123,13 +185,14 @@ default void persistTable(Table hmsTable, boolean updateHive

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535418537 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -76,11 +138,11 @@ default boolean exposeInHmsProperties() { return maxHiveT

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535416777 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -62,6 +82,48 @@ interface HiveOperationsBase { String table(); + String

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

nastra commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1535409457 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -76,10 +159,10 @@ default boolean exposeInHmsProperties() { return maxHiveTab

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535406841 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -62,6 +82,48 @@ interface HiveOperationsBase { String table(); + String

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535407624 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java: ## @@ -62,6 +82,48 @@ interface HiveOperationsBase { String table(); + String

Re: [I] Implement Glue Catalog [iceberg-rust]

marvinlanhenke commented on issue #249: URL: https://github.com/apache/iceberg-rust/issues/249#issuecomment-2014833603 > I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue? I looked at the pyiceberg tests and saw that here `moto` is use

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535393968 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -273,6 +284,21 @@ public void renameTable(TableIdentifier from, TableIdentifier origin

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535393632 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -273,6 +284,21 @@ public void renameTable(TableIdentifier from, TableIdentifier origin

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535393422 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -239,7 +250,7 @@ public void renameTable(TableIdentifier from, TableIdentifier origina

Re: [I] Implement Glue Catalog [iceberg-rust]

liurenjie1024 commented on issue #249: URL: https://github.com/apache/iceberg-rust/issues/249#issuecomment-2014822544 Hi, @marvinlanhenke Thanks for this. I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue? -- This is an automated message f

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535389326 ## core/src/test/java/org/apache/iceberg/TestCatalogUtil.java: ## @@ -223,6 +223,32 @@ public void loadCustomMetricsReporter_badClass() { .hasMessageContainin

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535389035 ## core/src/test/java/org/apache/iceberg/TestCatalogUtil.java: ## @@ -223,6 +223,32 @@ public void loadCustomMetricsReporter_badClass() { .hasMessageContainin

Re: [PR] Hive: Arrange common part of the code for Iceberg View. [iceberg]

nastra commented on code in PR #10001: URL: https://github.com/apache/iceberg/pull/10001#discussion_r1535385693 ## core/src/main/java/org/apache/iceberg/BaseMetastoreOperations.java: ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] WIP: Incremental Append Scan [iceberg-python]

hililiwei commented on PR #533: URL: https://github.com/apache/iceberg-python/pull/533#issuecomment-2014591526 In the latest code commit, I tinkered with the class inheritance by introducing a new base class, BaseIncrementalScan, which inherits from TableScan. I also pushed the snapshot_id

Re: [PR] Core: Calling rewrite_position_delete_files fails on tables with more than 1k columns [iceberg]

szehon-ho commented on code in PR #10020: URL: https://github.com/apache/iceberg/pull/10020#discussion_r1535196147 ## api/src/main/java/org/apache/iceberg/types/AssignFreshIds.java: ## @@ -18,19 +18,13 @@ */ package org.apache.iceberg.types; -import java.util.Iterator; -imp

Re: [PR] Core: Calling rewrite_position_delete_files fails on tables with more than 1k columns [iceberg]

szehon-ho commented on code in PR #10020: URL: https://github.com/apache/iceberg/pull/10020#discussion_r1535196147 ## api/src/main/java/org/apache/iceberg/types/AssignFreshIds.java: ## @@ -18,19 +18,13 @@ */ package org.apache.iceberg.types; -import java.util.Iterator; -imp

[PR] Core: Calling rewrite_position_delete_files fails on tables with more than 1k columns [iceberg]

szehon-ho opened a new pull request, #10020: URL: https://github.com/apache/iceberg/pull/10020 The position_deletes metadata table (used to rewrite_position_deletes) has both 'partition' field and row field (esesentially the table schema again, for the optional 'row' column of position dele

[I] flink to iceberg (hiveCatalog) commit fail [iceberg]

SML0127 opened a new issue, #10019: URL: https://github.com/apache/iceberg/issues/10019 ### Apache Iceberg version 1.5.0 (latest release) ### Query engine Flink ### Please describe the bug 🐞 First commit at create table is success, after commit is always fai

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

lirui-apache commented on PR #10016: URL: https://github.com/apache/iceberg/pull/10016#issuecomment-2014546238 @pvary I added a new test to verify changing lock mechanism should use the current lock type rather than the new one. -- This is an automated message from the Apache Git Service.

Re: [PR] [core] fix #9997 - Handle s3a file upload interrupt which results in table metadata pointing to files that doesn't exist [iceberg]

nastra commented on code in PR #9998: URL: https://github.com/apache/iceberg/pull/9998#discussion_r1535162433 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopStreams.java: ## @@ -187,6 +187,16 @@ public void flush() throws IOException { public void close() throws IOEx

Re: [PR] [core] fix #9997 - Handle s3a file upload interrupt which results in table metadata pointing to files that doesn't exist [iceberg]

nastra commented on code in PR #9998: URL: https://github.com/apache/iceberg/pull/9998#discussion_r1535159638 ## core/src/test/java/org/apache/iceberg/hadoop/TestHadoopStreams.java: ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] [core] fix #9997 - Handle s3a file upload interrupt which results in table metadata pointing to files that doesn't exist [iceberg]

nastra commented on code in PR #9998: URL: https://github.com/apache/iceberg/pull/9998#discussion_r1535159074 ## core/src/test/java/org/apache/hadoop/fs/s3a/S3ABlockOutputStream.java: ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Spark: Bump Spark minor versions for 3.3 and 3.4 [iceberg]

manuzhang commented on PR #9187: URL: https://github.com/apache/iceberg/pull/9187#issuecomment-2014511798 @chinnaraolalam which versions of Iceberg and Spark are you using in test cases? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] many doc lost in iceberg official website [iceberg]

nastra commented on issue #10018: URL: https://github.com/apache/iceberg/issues/10018#issuecomment-2014500680 Thanks for reporting @tcguanshuhuai, I believe @bitsondatadev wanted to go over the docs and fix such things. @tcguanshuhuai any help is greatly appreciated in case you'd like to co

Re: [PR] Add local nightly build to test current docs changes [iceberg]

nastra merged PR #9943: URL: https://github.com/apache/iceberg/pull/9943 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

  1   2   >