Re: [PR] Add filter pushdown API for contains and endsWith [iceberg]

2024-03-02 Thread via GitHub
yabola commented on code in PR #9710: URL: https://github.com/apache/iceberg/pull/9710#discussion_r1510203932 ## api/src/test/java/org/apache/iceberg/transforms/TestNotStartsWith.java: ## @@ -106,23 +75,23 @@ public void

Re: [PR] Add filter pushdown API for contains and endsWith [iceberg]

2024-03-02 Thread via GitHub
yabola commented on code in PR #9710: URL: https://github.com/apache/iceberg/pull/9710#discussion_r1510203932 ## api/src/test/java/org/apache/iceberg/transforms/TestNotStartsWith.java: ## @@ -106,23 +75,23 @@ public void

Re: [PR] Add filter pushdown API for contains and endsWith [iceberg]

2024-03-02 Thread via GitHub
yabola commented on PR #9710: URL: https://github.com/apache/iceberg/pull/9710#issuecomment-1975070635 @amogh-jahagirdar Thanks for your review! I have updated my code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Add filter pushdown API for contains and endsWith [iceberg]

2024-03-02 Thread via GitHub
yabola commented on code in PR #9710: URL: https://github.com/apache/iceberg/pull/9710#discussion_r1510200997 ## api/src/test/java/org/apache/iceberg/expressions/TestEvaluator.java: ## @@ -348,6 +352,102 @@ public void testNotStartsWith() { .isTrue(); } + @Test +

Re: [PR] Core: Fix REST catalog handling when the service has no view support [iceberg]

2024-03-02 Thread via GitHub
ajantha-bhat commented on PR #9853: URL: https://github.com/apache/iceberg/pull/9853#issuecomment-1975066720 LGTM. How can we catch these errors early? Can we add the testcase? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Build: Bump mkdocs-material from 9.5.9 to 9.5.11 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] commented on PR #9801: URL: https://github.com/apache/iceberg/pull/9801#issuecomment-1975023077 Superseded by #9859. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Build: Bump mkdocs-material from 9.5.9 to 9.5.11 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] closed pull request #9801: Build: Bump mkdocs-material from 9.5.9 to 9.5.11 URL: https://github.com/apache/iceberg/pull/9801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] Build: Bump mkdocs-material from 9.5.9 to 9.5.12 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #9859: URL: https://github.com/apache/iceberg/pull/9859 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.9 to 9.5.12. Release notes Sourced from

[PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.43.0 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #9858: URL: https://github.com/apache/iceberg/pull/9858 Bumps [com.palantir.baseline:gradle-baseline-java](https://github.com/palantir/gradle-baseline) from 4.42.0 to 5.43.0. Release notes Sourced from

[PR] Build: Bump com.azure:azure-sdk-bom from 1.2.20 to 1.2.21 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #9857: URL: https://github.com/apache/iceberg/pull/9857 Bumps [com.azure:azure-sdk-bom](https://github.com/azure/azure-sdk-for-java) from 1.2.20 to 1.2.21. Commits

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.24.10 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] closed pull request #9795: Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.24.10 URL: https://github.com/apache/iceberg/pull/9795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.24.10 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] commented on PR #9795: URL: https://github.com/apache/iceberg/pull/9795#issuecomment-1975015172 Superseded by #9856. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.25.1 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #9856: URL: https://github.com/apache/iceberg/pull/9856 Bumps software.amazon.awssdk:bom from 2.24.5 to 2.25.1. [![Dependabot compatibility

[PR] Build: Bump kafka from 3.6.1 to 3.7.0 [iceberg]

2024-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #9855: URL: https://github.com/apache/iceberg/pull/9855 Bumps `kafka` from 3.6.1 to 3.7.0. Updates `org.apache.kafka:kafka-clients` from 3.6.1 to 3.7.0 Updates `org.apache.kafka:connect-api` from 3.6.1 to 3.7.0 Updates

Re: [I] Add "Help" command for CALL methods [iceberg]

2024-03-02 Thread via GitHub
github-actions[bot] commented on issue #1856: URL: https://github.com/apache/iceberg/issues/1856#issuecomment-1974950292 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] Why are there few methods for hive/flinkcatalog? [iceberg]

2024-03-02 Thread via GitHub
github-actions[bot] commented on issue #1834: URL: https://github.com/apache/iceberg/issues/1834#issuecomment-1974950285 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] Make changes to write a Partition stats [iceberg]

2024-03-02 Thread via GitHub
github-actions[bot] commented on issue #1833: URL: https://github.com/apache/iceberg/issues/1833#issuecomment-1974950279 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] Core: Fix REST catalog handling when the service has no view support [iceberg]

2024-03-02 Thread via GitHub
danielcweeks merged PR #9853: URL: https://github.com/apache/iceberg/pull/9853 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Implement Remaining Catalog operations for REST catalog [iceberg-go]

2024-03-02 Thread via GitHub
rtadepalli commented on issue #63: URL: https://github.com/apache/iceberg-go/issues/63#issuecomment-1974912793 Trying to implement `RenameTable` using the OpenAPI spec listed here: https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml Is that correct? If

Re: [PR] Fix the main image on the spec. [iceberg]

2024-03-02 Thread via GitHub
amogh-jahagirdar merged PR #9843: URL: https://github.com/apache/iceberg/pull/9843 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix the main image on the spec. [iceberg]

2024-03-02 Thread via GitHub
amogh-jahagirdar commented on PR #9843: URL: https://github.com/apache/iceberg/pull/9843#issuecomment-1974908725 Discussed with @bitsondatadev right now we're aiming to get the image fixed first, that works for me locally. -- This is an automated message from the Apache Git Service. To

[PR] Kevinjqliu/dead links [iceberg-python]

2024-03-02 Thread via GitHub
kevinjqliu opened a new pull request, #493: URL: https://github.com/apache/iceberg-python/pull/493 #324 added Github action to check link html links in docs. There are a few dead links in the most recent run.

Re: [PR] add support for glue.id [iceberg-python]

2024-03-02 Thread via GitHub
HonahX commented on code in PR #490: URL: https://github.com/apache/iceberg-python/pull/490#discussion_r1510041483 ## pyiceberg/catalog/glue.py: ## @@ -252,6 +259,22 @@ def _construct_database_input(database_name: str, properties: Properties) -> Dat return database_input

Re: [PR] Move format docs to mkdocs from hugo and fix links [iceberg]

2024-03-02 Thread via GitHub
bitsondatadev commented on code in PR #9843: URL: https://github.com/apache/iceberg/pull/9843#discussion_r1510039686 ## format/spec.md: ## @@ -791,7 +788,7 @@ Each version of table metadata is stored in a metadata folder under the table’ Notes: -1. The file system table

Re: [PR] Improve CLI Text by Adding Verbose Text for Commands [iceberg-go]

2024-03-02 Thread via GitHub
rtadepalli commented on code in PR #68: URL: https://github.com/apache/iceberg-go/pull/68#discussion_r1510032756 ## cmd/iceberg/main.go: ## @@ -53,12 +58,22 @@ Arguments: VALUE value to set Options: - -h --help show this helpe messages and exit

[PR] Improve CLI Text by Adding Verbose Text for Commands [iceberg-go]

2024-03-02 Thread via GitHub
rtadepalli opened a new pull request, #68: URL: https://github.com/apache/iceberg-go/pull/68 Add some verbose descriptions (similar to PyIceberg) for the CLI options when `iceberg --help` is used. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Add example of using PyIceberg with minimal external dependencies [iceberg-python]

2024-03-02 Thread via GitHub
kevinjqliu closed issue #326: Add example of using PyIceberg with minimal external dependencies URL: https://github.com/apache/iceberg-python/issues/326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[I] Integration test broken [iceberg-python]

2024-03-02 Thread via GitHub
kevinjqliu opened a new issue, #492: URL: https://github.com/apache/iceberg-python/issues/492 ### Apache Iceberg version main (development) ### Please describe the bug  As of [current

Re: [PR] Core: Fix REST catalog handling when the service has no view support [iceberg]

2024-03-02 Thread via GitHub
rdblue commented on PR #9853: URL: https://github.com/apache/iceberg/pull/9853#issuecomment-1974887622 Okay, I can confirm that this fixes the problem locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] `schema_id` not incremented during schema evolution [iceberg-python]

2024-03-02 Thread via GitHub
kevinjqliu commented on issue #290: URL: https://github.com/apache/iceberg-python/issues/290#issuecomment-1974887039 Awesome. Thank you @anupam-saini I have another follow-up PR that adds `schema_id` checks in places where it was previously implicitly checked #487. We should be good

Re: [PR] Core: Fix REST catalog handling when the service has no view support [iceberg]

2024-03-02 Thread via GitHub
rdblue commented on code in PR #9853: URL: https://github.com/apache/iceberg/pull/9853#discussion_r1510022119 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -1058,12 +1058,17 @@ public List listViews(SessionContext context, Namespace namespa

Re: [PR] Core: Fix REST catalog handling when the service has no view support [iceberg]

2024-03-02 Thread via GitHub
danielcweeks commented on code in PR #9853: URL: https://github.com/apache/iceberg/pull/9853#discussion_r1510022083 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -1058,12 +1058,17 @@ public List listViews(SessionContext context, Namespace namespa

[PR] Core: Fix REST catalog handling when the service has no view support [iceberg]

2024-03-02 Thread via GitHub
rdblue opened a new pull request, #9853: URL: https://github.com/apache/iceberg/pull/9853 This is a fix for Spark table loading when a REST catalog does not support view extensions. The problem is that Spark runs `ResolveViews` before `ResolveTables` in the substitution batch of

Re: [PR] minor: Fix `double` API doc typo [iceberg-rust]

2024-03-02 Thread via GitHub
viirya commented on code in PR #226: URL: https://github.com/apache/iceberg-rust/pull/226#discussion_r1510017267 ## crates/iceberg/src/spec/values.rs: ## @@ -265,7 +265,7 @@ impl Datum { } } -/// Creates a date in `%Y-%m-%d` format, assume in utc timezone. +

Re: [PR] minor: Fix `double` API doc typo [iceberg-rust]

2024-03-02 Thread via GitHub
viirya commented on code in PR #226: URL: https://github.com/apache/iceberg-rust/pull/226#discussion_r1510017188 ## crates/iceberg/src/spec/datatypes.rs: ## @@ -171,13 +171,13 @@ pub enum PrimitiveType { Int, /// 64-bit signed integer Long, -/// 32-bit IEEE

Re: [PR] minor: Fix `double` API doc typo [iceberg-rust]

2024-03-02 Thread via GitHub
viirya commented on code in PR #226: URL: https://github.com/apache/iceberg-rust/pull/226#discussion_r1510017188 ## crates/iceberg/src/spec/datatypes.rs: ## @@ -171,13 +171,13 @@ pub enum PrimitiveType { Int, /// 64-bit signed integer Long, -/// 32-bit IEEE

[PR] minor: Fix `double` API doc typo [iceberg-rust]

2024-03-02 Thread via GitHub
viirya opened a new pull request, #226: URL: https://github.com/apache/iceberg-rust/pull/226 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Core: Common metadata for TableMetadata and ViewMetadata [iceberg]

2024-03-02 Thread via GitHub
nk1506 commented on PR #9682: URL: https://github.com/apache/iceberg/pull/9682#issuecomment-1974868108 > Can we please continue this effort , it has been stalled 2 weeks? > > Let's have a PR that is the end to end version (please not on your own github, but against Iceberg github).

[PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-02 Thread via GitHub
nk1506 opened a new pull request, #9852: URL: https://github.com/apache/iceberg/pull/9852 This changes include: 1. Introduction of common metadata interface(BaseMetadata) for table and view. 2. Refactor for HiveTableOperation to have common code for table and view commits.

Re: [PR] Core: Avro writers use BlockingBinaryEncoder to enable array/map size calculations. [iceberg]

2024-03-02 Thread via GitHub
rustyconover commented on PR #8625: URL: https://github.com/apache/iceberg/pull/8625#issuecomment-1974820592 Any updates on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Optimize `plan_files` with filter in case whe it is fully evaluated on Iceberg metadata [iceberg-python]

2024-03-02 Thread via GitHub
srilman commented on issue #491: URL: https://github.com/apache/iceberg-python/issues/491#issuecomment-1974819638 > When the filter is always true for all output files, we can skip the row-level filter in DataScan.to_arrow I have a somewhat working implementation of this optimization

[I] Optimize `plan_files` with filter in case whe it is fully evaluated on Iceberg metadata [iceberg-python]

2024-03-02 Thread via GitHub
srilman opened a new issue, #491: URL: https://github.com/apache/iceberg-python/issues/491 ### Feature Request / Improvement I noticed that in DataScan.plan_files, when we apply filters at a partition and file metadata level, all we try to determine is whether the file has rows that

Re: [PR] add support for glue.id [iceberg-python]

2024-03-02 Thread via GitHub
jrouly commented on code in PR #490: URL: https://github.com/apache/iceberg-python/pull/490#discussion_r1509829128 ## pyiceberg/catalog/glue.py: ## @@ -252,6 +259,22 @@ def _construct_database_input(database_name: str, properties: Properties) -> Dat return database_input

Re: [PR] add support for glue.id [iceberg-python]

2024-03-02 Thread via GitHub
Fokko commented on PR #490: URL: https://github.com/apache/iceberg-python/pull/490#issuecomment-1974732024 @HonahX since you're more familiar with Glue, would you be able to do this review? :) -- This is an automated message from the Apache Git Service. To respond to the message, please