Re: [PR] chore: doc-test as a target [iceberg-rust]

2024-03-08 Thread via GitHub
Fokko merged PR #235: URL: https://github.com/apache/iceberg-rust/pull/235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Allow setting non-string typed values in `set_properties` [iceberg-python]

2024-03-08 Thread via GitHub
Fokko merged PR #504: URL: https://github.com/apache/iceberg-python/pull/504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[I] Add `table_exists` method to the Catalog [iceberg-python]

2024-03-08 Thread via GitHub
Fokko opened a new issue, #507: URL: https://github.com/apache/iceberg-python/issues/507 ### Feature Request / Improvement Prolonging the discussion in https://github.com/apache/iceberg-python/issues/406 I think having this method on the `Catalog` would be a great addition. On

Re: [PR] Spark 3.4, 3.3 : support output-spec-id in rewrite (back port #9803) [iceberg]

2024-03-08 Thread via GitHub
RussellSpitzer merged PR #9901: URL: https://github.com/apache/iceberg/pull/9901 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceb

Re: [I] Rename `stat_table` in `Catalog` trait [iceberg-rust]

2024-03-08 Thread via GitHub
Fokko commented on issue #236: URL: https://github.com/apache/iceberg-rust/issues/236#issuecomment-1986776464 Thanks for raising this. `stat_table` is not the most descriptive name to me. I would suggest a `table_exist(identifier)` method. Information about the tables is typica

Re: [PR] Add hive metastore catalog support (part 1/2) [iceberg-rust]

2024-03-08 Thread via GitHub
Fokko commented on code in PR #237: URL: https://github.com/apache/iceberg-rust/pull/237#discussion_r1518506389 ## crates/catalog/hms/testdata/hms_catalog/Dockerfile: ## Review Comment: I prefer to keep the Dockerfile to be part of the repository. This way we're in full co

Re: [PR] Core: add support to add custom schemes via properties in ResolvingFileIO [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on PR #9884: URL: https://github.com/apache/iceberg/pull/9884#issuecomment-1986761215 > I don't understand why we have s3 and s3a in ResolvingFileIO, actually why do we have ResolvingFileIO ? This is a good question and somewhat gets to the point. ResolvingFile

Re: [PR] Core: add support to add custom schemes via properties in ResolvingFileIO [iceberg]

2024-03-08 Thread via GitHub
jbonofre commented on PR #9884: URL: https://github.com/apache/iceberg/pull/9884#issuecomment-1986753344 @danielcweeks I know for FileIO, but I don't understand why we have `s3` and `s3a` in `ResolvingFileIO`, actually why do we have `ResolvingFileIO` ? My understanding is that `ResolvingFi

Re: [I] Parquet bloom filter doesn't work with nested fields [iceberg]

2024-03-08 Thread via GitHub
amogh-jahagirdar commented on issue #9898: URL: https://github.com/apache/iceberg/issues/9898#issuecomment-1986752265 Also curious which Spark version are you using? I just tested via Spark 3.4 and Spark 3.5, and bloom filters for nested type appears to be written out based on the parquet-c

Re: [PR] Core: add support to add custom schemes via properties in ResolvingFileIO [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on PR #9884: URL: https://github.com/apache/iceberg/pull/9884#issuecomment-1986751554 @jbonofre FileIO is already pluggable without this. If you have a license constrained/custom/external implementation, you can configure that directly and it can use any scheme.

Re: [PR] Add hive metastore catalog support (part 1/2) [iceberg-rust]

2024-03-08 Thread via GitHub
liurenjie1024 commented on code in PR #237: URL: https://github.com/apache/iceberg-rust/pull/237#discussion_r1518487095 ## crates/catalog/hms/src/catalog.rs: ## @@ -95,9 +106,156 @@ impl HmsCatalog { client: HmsClient(client), }) } + +/// Create an

Re: [PR] Core: add support to add custom schemes via properties in ResolvingFileIO [iceberg]

2024-03-08 Thread via GitHub
jbonofre commented on PR #9884: URL: https://github.com/apache/iceberg/pull/9884#issuecomment-1986747262 @danielcweeks I think it's not possible to Iceberg (or any project) to define/support all file schema on its own because: 1. From a legal standpoint, it's not possible (for instance, y

Re: [I] Rename `stat_table` in `Catalog` trait [iceberg-rust]

2024-03-08 Thread via GitHub
liurenjie1024 commented on issue #236: URL: https://github.com/apache/iceberg-rust/issues/236#issuecomment-1986743198 Current implementation in fact only checks table exists, and it's also the operation id in rest catalog spec: https://github.com/apache/iceberg/blob/621d301b17417af4b71a0e8c

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-08 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1518485878 ## core/src/main/java/org/apache/iceberg/BaseMetadata.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-08 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1518485878 ## core/src/main/java/org/apache/iceberg/BaseMetadata.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-08 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1518485878 ## core/src/main/java/org/apache/iceberg/BaseMetadata.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-08 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1518485714 ## core/src/main/java/org/apache/iceberg/BaseMetadata.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-08 Thread via GitHub
szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1518485623 ## core/src/main/java/org/apache/iceberg/BaseMetadata.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Allow setting non-string typed values in `set_properties` [iceberg-python]

2024-03-08 Thread via GitHub
HonahX commented on code in PR #504: URL: https://github.com/apache/iceberg-python/pull/504#discussion_r1518482846 ## pyiceberg/table/__init__.py: ## @@ -472,7 +473,11 @@ class SetLocationUpdate(TableUpdate): class SetPropertiesUpdate(TableUpdate): action: TableUpdateAct

Re: [PR] Add hive metastore catalog support (part 1/2) [iceberg-rust]

2024-03-08 Thread via GitHub
Xuanwo commented on PR #237: URL: https://github.com/apache/iceberg-rust/pull/237#issuecomment-1986734310 Also cc @liurenjie1024 to take another look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Add hive metastore catalog support (part 1/2) [iceberg-rust]

2024-03-08 Thread via GitHub
Xuanwo commented on code in PR #237: URL: https://github.com/apache/iceberg-rust/pull/237#discussion_r1518480899 ## crates/catalog/hms/Cargo.toml: ## @@ -33,5 +33,12 @@ anyhow = { workspace = true } async-trait = { workspace = true } hive_metastore = { workspace = true } iceb

Re: [I] Add OAuth Support for iceberg rest client. [iceberg-rust]

2024-03-08 Thread via GitHub
TennyZhuang commented on issue #238: URL: https://github.com/apache/iceberg-rust/issues/238#issuecomment-1986722586 I’ll take a look on the issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] feat: Implement expression projection evaluator. [iceberg-rust]

2024-03-08 Thread via GitHub
TennyZhuang commented on issue #151: URL: https://github.com/apache/iceberg-rust/issues/151#issuecomment-1986719221 I’ll work on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Core: add support to add custom schemes via properties in ResolvingFileIO [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on PR #9884: URL: https://github.com/apache/iceberg/pull/9884#issuecomment-1986717618 @jbonofre I don't necessarily think this is a good idea. The issue is that if we start using schemes that are not well-defined or standardized, then you can run into compatibility p

[PR] Core:Add test for removing partition filed [iceberg]

2024-03-08 Thread via GitHub
rfyu opened a new pull request, #9903: URL: https://github.com/apache/iceberg/pull/9903 For a table in v1 format,after deleting a partition, the new data will be stored in the partition with a null partition value. When querying this data using `FindFiles.inPartitions`, the following except

Re: [PR] Spark: Add a test to check if the bloom filters are added to the parquet files [iceberg]

2024-03-08 Thread via GitHub
amogh-jahagirdar commented on code in PR #9902: URL: https://github.com/apache/iceberg/pull/9902#discussion_r1518451202 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReaderWithBloomFilter.java: ## @@ -367,11 +374,28 @@ public void testReadWithFilter(

Re: [PR] Spark: Add a test to check if the bloom filters are added to the parquet files [iceberg]

2024-03-08 Thread via GitHub
amogh-jahagirdar commented on code in PR #9902: URL: https://github.com/apache/iceberg/pull/9902#discussion_r1518451041 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReaderWithBloomFilter.java: ## @@ -367,11 +374,28 @@ public void testReadWithFilter(

Re: [PR] Core: add missing `@Test` annotation in TestRESTCatalog [iceberg]

2024-03-08 Thread via GitHub
amogh-jahagirdar merged PR #9899: URL: https://github.com/apache/iceberg/pull/9899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core: Mark 502 and 504 statuses as retryable to the REST exponential retry strategy [iceberg]

2024-03-08 Thread via GitHub
amogh-jahagirdar merged PR #9885: URL: https://github.com/apache/iceberg/pull/9885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core: Mark 502 and 504 statuses as retryable to the REST exponential retry strategy [iceberg]

2024-03-08 Thread via GitHub
amogh-jahagirdar commented on PR #9885: URL: https://github.com/apache/iceberg/pull/9885#issuecomment-1986676541 Thanks for the review @nastra ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Parquet bloom filter doesn't work with nested fields [iceberg]

2024-03-08 Thread via GitHub
amogh-jahagirdar commented on issue #9898: URL: https://github.com/apache/iceberg/issues/9898#issuecomment-1986675844 Sounds great! I'll take a look at the PR. I spent some time debugging this today, and added a test in the same class to try and repro and I saw the same thing. In `Col

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518448903 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,41 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518448853 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,41 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518448777 ## open-api/rest-catalog-open-api.yaml: ## @@ -3804,6 +4025,21 @@ components: } } +UnprocessableEntityResponse: + description: +

Re: [I] Parquet bloom filter doesn't work with nested fields [iceberg]

2024-03-08 Thread via GitHub
hussein-awala commented on issue #9898: URL: https://github.com/apache/iceberg/issues/9898#issuecomment-1986674145 @amogh-jahagirdar I created https://github.com/apache/iceberg/pull/9902 to test if the bloom filters are added to the files, and they seem to be added for the nested field.

[PR] Spark: Add a test to check if the bloom filters are added to the parquet files [iceberg]

2024-03-08 Thread via GitHub
hussein-awala opened a new pull request, #9902: URL: https://github.com/apache/iceberg/pull/9902 related: #9898 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] Clarify pagination description [iceberg]

2024-03-08 Thread via GitHub
rdblue commented on PR #9872: URL: https://github.com/apache/iceberg/pull/9872#issuecomment-1986642645 @jackye1995, as @danielcweeks said, this was originally a much more narrow fix that got larger as we went along. Also, most of the reason why the text was replaced was reformatting (see th

Re: [PR] Clarify pagination description [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on PR #9872: URL: https://github.com/apache/iceberg/pull/9872#issuecomment-1986606686 > I am quite confused about this situation. We literally merged a spec change, and then immediately published a revision which completely rewrites the description. We could have disc

Re: [PR] Clarify pagination description [iceberg]

2024-03-08 Thread via GitHub
jackye1995 commented on PR #9872: URL: https://github.com/apache/iceberg/pull/9872#issuecomment-1986602292 I am quite confused about this situation. We literally merged a spec change, and then immediately published a revision which completely rewrites the description. We could have discusse

Re: [PR] Clarify pagination description [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on code in PR #9872: URL: https://github.com/apache/iceberg/pull/9872#discussion_r1518427240 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,19 @@ components: PageToken: description: -An opaque token which allows clients

Re: [PR] Clarify pagination description [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on code in PR #9872: URL: https://github.com/apache/iceberg/pull/9872#discussion_r1518426589 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,26 @@ components: PageToken: description: -An opaque token which allows clients

Re: [PR] Clarify pagination description [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on PR #9872: URL: https://github.com/apache/iceberg/pull/9872#issuecomment-1986589475 > How should we deal with this? There are 2 PRs opened: #9866 @jackye1995 I commented on the other PR, but I think we should close #9866 as it's missing a lot of corrections th

Re: [PR] Fix pagination spec description [iceberg]

2024-03-08 Thread via GitHub
danielcweeks commented on PR #9866: URL: https://github.com/apache/iceberg/pull/9866#issuecomment-1986589003 I didn't realize this PR was opened, but I think we should close in favor of #9872. This PR still doesn't address a number of issues that need to be clarified. -- This is an auto

Re: [I] BaseMetastoreCatalog's Base Metadata Tables use Data Table's Operations and EncryptionManager [iceberg]

2024-03-08 Thread via GitHub
github-actions[bot] commented on issue #1928: URL: https://github.com/apache/iceberg/issues/1928#issuecomment-1986587925 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] remove a list .metadata.json in HadoopCatalog [iceberg]

2024-03-08 Thread via GitHub
github-actions[bot] commented on issue #1934: URL: https://github.com/apache/iceberg/issues/1934#issuecomment-1986587939 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Hive: Schema conversion should convert NOT NULL/required fields [iceberg]

2024-03-08 Thread via GitHub
github-actions[bot] commented on issue #1922: URL: https://github.com/apache/iceberg/issues/1922#issuecomment-1986587916 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Spark Table With Timestamp Cannot Query By Flink [iceberg]

2024-03-08 Thread via GitHub
github-actions[bot] commented on issue #1916: URL: https://github.com/apache/iceberg/issues/1916#issuecomment-1986587898 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Flink 1.11.1 Not Supoort timestamptz [iceberg]

2024-03-08 Thread via GitHub
github-actions[bot] commented on issue #1915: URL: https://github.com/apache/iceberg/issues/1915#issuecomment-1986587888 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on PR #9892: URL: https://github.com/apache/iceberg/pull/9892#issuecomment-1986579621 @nastra Thanks for the review. Reflected your comments. Could you review the new one? -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[PR] Spark 3.4, 3.3 : support output-spec-id in rewrite (back port #9803) [iceberg]

2024-03-08 Thread via GitHub
himadripal opened a new pull request, #9901: URL: https://github.com/apache/iceberg/pull/9901 Backport #9803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Spark 3.4, 3.3 : support output-spec-id in rewrite (back port #9803) [iceberg]

2024-03-08 Thread via GitHub
himadripal commented on PR #9901: URL: https://github.com/apache/iceberg/pull/9901#issuecomment-1986580223 @RussellSpitzer @aokolnychyi @flyrain @szehon-ho -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Minor: Parse rowGroupSize as a long [iceberg]

2024-03-08 Thread via GitHub
hussein-awala opened a new pull request, #9900: URL: https://github.com/apache/iceberg/pull/9900 `withRowGroupSize(int rowGroupSize)` is deprecated in `ParquetWriter` and currently, the max possible rowGroupSize in Iceberg is 2GB (max Java int) which will be fixed after this PR. -- This

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518375778 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +3093,59 @@ components: additionalProperties: type: string +PreplanTableRequest:

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518332560 ## open-api/rest-catalog-open-api.yaml: ## @@ -3804,6 +4025,21 @@ components: } } +UnprocessableEntityResponse: + description: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518331988 ## open-api/rest-catalog-open-api.yaml: ## @@ -3789,6 +3995,21 @@ components: EmptyResponse: $ref: '#/components/examples/ListNamespacesE

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518330379 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,41 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518329140 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile'

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
jackye1995 commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518328719 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,41 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1517180435 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +3093,59 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1517180435 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +3093,59 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518068800 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518184139 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,108 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518183297 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,108 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518182591 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,108 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518181730 ## open-api/rest-catalog-open-api.yaml: ## @@ -3789,6 +4097,21 @@ components: EmptyResponse: $ref: '#/components/examples/ListNamespacesEmpt

Re: [PR] Feature make oauth `audience` configurable [iceberg]

2024-03-08 Thread via GitHub
flyrain commented on code in PR #9839: URL: https://github.com/apache/iceberg/pull/9839#discussion_r1518181685 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -471,7 +515,13 @@ static void setTokenRefreshNumRetries(int retries) { */ public st

Re: [I] Getting java.lang.NoSuchMethodError org.apache.iceberg.util.JsonUtil.mapper() With Hive catalog [iceberg]

2024-03-08 Thread via GitHub
ismailsimsek commented on issue #9848: URL: https://github.com/apache/iceberg/issues/9848#issuecomment-1986226824 Fixed https://github.com/memiiso/debezium-server-iceberg/issues/271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Getting java.lang.NoSuchMethodError org.apache.iceberg.util.JsonUtil.mapper() With Hive catalog [iceberg]

2024-03-08 Thread via GitHub
ismailsimsek closed issue #9848: Getting java.lang.NoSuchMethodError org.apache.iceberg.util.JsonUtil.mapper() With Hive catalog URL: https://github.com/apache/iceberg/issues/9848 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518118829 ## open-api/rest-catalog-open-api.yaml: ## @@ -3789,6 +4097,21 @@ components: EmptyResponse: $ref: '#/components/examples/ListNamespacesEmpt

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518113629 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,108 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518068800 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518075237 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518075237 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518068800 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-08 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1518068800 ## open-api/rest-catalog-open-api.yaml: ## @@ -2068,6 +2162,145 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Allow setting non-string typed values in `set_properties` [iceberg-python]

2024-03-08 Thread via GitHub
kevinjqliu commented on PR #504: URL: https://github.com/apache/iceberg-python/pull/504#issuecomment-1985996781 rebased after #503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Add hive metastore catalog support [iceberg-rust]

2024-03-08 Thread via GitHub
marvinlanhenke commented on issue #113: URL: https://github.com/apache/iceberg-rust/issues/113#issuecomment-1985986494 > In short, submit a PR and then we can polish it together. Sounds good to me - looking forward to it. Best regards. -- This is an automated message from the

[PR] Add hive metastore catalog support (part 1/2) [iceberg-rust]

2024-03-08 Thread via GitHub
marvinlanhenke opened a new pull request, #237: URL: https://github.com/apache/iceberg-rust/pull/237 ### Which issue does this PR close? Closes #113 partly. ### Rationale for this change Add hive metastore support, to reach feature parity with other iceberg implementations.

[PR] Core: add missing `@Test` annotation in TestRESTCatalog [iceberg]

2024-03-08 Thread via GitHub
adutra opened a new pull request, #9899: URL: https://github.com/apache/iceberg/pull/9899 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [I] Rename `stat_table` in `Catalog` trait [iceberg-rust]

2024-03-08 Thread via GitHub
marvinlanhenke commented on issue #236: URL: https://github.com/apache/iceberg-rust/issues/236#issuecomment-1985944355 > I remember that `stat_table` will also return some metadata of this table. I only took a look at the java & python hive implementation and I couldn't find a similia

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on code in PR #9892: URL: https://github.com/apache/iceberg/pull/9892#discussion_r1517877405 ## core/src/test/java/org/apache/iceberg/TestSnapshotManager.java: ## @@ -700,81 +689,82 @@ public void testAttemptToRollbackToCurrentSnapshot() { table.manageSn

Re: [I] Library public api isolation and import decoupling [iceberg-python]

2024-03-08 Thread via GitHub
ndrluis commented on issue #499: URL: https://github.com/apache/iceberg-python/issues/499#issuecomment-1985907062 Issue to follow up https://github.com/astral-sh/ruff/issues/10300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on code in PR #9892: URL: https://github.com/apache/iceberg/pull/9892#discussion_r1517865244 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -49,12 +50,23 @@ public void testJsonConversion() throws IOException { String json = Snapsh

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on code in PR #9892: URL: https://github.com/apache/iceberg/pull/9892#discussion_r1517865244 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -49,12 +50,23 @@ public void testJsonConversion() throws IOException { String json = Snapsh

Re: [I] Rename `stat_table` in `Catalog` trait [iceberg-rust]

2024-03-08 Thread via GitHub
Xuanwo commented on issue #236: URL: https://github.com/apache/iceberg-rust/issues/236#issuecomment-1985889243 I remember that `stat_table` will also return some metadata of this table. cc @Fokko @liurenjie1024, any comments? -- This is an automated message from the Apache Git Servi

Re: [I] Add hive metastore catalog support [iceberg-rust]

2024-03-08 Thread via GitHub
Xuanwo commented on issue #113: URL: https://github.com/apache/iceberg-rust/issues/113#issuecomment-1985888081 > I 'finished' the work on all the namespace operations. Awesome! > I just want to make sure it aligns with the rest of the codebase, before I put in the effort to imp

Re: [PR] Flink: implement range partitioner for map data statistics [iceberg]

2024-03-08 Thread via GitHub
stevenzwu commented on code in PR #9321: URL: https://github.com/apache/iceberg/pull/9321#discussion_r1517846647 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/MapRangePartitioner.java: ## @@ -0,0 +1,378 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on code in PR #9892: URL: https://github.com/apache/iceberg/pull/9892#discussion_r1517839588 ## core/src/test/java/org/apache/iceberg/TestSnapshotManager.java: ## @@ -177,120 +177,115 @@ public void testCherryPickFromBranch() { long lastSnapshotId = tabl

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on code in PR #9892: URL: https://github.com/apache/iceberg/pull/9892#discussion_r1517838222 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -1731,7 +1730,7 @@ public void testNoTrailingLocationSlash() { private String createManife

[I] Parquet bloom filter doesn't work with nested fields [iceberg]

2024-03-08 Thread via GitHub
hussein-awala opened a new issue, #9898: URL: https://github.com/apache/iceberg/issues/9898 ### Apache Iceberg version 1.4.3 (latest release) ### Query engine Spark ### Please describe the bug 🐞 I have an Iceberg table, and I want to create two bloom filters

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on code in PR #9892: URL: https://github.com/apache/iceberg/pull/9892#discussion_r1517827090 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -19,23 +19,24 @@ package org.apache.iceberg; import static org.apache.iceberg.Files.localInp

Re: [PR] Migrate Snapshot files in Core to JUnit5 [iceberg]

2024-03-08 Thread via GitHub
tomtongue commented on code in PR #9892: URL: https://github.com/apache/iceberg/pull/9892#discussion_r1517821079 ## core/src/test/java/org/apache/iceberg/LocalTableOperations.java: ## @@ -63,7 +65,9 @@ public String metadataFileLocation(String fileName) { fileName,

Re: [PR] Build: Bump com.diffplug.spotless:spotless-plugin-gradle from 6.13.0 to 6.25.0 [iceberg]

2024-03-08 Thread via GitHub
nastra closed pull request #9569: Build: Bump com.diffplug.spotless:spotless-plugin-gradle from 6.13.0 to 6.25.0 URL: https://github.com/apache/iceberg/pull/9569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Build: Bump com.diffplug.spotless:spotless-plugin-gradle from 6.13.0 to 6.25.0 [iceberg]

2024-03-08 Thread via GitHub
dependabot[bot] commented on PR #9569: URL: https://github.com/apache/iceberg/pull/9569#issuecomment-1985765560 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [I] Add hive metastore catalog support [iceberg-rust]

2024-03-08 Thread via GitHub
marvinlanhenke commented on issue #113: URL: https://github.com/apache/iceberg-rust/issues/113#issuecomment-1985760454 @Xuanwo @liurenjie1024 ...just a quick update and a request for feedback. I 'finished' the work on all the namespace operations. Since this would be my first co

Re: [PR] HIVE-28021: escape percent symbol [iceberg]

2024-03-08 Thread via GitHub
nastra commented on code in PR #9667: URL: https://github.com/apache/iceberg/pull/9667#discussion_r1517758155 ## mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java: ## @@ -966,4 +966,41 @@ private String getCurrentSnapshotForHiveCatalogTable(or

Re: [PR] Build: Bump org.openapitools:openapi-generator-gradle-plugin from 6.6.0 to 7.3.0 [iceberg]

2024-03-08 Thread via GitHub
dependabot[bot] commented on PR #9703: URL: https://github.com/apache/iceberg/pull/9703#issuecomment-1985747945 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Build: Bump org.openapitools:openapi-generator-gradle-plugin from 6.6.0 to 7.3.0 [iceberg]

2024-03-08 Thread via GitHub
nastra closed pull request #9703: Build: Bump org.openapitools:openapi-generator-gradle-plugin from 6.6.0 to 7.3.0 URL: https://github.com/apache/iceberg/pull/9703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.24.1 to 2.25.0 [iceberg]

2024-03-08 Thread via GitHub
dependabot[bot] commented on PR #9746: URL: https://github.com/apache/iceberg/pull/9746#issuecomment-1985746185 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.24.1 to 2.25.0 [iceberg]

2024-03-08 Thread via GitHub
nastra closed pull request #9746: Build: Bump com.google.errorprone:error_prone_annotations from 2.24.1 to 2.25.0 URL: https://github.com/apache/iceberg/pull/9746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

  1   2   >