Re: [PR] #5695 Increasing test coverage pinot-common [pinot]

2024-02-26 Thread via GitHub
gortiz commented on PR #12294: URL: https://github.com/apache/pinot/pull/12294#issuecomment-1965939547 Merged, thanks for this contribution and my personal apologies for taking so much time to merge it. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] #5695 Increasing test coverage pinot-common [pinot]

2024-02-26 Thread via GitHub
gortiz merged PR #12294: URL: https://github.com/apache/pinot/pull/12294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: Lint failure (#12294)

2024-02-26 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 74989c0191 Lint failure (#12294) 74989c0191 is de

(pinot) branch master updated: Adds per-column, query-time index skip option (#12414)

2024-02-26 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 5dc807bb0a Adds per-column, query-time index skip

Re: [I] Add QueryOption support for column-level index skipping at query time [pinot]

2024-02-26 Thread via GitHub
gortiz closed issue #12355: Add QueryOption support for column-level index skipping at query time URL: https://github.com/apache/pinot/issues/12355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Adds per-column, query-time index skip option [pinot]

2024-02-26 Thread via GitHub
gortiz merged PR #12414: URL: https://github.com/apache/pinot/pull/12414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

(pinot) branch master updated: Add config option for timezone (#12386)

2024-02-26 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new d9ac0efa7d Add config option for timezone (#1238

Re: [I] Allow setup timezone in cluster config or config file. [pinot]

2024-02-26 Thread via GitHub
xiangfu0 closed issue #12299: Allow setup timezone in cluster config or config file. URL: https://github.com/apache/pinot/issues/12299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Add config option for timezone [pinot]

2024-02-26 Thread via GitHub
xiangfu0 merged PR #12386: URL: https://github.com/apache/pinot/pull/12386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Add config option for timezone [pinot]

2024-02-26 Thread via GitHub
xiangfu0 commented on PR #12386: URL: https://github.com/apache/pinot/pull/12386#issuecomment-1965926125 Cool! Please add a section of Release notes in the PR description, so next release manager will just pick it up :) -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Add controller API to get allLiveInstances [pinot]

2024-02-26 Thread via GitHub
ankitsultana commented on code in PR #12498: URL: https://github.com/apache/pinot/pull/12498#discussion_r1503731956 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotInstanceRestletResource.java: ## @@ -108,6 +108,19 @@ public Instances getAllInsta

(pinot) branch master updated: auto renew jvm default sslconext when it's loaded from files (#12462)

2024-02-26 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new b42010a1ec auto renew jvm default sslconext when

Re: [PR] auto renew jvm default sslconext when it's loaded from files [pinot]

2024-02-26 Thread via GitHub
xiangfu0 merged PR #12462: URL: https://github.com/apache/pinot/pull/12462 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503692048 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-26 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1503671751 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,90 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-26 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1503669959 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,90 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-26 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1503667801 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -1660,10 +1640,25 @@ static String getActualColumnName(St

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503665318 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-26 Thread via GitHub
shounakmk219 commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1503661878 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java: ## @@ -1660,10 +1640,25 @@ static String getActualColumnName(St

(pinot) branch master updated: Record enricher (#12243)

2024-02-26 Thread saurabhd336
This is an automated email from the ASF dual-hosted git repository. saurabhd336 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 07daa7bbaa Record enricher (#12243) 07daa7bb

Re: [PR] Record enricher [pinot]

2024-02-26 Thread via GitHub
saurabhd336 merged PR #12243: URL: https://github.com/apache/pinot/pull/12243 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.

Re: [I] Is pinot possible to support batch Ingestion with Upsert? [pinot]

2024-02-26 Thread via GitHub
rohityadav1993 commented on issue #10896: URL: https://github.com/apache/pinot/issues/10896#issuecomment-1965753872 The segments are partitioned similarly how the stream is pre-partitioned during realtime ingestion. The challenge is validations that are in place, segments have to follow a n

Re: [PR] Custom configuration property reader for segment metadata files [pinot]

2024-02-26 Thread via GitHub
abhioncbr commented on code in PR #12440: URL: https://github.com/apache/pinot/pull/12440#discussion_r1503606292 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/SegmentMetadataPropertyReader.java: ## @@ -0,0 +1,35 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] fix GuavaRateLimiter acquire method [pinot]

2024-02-26 Thread via GitHub
sullis commented on PR #12500: URL: https://github.com/apache/pinot/pull/12500#issuecomment-1965699584 ready to merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

(pinot) branch master updated: int96 parity with native parquet reader (#12496)

2024-02-26 Thread snlee
This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 3267a749f8 int96 parity with native parquet reader

Re: [PR] ParquetAvro int96 parity with parquetNative reader [pinot]

2024-02-26 Thread via GitHub
snleee merged PR #12496: URL: https://github.com/apache/pinot/pull/12496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[I] Configurable heap size for query and realtime segment persistence. [pinot]

2024-02-26 Thread via GitHub
Lvnszn opened a new issue, #12503: URL: https://github.com/apache/pinot/issues/12503 When pinot query data, pinot will load the segment from disk to offheap, but when persisting segment, pinot need to use offheap. This memory will interact with each other and have a probability of causing w

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on code in PR #12502: URL: https://github.com/apache/pinot/pull/12502#discussion_r1503471880 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/columnminmaxvalue/ColumnMinMaxValueGenerator.java: ## @@ -124,227 +131,251

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on code in PR #12502: URL: https://github.com/apache/pinot/pull/12502#discussion_r1503471374 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/columnminmaxvalue/ColumnMinMaxValueGenerator.java: ## @@ -124,227 +131,251

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on code in PR #12502: URL: https://github.com/apache/pinot/pull/12502#discussion_r1503469768 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -639,64 +636,17 @@ public static v

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
abhioncbr commented on code in PR #12502: URL: https://github.com/apache/pinot/pull/12502#discussion_r1503454225 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -639,64 +636,17 @@ public static void

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
abhioncbr commented on code in PR #12502: URL: https://github.com/apache/pinot/pull/12502#discussion_r1503454225 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java: ## @@ -639,64 +636,17 @@ public static void

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
codecov-commenter commented on PR #12502: URL: https://github.com/apache/pinot/pull/12502#issuecomment-1965534131 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12502?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] Allow passing database context through `database` http header [pinot]

2024-02-26 Thread via GitHub
klsince commented on code in PR #12417: URL: https://github.com/apache/pinot/pull/12417#discussion_r1503447394 ## pinot-common/src/main/java/org/apache/pinot/common/utils/DatabaseUtils.java: ## @@ -0,0 +1,90 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] do not throw exception for unsupported min/max no dictionary columns [pinot]

2024-02-26 Thread via GitHub
jadami10 commented on PR #12499: URL: https://github.com/apache/pinot/pull/12499#issuecomment-1965526248 jackie's PR is better than mine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] do not throw exception for unsupported min/max no dictionary columns [pinot]

2024-02-26 Thread via GitHub
jadami10 closed pull request #12499: do not throw exception for unsupported min/max no dictionary columns URL: https://github.com/apache/pinot/pull/12499 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [SQL Compatible] Fix literal type handling [pinot]

2024-02-26 Thread via GitHub
elonazoulay commented on PR #11763: URL: https://github.com/apache/pinot/pull/11763#issuecomment-1965524968 This is great, thanks so much for fixing this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
jadami10 commented on code in PR #12502: URL: https://github.com/apache/pinot/pull/12502#discussion_r1503435128 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/columnminmaxvalue/ColumnMinMaxValueGenerator.java: ## @@ -124,227 +131,251 @@

Re: [PR] do not throw exception for unsupported min/max no dictionary columns [pinot]

2024-02-26 Thread via GitHub
codecov-commenter commented on PR #12499: URL: https://github.com/apache/pinot/pull/12499#issuecomment-1965514898 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12499?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] fix GuavaRateLimiter acquire method [pinot]

2024-02-26 Thread via GitHub
codecov-commenter commented on PR #12500: URL: https://github.com/apache/pinot/pull/12500#issuecomment-1965512740 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12500?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] fix GuavaRateLimiter acquire method [pinot]

2024-02-26 Thread via GitHub
sullis commented on code in PR #12500: URL: https://github.com/apache/pinot/pull/12500#discussion_r1503433065 ## pinot-tools/src/test/java/org/apache/pinot/tools/streams/PinotRealtimeSourceTest.java: ## @@ -49,6 +49,28 @@ public void testBuilder() { Assert.assertEquals(real

Re: [PR] fix GuavaRateLimiter acquire method [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on code in PR #12500: URL: https://github.com/apache/pinot/pull/12500#discussion_r1503427564 ## pinot-tools/src/test/java/org/apache/pinot/tools/streams/PinotRealtimeSourceTest.java: ## @@ -49,6 +49,28 @@ public void testBuilder() { Assert.assertEqual

Re: [PR] do not throw exception for unsupported min/max no dictionary columns [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on PR #12499: URL: https://github.com/apache/pinot/pull/12499#issuecomment-1965492819 I did a more thorough fix in #12502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on PR #12502: URL: https://github.com/apache/pinot/pull/12502#issuecomment-1965492288 @jadami10 Please take a look @abhioncbr I have to revert some of the changes in #10990 as I just realize we cannot keep approximate value as min/max because they can be used as

[PR] Fix ColumnMinMaxValueGenerator [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang opened a new pull request, #12502: URL: https://github.com/apache/pinot/pull/12502 Fix the following issues: - Support generating min/max value for BIG_DECIMAL column - Do not abort the segment load when min/max value cannot be generated - Do not persist min/max value wh

[PR] fix GuavaRateLimiter acquire method [pinot]

2024-02-26 Thread via GitHub
sullis opened a new pull request, #12500: URL: https://github.com/apache/pinot/pull/12500 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] do not throw exception for unsupported min/max no dictionary columns [pinot]

2024-02-26 Thread via GitHub
jadami10 commented on PR #12499: URL: https://github.com/apache/pinot/pull/12499#issuecomment-1965470584 cc @Jackie-Jiang are you 1. ok with this approach for now 2. know of any new assumptions that have been made that all segments must have min/max in their column metadata -- This

[PR] do not throw exception for unsupported min/max no dictionary columns [pinot]

2024-02-26 Thread via GitHub
jadami10 opened a new pull request, #12499: URL: https://github.com/apache/pinot/pull/12499 This is a bugfix related to #10891. That PR added support for setting min/max column metadata for no dictionary columns, but it didn't add support for all data types. This specifically affected us be

Re: [I] Change Future to CompletableFuture in java client [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang closed issue #10325: Change Future to CompletableFuture in java client URL: https://github.com/apache/pinot/issues/10325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] Support modifying/removing indexes for v3 segment format [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang closed issue #6114: Support modifying/removing indexes for v3 segment format URL: https://github.com/apache/pinot/issues/6114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] Alternative HyperLogLog implementation with higher accuracy [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang closed issue #4248: Alternative HyperLogLog implementation with higher accuracy URL: https://github.com/apache/pinot/issues/4248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] Upsert Enhancements [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang closed issue #8760: Upsert Enhancements URL: https://github.com/apache/pinot/issues/8760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: co

Re: [PR] Add config option for timezone [pinot]

2024-02-26 Thread via GitHub
dttung2905 commented on PR #12386: URL: https://github.com/apache/pinot/pull/12386#issuecomment-1965369289 I will update the gitbook with a separate PR for docs, not sure where to update for the release note. Could you point me to it? -- This is an automated message from the Apache Git Se

Re: [PR] Custom configuration property reader for segment metadata files [pinot]

2024-02-26 Thread via GitHub
klsince commented on code in PR #12440: URL: https://github.com/apache/pinot/pull/12440#discussion_r1503346467 ## pinot-spi/src/main/java/org/apache/pinot/spi/env/SegmentMetadataPropertyReader.java: ## @@ -0,0 +1,35 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Add config option for timezone [pinot]

2024-02-26 Thread via GitHub
dttung2905 commented on PR #12386: URL: https://github.com/apache/pinot/pull/12386#issuecomment-1965347684 > Please also add a release note section, mention how to configure it as well as that all the default deployment will use UTC instead of local time zone by default, which is backward i

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
zhtaoxiang commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503291081 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Fix MergeRollup generator bugs [pinot]

2024-02-26 Thread via GitHub
jtao15 commented on PR #12092: URL: https://github.com/apache/pinot/pull/12092#issuecomment-1965182845 @snleee Added unit tests, and also fix another corner case for merge level ordering, PTAL. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Ensure min/max value generation in the segment metadata. [pinot]

2024-02-26 Thread via GitHub
jadami10 commented on code in PR #10891: URL: https://github.com/apache/pinot/pull/10891#discussion_r1503229608 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/columnminmaxvalue/ColumnMinMaxValueGenerator.java: ## @@ -119,65 +121,126 @@ p

Re: [PR] Add config option for timezone [pinot]

2024-02-26 Thread via GitHub
xiangfu0 commented on PR #12386: URL: https://github.com/apache/pinot/pull/12386#issuecomment-1965074955 Please also add a release note section, mention how to configure it as well as that all the default deployment will use UTC instead of local time zone by default, which is backward incom

Re: [PR] Ensure min/max value generation in the segment metadata. [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on code in PR #10891: URL: https://github.com/apache/pinot/pull/10891#discussion_r1503177630 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/columnminmaxvalue/ColumnMinMaxValueGenerator.java: ## @@ -119,65 +121,126

Re: [PR] Adds per-column, query-time index skip option [pinot]

2024-02-26 Thread via GitHub
egalpin commented on PR #12414: URL: https://github.com/apache/pinot/pull/12414#issuecomment-1964823595 @gortiz anything that needs attention at the moment? I want to be sure I've addressed any comments and can't find any unaddressed at the moment. -- This is an automated message from the

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503076566 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503068074 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503074799 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503068074 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
zhtaoxiang commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1503059811 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

(pinot) branch master updated: Remove all table metrics when a table is deleted (#12403)

2024-02-26 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 8977c85e4f Remove all table metrics when a table

Re: [PR] Remove all table metrics when a table is deleted [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang merged PR #12403: URL: https://github.com/apache/pinot/pull/12403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [I] Is pinot possible to support batch Ingestion with Upsert? [pinot]

2024-02-26 Thread via GitHub
Jackie-Jiang commented on issue #10896: URL: https://github.com/apache/pinot/issues/10896#issuecomment-1964733773 In order to update offline segments to upsert table, they need to be partitioned. @chenboat Could you please share the steps of bootstrapping an upsert table with offline seg

Re: [PR] Ensure min/max value generation in the segment metadata. [pinot]

2024-02-26 Thread via GitHub
jadami10 commented on code in PR #10891: URL: https://github.com/apache/pinot/pull/10891#discussion_r1503020298 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/columnminmaxvalue/ColumnMinMaxValueGenerator.java: ## @@ -119,65 +121,126 @@ p

Re: [PR] Increasing test coverage pinot-common [pinot]

2024-02-26 Thread via GitHub
aditya0811 commented on PR #12294: URL: https://github.com/apache/pinot/pull/12294#issuecomment-1964622269 > Can you please take a look at the linter failure? It would be very helpful if you can also include how many missing lines are covered within this new test Fixed. covers 79 miss

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1502968851 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1502967065 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1502967065 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1502966292 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Increasing test coverage pinot-common [pinot]

2024-02-26 Thread via GitHub
aditya0811 commented on PR #12294: URL: https://github.com/apache/pinot/pull/12294#issuecomment-1964602728 > It seems there are some compilation issues. Can you fix them? updated with latest code, constructor changed -- This is an automated message from the Apache Git Service. To re

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1502963805 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1502963805 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( }

Re: [PR] Record enricher [pinot]

2024-02-26 Thread via GitHub
gortiz commented on PR #12243: URL: https://github.com/apache/pinot/pull/12243#issuecomment-1964598672 Another small detail: can we standarise the name of the columns to add? CLP is using `fields` while groovy uses `columns` -- This is an automated message from the Apache Git Service. To

(pinot) branch master updated (dcbaca1f1a -> a1ed8bdb62)

2024-02-26 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from dcbaca1f1a bump maven-dependency-plugin to 3.6.1 (#12495) add a1ed8bdb62 close JDBC connections (#12494) No new

Re: [PR] close JDBC connections [pinot]

2024-02-26 Thread via GitHub
xiangfu0 merged PR #12494: URL: https://github.com/apache/pinot/pull/12494 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Add the support to extract json array elements from json index for the transform function jsonExtractIndex [pinot]

2024-02-26 Thread via GitHub
wirybeaver commented on code in PR #12466: URL: https://github.com/apache/pinot/pull/12466#discussion_r1502008766 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/json/ImmutableJsonIndexReader.java: ## @@ -160,94 +161,26 @@ private Mutabl

(pinot) branch master updated: bump maven-dependency-plugin to 3.6.1 (#12495)

2024-02-26 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new dcbaca1f1a bump maven-dependency-plugin to 3.6.1

Re: [PR] bump maven-dependency-plugin to 3.6.1 [pinot]

2024-02-26 Thread via GitHub
xiangfu0 merged PR #12495: URL: https://github.com/apache/pinot/pull/12495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

Re: [PR] Propagate Disable User Agent Config to Http Client [pinot]

2024-02-26 Thread via GitHub
satishd commented on PR #12479: URL: https://github.com/apache/pinot/pull/12479#issuecomment-1964320158 We can extract a method like below and add a UT for this method to check whether the `HttpClientBuilder` instance is updated with all the targeted configs passed to `HttpClientConfig` whi

Re: [PR] [draft] Mode null benchmark [pinot]

2024-02-26 Thread via GitHub
gortiz commented on PR #12354: URL: https://github.com/apache/pinot/pull/12354#issuecomment-1964228389 I've created another benchmark where the null distribution is not constant but random while keeping a single null value per given number of rows (aka interval). Also in order to mes

Re: [PR] [draft] Mode null benchmark [pinot]

2024-02-26 Thread via GitHub
gortiz commented on PR #12354: URL: https://github.com/apache/pinot/pull/12354#issuecomment-1964199109 Extra investigations: ## Why performance in `normal` with `null handling` decreases so much when going from 2 to 4 interval? I've checked with perfnorm: ``` Benchmark

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
xiangfu0 commented on code in PR #12497: URL: https://github.com/apache/pinot/pull/12497#discussion_r1502493581 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ZookeeperResource.java: ## @@ -256,6 +257,56 @@ public SuccessResponse putData( } }

Re: [PR] Add controller API to get allLiveInstances [pinot]

2024-02-26 Thread via GitHub
codecov-commenter commented on PR #12498: URL: https://github.com/apache/pinot/pull/12498#issuecomment-1963983090 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12498?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

[PR] Add controller API to get allLiveInstances [pinot]

2024-02-26 Thread via GitHub
tibrewalpratik17 opened a new pull request, #12498: URL: https://github.com/apache/pinot/pull/12498 label: api Adding a controller API to get all live instances in the cluster. - We fetch all instances for a cluster for attribution internally. Some of them are dead and cause

Re: [PR] Record enricher [pinot]

2024-02-26 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1502430862 ## pinot-spi/src/main/java/org/apache/pinot/spi/config/table/ingestion/EnrichmentConfig.java: ## @@ -0,0 +1,49 @@ +/** + * Licensed to the Apache Software Foundatio

Re: [PR] Record enricher [pinot]

2024-02-26 Thread via GitHub
saurabhd336 commented on code in PR #12243: URL: https://github.com/apache/pinot/pull/12243#discussion_r1502428922 ## pinot-spi/src/main/java/org/apache/pinot/spi/recordenricher/RecordEnricherFactoryInterface.java: ## @@ -0,0 +1,29 @@ +/** + * Licensed to the Apache Software Fou

Re: [PR] [draft] Mode null benchmark [pinot]

2024-02-26 Thread via GitHub
gortiz commented on PR #12354: URL: https://github.com/apache/pinot/pull/12354#issuecomment-1963827711 So the speedup in the new operations is explained because we are actually doing a better work in the new implementations by using primitive values. These implementations: - Are faster b

Re: [PR] Rest Endpoint to Create ZNode [pinot]

2024-02-26 Thread via GitHub
codecov-commenter commented on PR #12497: URL: https://github.com/apache/pinot/pull/12497#issuecomment-1963780351 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12497?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) R

Re: [PR] [draft] Mode null benchmark [pinot]

2024-02-26 Thread via GitHub
gortiz commented on PR #12354: URL: https://github.com/apache/pinot/pull/12354#issuecomment-1963704893 BTW, there is another change in this implementation (although I guess we are not planning to merge it). Results in the current implementation are less precise than the new implement

[PR] Rest Endpoint for Create ZNode [pinot]

2024-02-26 Thread via GitHub
suddendust opened a new pull request, #12497: URL: https://github.com/apache/pinot/pull/12497 The only way to create a znode right now via the controller API is using `/zk/put`. However, this overwrites any previous data already existing at the given path. There are certain use-cases in whi

Re: [PR] Remove all table metrics when a table is deleted [pinot]

2024-02-26 Thread via GitHub
gortiz commented on code in PR #12403: URL: https://github.com/apache/pinot/pull/12403#discussion_r1502209924 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/SegmentStatusChecker.java: ## @@ -354,39 +356,33 @@ protected void nonLeaderCleanup(List tableNamesW

Re: [PR] Increasing test coverage pinot-common [pinot]

2024-02-26 Thread via GitHub
gortiz commented on PR #12294: URL: https://github.com/apache/pinot/pull/12294#issuecomment-1963562714 It seems there are some compilation issues. Can you fix them? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] InterruptedException when acquiring partitonConsumerSemaphore [pinot]

2024-02-26 Thread via GitHub
ankitsultana commented on issue #12390: URL: https://github.com/apache/pinot/issues/12390#issuecomment-1963528138 I am not well versed with Helix, but I am kinda sure that the server started from OFFLINE state. There's this log coming from `HelixStateMachineEngine`: `Resetting HelixS