Re: [PR] Improve performance of DataBlock serde [pinot]

2024-07-19 Thread via GitHub
siddharthteotia commented on PR #13303: URL: https://github.com/apache/pinot/pull/13303#issuecomment-2240831452 @gortiz - are you planning to merge this soon ? I had actually reviewed this partially in June when the work was started but then lost track of it. I want to have either or

Re: [PR] Fix query response time metric [pinot]

2024-07-19 Thread via GitHub
Jackie-Jiang commented on PR #13661: URL: https://github.com/apache/pinot/pull/13661#issuecomment-2240801156 > The BaseBrokerStarter class returns the delegate class from the getBrokerRequestHandler() handler. So, this class (and the handleRequest() methods within this class) are meant to

(pinot) branch master updated (8db823b92b -> e58d74e3e9)

2024-07-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 8db823b92b Bump com.azure:azure-storage-file-datalake from 12.19.1 to 12.20.0 (#13663) add e58d74e3e9 Bump

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.26.21 deleted (was 83010cc236)

2024-07-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.26.21 in repository https://gitbox.apache.org/repos/asf/pinot.git was 83010cc236 Bump software.amazon.awssdk:bom from 2.26.20 to 2.26.21 The

Re: [PR] Bump com.azure:azure-storage-file-datalake from 12.19.1 to 12.20.0 [pinot]

2024-07-19 Thread via GitHub
Jackie-Jiang merged PR #13663: URL: https://github.com/apache/pinot/pull/13663 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(pinot) branch dependabot/maven/com.azure-azure-storage-file-datalake-12.20.0 deleted (was b37dddd806)

2024-07-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.azure-azure-storage-file-datalake-12.20.0 in repository https://gitbox.apache.org/repos/asf/pinot.git was b37806 Bump com.azure:azure-storage-file-datalake from

Re: [PR] Bump software.amazon.awssdk:bom from 2.26.20 to 2.26.21 [pinot]

2024-07-19 Thread via GitHub
Jackie-Jiang merged PR #13665: URL: https://github.com/apache/pinot/pull/13665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Bump com.google.errorprone:error_prone_annotations from 2.29.1 to 2.29.2 [pinot]

2024-07-19 Thread via GitHub
Jackie-Jiang merged PR #13662: URL: https://github.com/apache/pinot/pull/13662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(pinot) branch master updated: Bump com.azure:azure-storage-file-datalake from 12.19.1 to 12.20.0 (#13663)

2024-07-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 8db823b92b Bump

(pinot) branch master updated: Bump com.google.errorprone:error_prone_annotations from 2.29.1 to 2.29.2 (#13662)

2024-07-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 502ca20ad5 Bump

(pinot) branch dependabot/maven/com.google.errorprone-error_prone_annotations-2.29.2 deleted (was 412ac55507)

2024-07-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch dependabot/maven/com.google.errorprone-error_prone_annotations-2.29.2 in repository https://gitbox.apache.org/repos/asf/pinot.git was 412ac55507 Bump

Re: [PR] Fix query response time metric [pinot]

2024-07-19 Thread via GitHub
mcvsubbu commented on PR #13661: URL: https://github.com/apache/pinot/pull/13661#issuecomment-2240793647 > Currently, we always set request arrival time when the `RequestContext` is just created (since it is a property within it). Are you running some some custom code which is directly

Re: [PR] Fix colocated join when no mapping project is used [pinot]

2024-07-19 Thread via GitHub
Jackie-Jiang commented on code in PR #13666: URL: https://github.com/apache/pinot/pull/13666#discussion_r1685167251 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRelDistributionTraitRule.java: ## @@ -110,10 +117,22 @@ private static

Re: [PR] Improve realtime Lucene text index freshness/cpu/disk io usage [pinot]

2024-07-19 Thread via GitHub
chenboat commented on code in PR #13503: URL: https://github.com/apache/pinot/pull/13503#discussion_r1685164385 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeLuceneIndexRefreshManager.java: ## @@ -0,0 +1,268 @@ +/** + *

Re: [PR] Improve realtime Lucene text index freshness/cpu/disk io usage [pinot]

2024-07-19 Thread via GitHub
chenboat commented on code in PR #13503: URL: https://github.com/apache/pinot/pull/13503#discussion_r1685163993 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/LuceneNRTCachingMergePolicy.java: ## @@ -0,0 +1,60 @@ +/** + *

Re: [PR] Storage Quota imposition on Realtime tables [pinot]

2024-07-19 Thread via GitHub
sajjad-moradi commented on PR #13584: URL: https://github.com/apache/pinot/pull/13584#issuecomment-2240321105 > Mostly good. Can we add a test to verify this? > > @swaminathanmanish @sajjad-moradi please also take a look Will review next week. -- This is an automated message

(pinot) branch master updated (205a75debb -> 2ccfa74a8e)

2024-07-19 Thread gortiz
This is an automated email from the ASF dual-hosted git repository. gortiz pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 205a75debb API to expose gauge metric values on minion (#13550) add 2ccfa74a8e Disallow duplicate CTE aliases

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
gortiz merged PR #13664: URL: https://github.com/apache/pinot/pull/13664 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix colocated join when no mapping project is used [pinot]

2024-07-19 Thread via GitHub
codecov-commenter commented on PR #13666: URL: https://github.com/apache/pinot/pull/13666#issuecomment-2239391365 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13666?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
yashmayya commented on code in PR #13664: URL: https://github.com/apache/pinot/pull/13664#discussion_r1684468878 ## pinot-query-planner/src/test/java/org/apache/pinot/query/QueryCompilationTest.java: ## @@ -385,6 +385,12 @@ public void testGetTableNamesForQuery() {

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
yashmayya commented on code in PR #13664: URL: https://github.com/apache/pinot/pull/13664#discussion_r1684468878 ## pinot-query-planner/src/test/java/org/apache/pinot/query/QueryCompilationTest.java: ## @@ -385,6 +385,12 @@ public void testGetTableNamesForQuery() {

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
yashmayya commented on code in PR #13664: URL: https://github.com/apache/pinot/pull/13664#discussion_r1684469206 ## pinot-query-planner/src/test/java/org/apache/pinot/query/QueryCompilationTest.java: ## @@ -385,10 +385,15 @@ public void testGetTableNamesForQuery() {

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
yashmayya commented on code in PR #13664: URL: https://github.com/apache/pinot/pull/13664#discussion_r1684468878 ## pinot-query-planner/src/test/java/org/apache/pinot/query/QueryCompilationTest.java: ## @@ -385,6 +385,12 @@ public void testGetTableNamesForQuery() {

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13664: URL: https://github.com/apache/pinot/pull/13664#discussion_r1684467779 ## pinot-query-planner/src/test/java/org/apache/pinot/query/QueryCompilationTest.java: ## @@ -385,10 +385,15 @@ public void testGetTableNamesForQuery() {

Re: [I] Multi-stage: Perf issue when IN expression has a lot of entries [pinot]

2024-07-19 Thread via GitHub
gortiz commented on issue #13617: URL: https://github.com/apache/pinot/issues/13617#issuecomment-2239302248 We decided to merge #13614 given its impact in other queries is smaller than #13605. We still have the performance issue during validation time, but we need to explore our options a

Re: [PR] Improve performance of DataBlock serde [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13303: URL: https://github.com/apache/pinot/pull/13303#discussion_r1684456687 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/BaseDataBlock.java: ## @@ -426,168 +401,202 @@ public Map getExceptions() { return

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13664: URL: https://github.com/apache/pinot/pull/13664#discussion_r1684448232 ## pinot-query-planner/src/test/java/org/apache/pinot/query/QueryCompilationTest.java: ## @@ -385,6 +385,12 @@ public void testGetTableNamesForQuery() {

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13664: URL: https://github.com/apache/pinot/pull/13664#discussion_r1684447814 ## pinot-query-planner/src/main/java/org/apache/pinot/query/validate/Validator.java: ## @@ -147,4 +151,20 @@ protected void addToSelectList(List list, Set aliases,

Re: [PR] Fix colocated join when no mapping project is used [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13666: URL: https://github.com/apache/pinot/pull/13666#discussion_r1684442201 ## pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotRelDistributionTraitRule.java: ## @@ -110,10 +117,22 @@ private static RelDistribution

Re: [PR] Fix colocated join when no mapping project is used [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13666: URL: https://github.com/apache/pinot/pull/13666#discussion_r1684440903 ## pinot-query-planner/src/test/resources/queries/ExplainPhysicalPlans.json: ## @@ -449,6 +449,58 @@ "└──

Re: [PR] Fix colocated join when no mapping project is used [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13666: URL: https://github.com/apache/pinot/pull/13666#discussion_r1684440273 ## pinot-query-planner/src/test/resources/queries/ExplainPhysicalPlans.json: ## @@ -449,6 +449,58 @@ "└──

[PR] Fix colocated join when no mapping project is used [pinot]

2024-07-19 Thread via GitHub
gortiz opened a new pull request, #13666: URL: https://github.com/apache/pinot/pull/13666 This PR fixes an issue related to colocated join. For example, in ColocatedJoinEngineQuickStart, the following query should be executed in colocated fashion given: 1. We are specifying the

Re: [PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
codecov-commenter commented on PR #13664: URL: https://github.com/apache/pinot/pull/13664#issuecomment-2239049441 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13664?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report

[PR] Bump software.amazon.awssdk:bom from 2.26.20 to 2.26.21 [pinot]

2024-07-19 Thread via GitHub
dependabot[bot] opened a new pull request, #13665: URL: https://github.com/apache/pinot/pull/13665 Bumps software.amazon.awssdk:bom from 2.26.20 to 2.26.21. [![Dependabot compatibility

(pinot) branch dependabot/maven/software.amazon.awssdk-bom-2.26.21 created (now 83010cc236)

2024-07-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/software.amazon.awssdk-bom-2.26.21 in repository https://gitbox.apache.org/repos/asf/pinot.git at 83010cc236 Bump software.amazon.awssdk:bom from 2.26.20 to 2.26.21

[PR] Disallow duplicate CTE aliases [pinot]

2024-07-19 Thread via GitHub
yashmayya opened a new pull request, #13664: URL: https://github.com/apache/pinot/pull/13664 - Currently, a query like `WITH tmp AS (SELECT * FROM a LIMIT 1), tmp AS (SELECT * FROM a LIMIT 2) SELECT * FROM tmp` doesn't return an error and runs fine. - By default, Calcite just uses the

Re: [PR] Add Debug APIs that return Thread and Query Resource Usage [pinot]

2024-07-19 Thread via GitHub
vrajat commented on PR #13583: URL: https://github.com/apache/pinot/pull/13583#issuecomment-2238939678 The test failure is not related to these changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(pinot) branch dependabot/maven/com.azure-azure-storage-file-datalake-12.20.0 created (now b37dddd806)

2024-07-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.azure-azure-storage-file-datalake-12.20.0 in repository https://gitbox.apache.org/repos/asf/pinot.git at b37806 Bump com.azure:azure-storage-file-datalake

[PR] Bump com.azure:azure-storage-file-datalake from 12.19.1 to 12.20.0 [pinot]

2024-07-19 Thread via GitHub
dependabot[bot] opened a new pull request, #13663: URL: https://github.com/apache/pinot/pull/13663 Bumps [com.azure:azure-storage-file-datalake](https://github.com/Azure/azure-sdk-for-java) from 12.19.1 to 12.20.0. Release notes Sourced from

[PR] Bump com.google.errorprone:error_prone_annotations from 2.29.1 to 2.29.2 [pinot]

2024-07-19 Thread via GitHub
dependabot[bot] opened a new pull request, #13662: URL: https://github.com/apache/pinot/pull/13662 Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.29.1 to 2.29.2. Release notes Sourced from

(pinot) branch dependabot/maven/com.google.errorprone-error_prone_annotations-2.29.2 created (now 412ac55507)

2024-07-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/com.google.errorprone-error_prone_annotations-2.29.2 in repository https://gitbox.apache.org/repos/asf/pinot.git at 412ac55507 Bump

Re: [PR] Disable Implicit Cast of Varchar to Int in Multi-Stage engine [pinot]

2024-07-19 Thread via GitHub
vrajat commented on code in PR #13650: URL: https://github.com/apache/pinot/pull/13650#discussion_r1684139728 ## pinot-query-planner/src/test/java/org/apache/pinot/query/validate/PinotTypeCoercionImplTest.java: ## @@ -0,0 +1,184 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add Debug APIs that return Thread and Query Resource Usage [pinot]

2024-07-19 Thread via GitHub
vrajat commented on code in PR #13583: URL: https://github.com/apache/pinot/pull/13583#discussion_r1684125982 ## pinot-tools/src/main/java/org/apache/pinot/tools/utils/PinotConfigUtils.java: ## @@ -215,6 +216,27 @@ public static Map generateMinionConf(String clusterName,

Re: [PR] Add Debug APIs that return Thread and Query Resource Usage [pinot]

2024-07-19 Thread via GitHub
vrajat commented on code in PR #13583: URL: https://github.com/apache/pinot/pull/13583#discussion_r1684121727 ## pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotBrokerDebug.java: ## @@ -268,4 +273,24 @@ public String getServerRoutingStats() { private

Re: [PR] [Draft] TLS config support for QueryServer and Dispatch client [pinot]

2024-07-19 Thread via GitHub
mgranderath commented on code in PR #13645: URL: https://github.com/apache/pinot/pull/13645#discussion_r1683940105 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/service/server/QueryServer.java: ## @@ -184,4 +211,27 @@ public void cancel(Worker.CancelRequest

Re: [PR] Improve performance of DataBlock serde [pinot]

2024-07-19 Thread via GitHub
Jackie-Jiang commented on code in PR #13303: URL: https://github.com/apache/pinot/pull/13303#discussion_r1683828565 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/BaseDataBlock.java: ## @@ -109,92 +114,27 @@ public BaseDataBlock(int numRows, @Nullable

(pinot) branch master updated: API to expose gauge metric values on minion (#13550)

2024-07-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 205a75debb API to expose gauge metric values on

Re: [PR] API to expose gauge metric values on minion [pinot]

2024-07-19 Thread via GitHub
Jackie-Jiang merged PR #13550: URL: https://github.com/apache/pinot/pull/13550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [Draft] TLS config support for QueryServer and Dispatch client [pinot]

2024-07-19 Thread via GitHub
mgranderath commented on code in PR #13645: URL: https://github.com/apache/pinot/pull/13645#discussion_r1683938498 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/service/server/QueryServer.java: ## @@ -184,4 +211,27 @@ public void cancel(Worker.CancelRequest

Re: [PR] Automatically detect whether a v1 query could have run on the v2 query engine [pinot]

2024-07-19 Thread via GitHub
gortiz commented on code in PR #13628: URL: https://github.com/apache/pinot/pull/13628#discussion_r1683870200 ## pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java: ## @@ -478,8 +481,17 @@ protected BrokerResponse