[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017640531 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java: ## @@ -112,20 +122,50 @@ public void updateIndi

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017643257 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java: ## @@ -147,11 +187,48 @@ Map computeOperation(

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017644840 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java: ## @@ -147,11 +187,48 @@ Map computeOperation(

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017673936 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java: ## @@ -147,11 +187,48 @@ Map computeOperation(

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017676767 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/IndexHandler.java: ## @@ -39,4 +39,12 @@ void updateIndices(SegmentDirect

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017678133 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/IndexLoadingConfig.java: ## @@ -512,7 +512,7 @@ public void setOnHeapDict

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017682610 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/IndexHandler.java: ## @@ -39,4 +39,12 @@ void updateIndices(SegmentDirect

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017685585 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java: ## @@ -226,17 +226,19 @@ Map

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017685585 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/defaultcolumn/BaseDefaultColumnHandler.java: ## @@ -226,17 +226,19 @@ Map

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017727040 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java: ## @@ -1903,19 +1908,76 @@ public void tes

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017730180 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java: ## @@ -1903,19 +1908,76 @@ public void tes

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017731282 ## pinot-core/src/test/java/org/apache/pinot/queries/ForwardIndexDisabledMultiValueQueriesTestWithReload.java: ## @@ -0,0 +1,525 @@ +/** Review Comment: +1

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

siddharthteotia commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1017736639 ## pinot-core/src/test/java/org/apache/pinot/queries/ForwardIndexDisabledSingleValueQueriesTest.java: ## @@ -700,6 +711,142 @@ public void testSelectQueries() {

[GitHub] [pinot] somandal commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

somandal commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1018015250 ## pinot-core/src/test/java/org/apache/pinot/queries/ForwardIndexDisabledMultiValueQueriesTestWithReload.java: ## @@ -0,0 +1,525 @@ +/** Review Comment: Yes, there a

[GitHub] [pinot] walterddr commented on pull request #9755: [multistage] leaf-node return data in non-serialized format

walterddr commented on PR #9755: URL: https://github.com/apache/pinot/pull/9755#issuecomment-1309003496 FYI @ankitsultana @agavra @siddharthteotia -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1017353407 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/queries/BasicQueryTest.java: ## @@ -0,0 +1,50 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] [pinot] agavra commented on a diff in pull request #9767: [multistage] clean up AggregateOperator and add test coverage

agavra commented on code in PR #9767: URL: https://github.com/apache/pinot/pull/9767#discussion_r1018222967 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/AggregateOperatorTest.java: ## @@ -51,4 +233,27 @@ public void testGroupByAggregateWithHashCol

[GitHub] [pinot] walterddr commented on a diff in pull request #9767: [multistage] clean up AggregateOperator and add test coverage

walterddr commented on code in PR #9767: URL: https://github.com/apache/pinot/pull/9767#discussion_r1018240339 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java: ## @@ -53,17 +56,32 @@ * If the input is single value, the output

[GitHub] [pinot] agavra commented on a diff in pull request #9767: [multistage] clean up AggregateOperator and add test coverage

agavra commented on code in PR #9767: URL: https://github.com/apache/pinot/pull/9767#discussion_r1018254221 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java: ## @@ -53,17 +56,32 @@ * If the input is single value, the output ty

[GitHub] [pinot] agavra commented on a diff in pull request #9767: [multistage] clean up AggregateOperator and add test coverage

agavra commented on code in PR #9767: URL: https://github.com/apache/pinot/pull/9767#discussion_r1018254481 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java: ## @@ -53,17 +56,32 @@ * If the input is single value, the output ty

[GitHub] [pinot] agavra commented on a diff in pull request #9767: [multistage] clean up AggregateOperator and add test coverage

agavra commented on code in PR #9767: URL: https://github.com/apache/pinot/pull/9767#discussion_r1018255056 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/AggregateOperatorTest.java: ## @@ -18,26 +18,208 @@ */ package org.apache.pinot.query.runti

[GitHub] [pinot] walterddr merged pull request #9767: [multistage] clean up AggregateOperator and add test coverage

walterddr merged PR #9767: URL: https://github.com/apache/pinot/pull/9767 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apac

[pinot] branch master updated (715df15ffa -> 8c9af20c3f)

This is an automated email from the ASF dual-hosted git repository. rongr pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 715df15ffa Add query tests for changeCompression and enableDictionary changes (#9768) add 8c9af20c3f [multistage]

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

vvivekiyer commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1018265444 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java: ## @@ -1903,19 +1908,76 @@ public void testForw

[GitHub] [pinot] agavra opened a new pull request, #9769: [multistage] add test coverage for SortOperator

agavra opened a new pull request, #9769: URL: https://github.com/apache/pinot/pull/9769 follow up to #9767 but for `SortOperator` (also addresses some leftover comments on `AggregateOperator`) -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

siddharthteotia commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018303289 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(T

[GitHub] [pinot] nizarhejazi opened a new issue, #9770: Sorted index on a non-dictionary-encoded column results in full segment scan

nizarhejazi opened a new issue, #9770: URL: https://github.com/apache/pinot/issues/9770 Defining a `sorted index` on a `non-dictionary-encoded` column causes Pinot to fully scan the content of segments (no inverted index is utilized). `user` table config: ``` "noDictionary

[GitHub] [pinot] navina opened a new issue, #9771: Partial upsert doesn't support new columns in the schema

navina opened a new issue, #9771: URL: https://github.com/apache/pinot/issues/9771 We noticed that when a partial upsert enabled table is reloaded with a new column in the schema, the upsert behavior is not honored for the new column. This can be reproduced deterministically and the likely

[GitHub] [pinot] klsince commented on issue #9770: Sorted index on a non-dictionary-encoded column results in full segment scan

klsince commented on issue #9770: URL: https://github.com/apache/pinot/issues/9770#issuecomment-1309223054 Thanks for the report. The planning logic for the IN clause in the query would hit this [if-check](https://github.com/apache/pinot/blob/master/pinot-core/src/main/java/org/apache/pinot

[GitHub] [pinot] mayankshriv commented on issue #9770: Sorted index on a non-dictionary-encoded column results in full segment scan

mayankshriv commented on issue #9770: URL: https://github.com/apache/pinot/issues/9770#issuecomment-1309226060 In case of realtime ingestion, the following happens: - The column does get stored wtih no-dict (or raw values). - The column is also sorted on raw values, and that is reflecte

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

siddharthteotia commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018313618 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(T

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

siddharthteotia commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018314119 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(T

[GitHub] [pinot] codecov-commenter commented on pull request #9769: [multistage] add test coverage for SortOperator

codecov-commenter commented on PR #9769: URL: https://github.com/apache/pinot/pull/9769#issuecomment-1309242969 # [Codecov](https://codecov.io/gh/apache/pinot/pull/9769?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] klsince commented on issue #9770: Sorted index on a non-dictionary-encoded column results in full segment scan

klsince commented on issue #9770: URL: https://github.com/apache/pinot/issues/9770#issuecomment-1309257784 FYI, currently, column encoding can't be changed (like from raw to dict encoding and vice versa) by reloading segments with the reload or reset swagger APIs. There is [an ongoi

[GitHub] [pinot] klsince opened a new pull request, #9772: Reprocess immutable segments from REALTIME table conditionally when loading them

klsince opened a new pull request, #9772: URL: https://github.com/apache/pinot/pull/9772 The immutable segments from OFFLINE table are reprocessed only when they are not consistent with the TableConfig or schema any more, like need to update index types, thus avoiding redundant processing.

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018452600 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/queries/QueryTestExecutor.java: ## @@ -0,0 +1,171 @@ +/** + * Licensed to the Apache Software Founda

[GitHub] [pinot] walterddr commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

walterddr commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018462041 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(Transfe

[GitHub] [pinot] walterddr commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

walterddr commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018462505 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(Transfe

[GitHub] [pinot] codecov-commenter commented on pull request #9772: Preprocess immutable segments from REALTIME table conditionally when loading them

codecov-commenter commented on PR #9772: URL: https://github.com/apache/pinot/pull/9772#issuecomment-1309450813 # [Codecov](https://codecov.io/gh/apache/pinot/pull/9772?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] walterddr commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

walterddr commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018467674 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MailboxSendOperator.java: ## @@ -67,12 +67,13 @@ public class MailboxSendOperator extends

[GitHub] [pinot] walterddr commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

walterddr commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018468629 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(Transfe

[GitHub] [pinot] npawar merged pull request #9754: bubble up error messages from broker

npawar merged PR #9754: URL: https://github.com/apache/pinot/pull/9754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.

[pinot] branch master updated (8c9af20c3f -> 3252a62062)

This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 8c9af20c3f [multistage] clean up AggregateOperator and add test coverage (#9767) add 3252a62062 bubble up erro

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

siddharthteotia commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018480017 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(T

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

siddharthteotia commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018481538 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java: ## @@ -59,37 +60,67 @@ public static boolean isNoOpBlock(T

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9755: [multistage] leaf-node return data in non-serialized format

siddharthteotia commented on code in PR #9755: URL: https://github.com/apache/pinot/pull/9755#discussion_r1018481916 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MailboxSendOperator.java: ## @@ -67,12 +67,13 @@ public class MailboxSendOperator ext

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9769: [multistage] add test coverage for SortOperator

siddharthteotia commented on code in PR #9769: URL: https://github.com/apache/pinot/pull/9769#discussion_r1018484429 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/SortOperatorTest.java: ## @@ -0,0 +1,363 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9769: [multistage] add test coverage for SortOperator

siddharthteotia commented on code in PR #9769: URL: https://github.com/apache/pinot/pull/9769#discussion_r1018484797 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/SortOperatorTest.java: ## @@ -0,0 +1,363 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] agavra commented on a diff in pull request #9765: [multistage][test] restructure test compose

agavra commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018490355 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java: ## @@ -198,11 +242,27 @@ private static List toH2FieldNamesAndTypes(org.apac

[GitHub] [pinot] klsince commented on a diff in pull request #9309: Introduce Segment AssignmentStrategy Interface

klsince commented on code in PR #9309: URL: https://github.com/apache/pinot/pull/9309#discussion_r1018496195 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java: ## @@ -182,9 +185,21 @@ public Map> rebalance

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018496501 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/queries/QueryTestExecutor.java: ## @@ -0,0 +1,171 @@ +/** + * Licensed to the Apache Software Founda

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018496960 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java: ## @@ -70,6 +60,128 @@ public class QueryRunnerTestBase extends QueryTestS

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018497568 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/queries/QueryTestExecutor.java: ## @@ -0,0 +1,171 @@ +/** + * Licensed to the Apache Software Founda

[GitHub] [pinot] klsince commented on a diff in pull request #9309: Introduce Segment AssignmentStrategy Interface

klsince commented on code in PR #9309: URL: https://github.com/apache/pinot/pull/9309#discussion_r1018496195 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java: ## @@ -182,9 +185,21 @@ public Map> rebalance

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018498206 ## pinot-query-runtime/src/test/resources/queries/BasicQuery.json: ## @@ -0,0 +1,39 @@ + +{ + "basic_test": { +"description": "basic test case example", +"sql"

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9733: show table metadata info in aggregate index size form

siddharthteotia commented on code in PR #9733: URL: https://github.com/apache/pinot/pull/9733#discussion_r1018498327 ## pinot-server/src/test/java/org/apache/pinot/server/api/TablesResourceTest.java: ## @@ -99,6 +101,38 @@ public void getSegments() Assert.assertEquals(respo

[GitHub] [pinot] klsince commented on a diff in pull request #9309: Introduce Segment AssignmentStrategy Interface

klsince commented on code in PR #9309: URL: https://github.com/apache/pinot/pull/9309#discussion_r1018496195 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java: ## @@ -182,9 +185,21 @@ public Map> rebalance

[GitHub] [pinot] siddharthteotia merged pull request #9759: fix flaky QueryLoggerTest

siddharthteotia merged PR #9759: URL: https://github.com/apache/pinot/pull/9759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pino

[pinot] branch master updated: fix flaky QueryLoggerTest (#9759)

This is an automated email from the ASF dual-hosted git repository. siddteotia pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2c38d292ea fix flaky QueryLoggerTest (#9759)

[GitHub] [pinot] walterddr commented on a diff in pull request #9733: show table metadata info in aggregate index size form

walterddr commented on code in PR #9733: URL: https://github.com/apache/pinot/pull/9733#discussion_r1018506209 ## pinot-server/src/test/java/org/apache/pinot/server/api/TablesResourceTest.java: ## @@ -99,6 +101,38 @@ public void getSegments() Assert.assertEquals(response.ge

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018509384 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadResourceUsageProvider.java: ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018510317 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadResourceUsageProvider.java: ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018510552 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadResourceUsageProvider.java: ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018514254 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadResourceUsageProvider.java: ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] agavra commented on a diff in pull request #9769: [multistage] add test coverage for SortOperator

agavra commented on code in PR #9769: URL: https://github.com/apache/pinot/pull/9769#discussion_r1018514840 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/SortOperatorTest.java: ## @@ -0,0 +1,363 @@ +/** + * Licensed to the Apache Software Foundatio

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018518515 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java: ## @@ -198,11 +242,27 @@ private static List toH2FieldNamesAndTypes(org.a

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018518615 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java: ## @@ -70,6 +60,128 @@ public class QueryRunnerTestBase extends QueryTestS

[GitHub] [pinot] walterddr commented on a diff in pull request #9765: [multistage][test] restructure test compose

walterddr commented on code in PR #9765: URL: https://github.com/apache/pinot/pull/9765#discussion_r1018518804 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/queries/QueryTestExecutor.java: ## @@ -0,0 +1,171 @@ +/** + * Licensed to the Apache Software Founda

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018521765 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadResourceUsageProvider.java: ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018522262 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadResourceUsageProvider.java: ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018524903 ## pinot-core/src/main/java/org/apache/pinot/core/accounting/CPUMemThreadLevelAccountingObjects.java: ## @@ -0,0 +1,115 @@ +/** + * Licensed to the Apache Softwar

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018525204 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ExecutionContext.java: ## @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018525464 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ExecutionContext.java: ## @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018526113 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ExecutionContext.java: ## @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] vvivekiyer commented on a diff in pull request #9311: Adaptive Server Selection

vvivekiyer commented on code in PR #9311: URL: https://github.com/apache/pinot/pull/9311#discussion_r1018526457 ## pinot-common/src/main/java/org/apache/pinot/common/utils/ExponentialMovingAverage.java: ## @@ -0,0 +1,105 @@ +/** + * Licensed to the Apache Software Foundation (AS

[GitHub] [pinot] 61yao commented on issue #9728: [multi-stage] [bug] Inequi join produce wrong result when there is a mix of equi join and inequijoin

61yao commented on issue #9728: URL: https://github.com/apache/pinot/issues/9728#issuecomment-1309595157 @walterddr SELECT * from a WHERE a.col3 != 1 and a.col1 = b.col1. This is the query how it will fail. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

jasperjiaguo commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018544097 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ExecutionContext.java: ## @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [pinot] jasperjiaguo commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

jasperjiaguo commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018545020 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ExecutionContext.java: ## @@ -0,0 +1,34 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [pinot] agavra opened a new pull request, #9773: [multistage] refactor MailboxSendOperator for testability and tests

agavra opened a new pull request, #9773: URL: https://github.com/apache/pinot/pull/9773 This PR does two things: refactors `MailboxSendOperator` into testable components, and then tests each of those components. The refactor essentially factors out each of the exchange logic (hash, s

[GitHub] [pinot] 61yao commented on issue #9728: [multi-stage] [optimization] Move inequi join out to hashjoin

61yao commented on issue #9728: URL: https://github.com/apache/pinot/issues/9728#issuecomment-1309610092 @walterddr Sorry for the confusion. It is actually working because empty key gives the same hash, which is 1. Let's move this out of hashJoin in the future. -- This is an automated me

[GitHub] [pinot] somandal commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

somandal commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1018550060 ## pinot-core/src/test/java/org/apache/pinot/queries/ForwardIndexDisabledMultiValueQueriesTestWithReload.java: ## @@ -0,0 +1,525 @@ +/** + * Licensed to the Apache Softw

[GitHub] [pinot] somandal commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

somandal commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1018550943 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java: ## @@ -112,20 +122,50 @@ public void updateIndices(Seg

[GitHub] [pinot] somandal commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

somandal commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1018555380 ## pinot-core/src/test/java/org/apache/pinot/queries/ForwardIndexDisabledMultiValueQueriesTestWithReload.java: ## @@ -0,0 +1,525 @@ +/** + * Licensed to the Apache Softw

[GitHub] [pinot] codecov-commenter commented on pull request #9773: [multistage] refactor MailboxSendOperator for testability and tests

codecov-commenter commented on PR #9773: URL: https://github.com/apache/pinot/pull/9773#issuecomment-1309639663 # [Codecov](https://codecov.io/gh/apache/pinot/pull/9773?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Softwar

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018564803 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountant.java: ## @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018564803 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountant.java: ## @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018567752 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountant.java: ## @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018568445 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountant.java: ## @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018569126 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountant.java: ## @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] GSharayu commented on a diff in pull request #9309: Introduce Segment AssignmentStrategy Interface

GSharayu commented on code in PR #9309: URL: https://github.com/apache/pinot/pull/9309#discussion_r1018573095 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java: ## @@ -182,9 +185,21 @@ public Map> rebalanc

[GitHub] [pinot] GSharayu opened a new pull request, #9774: Fix the null pointer exception for segment assignment strategy in reassigning tiers (Table rebalance)

GSharayu opened a new pull request, #9774: URL: https://github.com/apache/pinot/pull/9774 The segment assignment strategy was not getting set during rebalance tiers. As mentioned in comment: https://github.com/apache/pinot/pull/9309#discussion_r1018496195 This code handles the nul

[GitHub] [pinot] GSharayu commented on a diff in pull request #9309: Introduce Segment AssignmentStrategy Interface

GSharayu commented on code in PR #9309: URL: https://github.com/apache/pinot/pull/9309#discussion_r1018579113 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java: ## @@ -182,9 +185,21 @@ public Map> rebalanc

[GitHub] [pinot] GSharayu commented on pull request #9774: Fix the null pointer exception for segment assignment strategy in reassigning tiers (Table rebalance)

GSharayu commented on PR #9774: URL: https://github.com/apache/pinot/pull/9774#issuecomment-1309663347 @snleee Please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018580004 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountant.java: ## @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018581154 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountantFactory.java: ## @@ -0,0 +1,26 @@ +/** + * Licensed to the Apache Software Foundation (

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018581723 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountantFactory.java: ## @@ -0,0 +1,26 @@ +/** + * Licensed to the Apache Software Foundation (

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018567752 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountant.java: ## @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018509384 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadResourceUsageProvider.java: ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #9727: Thread Level Usage Accounting and Query Killing on Server

siddharthteotia commented on code in PR #9727: URL: https://github.com/apache/pinot/pull/9727#discussion_r1018581723 ## pinot-spi/src/main/java/org/apache/pinot/spi/accounting/ThreadAccountantFactory.java: ## @@ -0,0 +1,26 @@ +/** + * Licensed to the Apache Software Foundation (

[GitHub] [pinot] somandal commented on a diff in pull request #9740: Add support to disable the forward index for existing columns

somandal commented on code in PR #9740: URL: https://github.com/apache/pinot/pull/9740#discussion_r1018586814 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandler.java: ## @@ -147,11 +187,48 @@ Map computeOperation(Segment

  1   2   >