[GitHub] [pinot] pjpringle opened a new issue, #11631: Documentation for argmax/argmin is inconsistent with actual function name exprmax/exprmin

2023-09-19 Thread via GitHub
pjpringle opened a new issue, #11631: URL: https://github.com/apache/pinot/issues/11631 Update documentation for argmax/min to use exprmax/min https://docs.pinot.apache.org/configuration-reference/functions/arg_min-arg_max -- This is an automated message from the Apache Git

[GitHub] [pinot] jadami10 commented on pull request #11608: Updated the response for reload API to be pretty printed

2023-09-19 Thread via GitHub
jadami10 commented on PR #11608: URL: https://github.com/apache/pinot/pull/11608#issuecomment-1726892496 looks like there's still some tests failing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [pinot] codecov-commenter commented on pull request #11630: Optimize group-by and join for single key scenario

2023-09-19 Thread via GitHub
codecov-commenter commented on PR #11630: URL: https://github.com/apache/pinot/pull/11630#issuecomment-1726824983 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11630?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report > Merging

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11624: modify the error response on controller.

2023-09-19 Thread via GitHub
Jackie-Jiang commented on code in PR #11624: URL: https://github.com/apache/pinot/pull/11624#discussion_r1330903391 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java: ## @@ -465,4 +471,12 @@ public String sendRequestRaw(String

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #11630: Optimize group-by and join for single key scenario

2023-09-19 Thread via GitHub
Jackie-Jiang opened a new pull request, #11630: URL: https://github.com/apache/pinot/pull/11630 - For single join key and group key scenario, no need to wrap the values into the `Key` object. Directly use the value as the key in the map. - Optimize the case of no join key - In

[GitHub] [pinot] klsince commented on a diff in pull request #11628: assign segments for upsert table with respect to ideal state

2023-09-19 Thread via GitHub
klsince commented on code in PR #11628: URL: https://github.com/apache/pinot/pull/11628#discussion_r1330857514 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/StrictRealtimeSegmentAssignment.java: ## @@ -0,0 +1,108 @@ +/** + *

[GitHub] [pinot] klsince commented on a diff in pull request #11628: assign segments for upsert table with respect to ideal state

2023-09-19 Thread via GitHub
klsince commented on code in PR #11628: URL: https://github.com/apache/pinot/pull/11628#discussion_r1330857397 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/StrictRealtimeSegmentAssignment.java: ## @@ -0,0 +1,108 @@ +/** + *

[GitHub] [pinot] walterddr commented on a diff in pull request #11437: Add broker query event listener

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1330856040 ## pinot-spi/src/main/java/org/apache/pinot/spi/eventlistener/query/PinotBrokerQueryEventListenerUtils.java: ## @@ -0,0 +1,106 @@ +/** + * Licensed to the Apache

[GitHub] [pinot] walterddr commented on a diff in pull request #11437: Add broker query event listener

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1330855743 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/RequestContext.java: ## @@ -19,6 +19,7 @@ package org.apache.pinot.spi.trace; public interface

[GitHub] [pinot] pjpringle commented on issue #11622: datatables.net vulnerability in Pinot 1.0.0

2023-09-19 Thread via GitHub
pjpringle commented on issue #11622: URL: https://github.com/apache/pinot/issues/11622#issuecomment-1726747804 This is the path to the datatables usage: jquery.dataTables.min.js located at

[GitHub] [pinot] klsince commented on a diff in pull request #11628: assign segments for upsert table with respect to ideal state

2023-09-19 Thread via GitHub
klsince commented on code in PR #11628: URL: https://github.com/apache/pinot/pull/11628#discussion_r1330852853 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/StrictRealtimeSegmentAssignment.java: ## @@ -0,0 +1,108 @@ +/** + *

[GitHub] [pinot] klsince commented on a diff in pull request #11628: assign segments for upsert table with respect to ideal state

2023-09-19 Thread via GitHub
klsince commented on code in PR #11628: URL: https://github.com/apache/pinot/pull/11628#discussion_r1330852721 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/StrictRealtimeSegmentAssignment.java: ## @@ -0,0 +1,108 @@ +/** + *

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11628: assign segments for upsert table with respect to ideal state

2023-09-19 Thread via GitHub
Jackie-Jiang commented on code in PR #11628: URL: https://github.com/apache/pinot/pull/11628#discussion_r1330837146 ## pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/StrictRealtimeSegmentAssignment.java: ## @@ -0,0 +1,108 @@ +/** + *

[GitHub] [pinot] somandal commented on issue #11627: [multistage][discuss] window agg with same partition but with/without order by

2023-09-19 Thread via GitHub
somandal commented on issue #11627: URL: https://github.com/apache/pinot/issues/11627#issuecomment-1726725403 Cool. Would it be fair to check the node’s child and if it’s a window function with same partitioning (and ordering if present in both) we skip adding the exchange? We still need

[GitHub] [pinot] jackjlli commented on a diff in pull request #11346: Add hyperLogLogPlus aggregation function for distinct count

2023-09-19 Thread via GitHub
jackjlli commented on code in PR #11346: URL: https://github.com/apache/pinot/pull/11346#discussion_r1330835073 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountHLLPlusAggregationFunction.java: ## @@ -0,0 +1,469 @@ +/** + * Licensed to

[GitHub] [pinot] Jackie-Jiang commented on pull request #11564: Use builder method for initalizing immutable map in integration test

2023-09-19 Thread via GitHub
Jackie-Jiang commented on PR #11564: URL: https://github.com/apache/pinot/pull/11564#issuecomment-1726721556 @ege-st Usually that 14% is caused by codecov posting report prematurely before receiving all coverage files. We should probably see if there is some config we can tune to fix this

[GitHub] [pinot] walterddr commented on issue #11627: [multistage][discuss] window agg with same partition but with/without order by

2023-09-19 Thread via GitHub
walterddr commented on issue #11627: URL: https://github.com/apache/pinot/issues/11627#issuecomment-1726720776 for now I am ok with (3) only b/c that's the performance boost part. syntatic suger part (2) can be debated --> if we have (1) we dont really need (2) -- This is an automated

[GitHub] [pinot] Jackie-Jiang commented on pull request #11496: Fix direct memory OOM on broker

2023-09-19 Thread via GitHub
Jackie-Jiang commented on PR #11496: URL: https://github.com/apache/pinot/pull/11496#issuecomment-1726717095 Wondering why other connections take up to 10 minutes to recover? Is that caused by more and more connections getting OOM? Ideally we should only close the connections that OOM'ed,

[GitHub] [pinot] kirkrodrigues commented on a diff in pull request #11604: JSON index: Add support for ignoring values longer than a given length.

2023-09-19 Thread via GitHub
kirkrodrigues commented on code in PR #11604: URL: https://github.com/apache/pinot/pull/11604#discussion_r1330829414 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/json/MutableJsonIndexImpl.java: ## @@ -106,8 +106,11 @@ private void

[GitHub] [pinot] walterddr commented on a diff in pull request #11511: Add auto generated query tests for tpch

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11511: URL: https://github.com/apache/pinot/pull/11511#discussion_r1330826840 ## pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/tpch/generator/TPCHQueryGeneratorV2.java: ## @@ -0,0 +1,553 @@ +/** + * Licensed to the

[GitHub] [pinot] Jackie-Jiang commented on issue #11613: Allow a default replication to be set at the Tenant level, instead of requiring it to be set at the table level

2023-09-19 Thread via GitHub
Jackie-Jiang commented on issue #11613: URL: https://github.com/apache/pinot/issues/11613#issuecomment-1726711435 This won't be easy because currently tenant is just a logical concept, and there is no physical representation stored in ZK. Tenant is used only for resource isolation, and

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11437: Add broker query event listener

2023-09-19 Thread via GitHub
Jackie-Jiang commented on code in PR #11437: URL: https://github.com/apache/pinot/pull/11437#discussion_r1330823054 ## pinot-spi/src/main/java/org/apache/pinot/spi/trace/RequestContext.java: ## @@ -19,6 +19,7 @@ package org.apache.pinot.spi.trace; public interface

[GitHub] [pinot] jadami10 commented on issue #11629: Columns with CAST in the select statement are return as OBJECT data types in the resultSet

2023-09-19 Thread via GitHub
jadami10 commented on issue #11629: URL: https://github.com/apache/pinot/issues/11629#issuecomment-1726706755 yes sorry, the actual query is more like `SELECT CAST(SUM(long_column) AS LONG)`. What is the limitation here? The type result of the cast should be unaffected by the aggregation,

[GitHub] [pinot] Jackie-Jiang commented on issue #11629: Columns with CAST in the select statement are return as OBJECT data types in the resultSet

2023-09-19 Thread via GitHub
Jackie-Jiang commented on issue #11629: URL: https://github.com/apache/pinot/issues/11629#issuecomment-1726704392 It this the complete query, or it is on top of an aggregation? Currently we cannot tell the result type of `CAST` when it is an post aggregation, but the result type should

[GitHub] [pinot] somandal commented on issue #11627: [multistage][discuss] window agg with same partition but with/without order by

2023-09-19 Thread via GitHub
somandal commented on issue #11627: URL: https://github.com/apache/pinot/issues/11627#issuecomment-1726701960 Yes breaking it into (2) + (3) and optimizing out the second exchange should be possible. The planner will need to be modified to a) rewrite the query and b) identify the exchange

[pinot] branch master updated: Add more array scalar functions (#11555)

2023-09-19 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 47eff886cd Add more array scalar functions

[GitHub] [pinot] xiangfu0 merged pull request #11555: Add more array scalar functions

2023-09-19 Thread via GitHub
xiangfu0 merged PR #11555: URL: https://github.com/apache/pinot/pull/11555 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot] xiangfu0 closed pull request #11330: [multistage] Fixing pinot client resolve table name

2023-09-19 Thread via GitHub
xiangfu0 closed pull request #11330: [multistage] Fixing pinot client resolve table name URL: https://github.com/apache/pinot/pull/11330 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [pinot] xiangfu0 commented on pull request #11330: [multistage] Fixing pinot client resolve table name

2023-09-19 Thread via GitHub
xiangfu0 commented on PR #11330: URL: https://github.com/apache/pinot/pull/11330#issuecomment-1726696240 Close this as suggested by @walterddr to delegate the job to controller. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [pinot] mcvsubbu commented on pull request #11591: Throw exception when schema name doesn't match table name during table creation

2023-09-19 Thread via GitHub
mcvsubbu commented on PR #11591: URL: https://github.com/apache/pinot/pull/11591#issuecomment-1726696135 I have a more general question on this PR. Should this PR not include places in pinot-spi where we can throw exception if a client tries to create a table config that does not have a

[GitHub] [pinot] jackjlli commented on a diff in pull request #11346: Add hyperLogLogPlus aggregation function for distinct count

2023-09-19 Thread via GitHub
jackjlli commented on code in PR #11346: URL: https://github.com/apache/pinot/pull/11346#discussion_r1330810915 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountHLLPlusAggregationFunction.java: ## @@ -96,16 +96,12 @@ public void

[pinot] branch master updated: Add FrequentStringsSketch and FrequentLonsSketch aggregation functions (#11098)

2023-09-19 Thread tingchen
This is an automated email from the ASF dual-hosted git repository. tingchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new cf017f1c63 Add FrequentStringsSketch and

[GitHub] [pinot] chenboat merged pull request #11098: Introduce FrequentStringsSketch and FrequentLongsSketch aggregation functions

2023-09-19 Thread via GitHub
chenboat merged PR #11098: URL: https://github.com/apache/pinot/pull/11098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot] jadami10 opened a new issue, #11629: Columns with CAST in the select statement are return as OBJECT data types in the resultSet

2023-09-19 Thread via GitHub
jadami10 opened a new issue, #11629: URL: https://github.com/apache/pinot/issues/11629 We had a query that was doing `select long_column from X`. The user updated it to do `select long_column - Y from X`. This change the `columnDataTypes` for this to a `DOUBLE`. The user's assertions on

[GitHub] [pinot] jadami10 closed issue #11533: Increase in ingestion lag when servers are shutting down

2023-09-19 Thread via GitHub
jadami10 closed issue #11533: Increase in ingestion lag when servers are shutting down URL: https://github.com/apache/pinot/issues/11533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [pinot] codecov-commenter commented on pull request #11628: assign segments for upsert table with respect to ideal state

2023-09-19 Thread via GitHub
codecov-commenter commented on PR #11628: URL: https://github.com/apache/pinot/pull/11628#issuecomment-1726660138 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11628?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report > Merging

[GitHub] [pinot] npawar commented on pull request #11568: Add integration test for rebalance in upsert tables

2023-09-19 Thread via GitHub
npawar commented on PR #11568: URL: https://github.com/apache/pinot/pull/11568#issuecomment-1726634929 > Mostly tests that partitions move as a whole unit to new servers is there an jira that prompted this item? did this not work in some env / scenario? -- This is an automated

[GitHub] [pinot] jugomezv commented on issue #11543: Queries that involve string timestamps do not work properly unless the string is singled quoted

2023-09-19 Thread via GitHub
jugomezv commented on issue #11543: URL: https://github.com/apache/pinot/issues/11543#issuecomment-1726629059 Uff that is a very misleading parsing, specially given the fact that the server knows that this field is a string: shouldn't the schema play a role here? If I am filtering on a

[GitHub] [pinot] xiangfu0 commented on pull request #11591: Throw exception when schema name doesn't match table name during table creation

2023-09-19 Thread via GitHub
xiangfu0 commented on PR #11591: URL: https://github.com/apache/pinot/pull/11591#issuecomment-1726627092 > > > Hi @xiangfu0 @Jackie-Jiang , > > > As @mcvsubbu mentioned, We are working with partner team to resolve the table creation flow since this will error out for new tables. As I

[pinot] branch master updated: Remove table state update operation in GET call (#11621)

2023-09-19 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new 2306298930 Remove table state update operation

[GitHub] [pinot] xiangfu0 merged pull request #11621: Remove table state update operation in GET call

2023-09-19 Thread via GitHub
xiangfu0 merged PR #11621: URL: https://github.com/apache/pinot/pull/11621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot] jugomezv commented on pull request #11534: do not emit lag once tabledatamanager shutdown

2023-09-19 Thread via GitHub
jugomezv commented on PR #11534: URL: https://github.com/apache/pinot/pull/11534#issuecomment-1726625682 Thanks for the change @jadami10 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [pinot] klsince opened a new pull request, #11628: assign segments for upsert table with respect to ideal state

2023-09-19 Thread via GitHub
klsince opened a new pull request, #11628: URL: https://github.com/apache/pinot/pull/11628 This PR adds more checks when assigning new segments for upsert table, in order to process queries correctly. The extra checks make sure that at any time the idealState has all segments from the same

[pinot] branch master updated (fa51477350 -> 987ebc0d0e)

2023-09-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from fa51477350 fix jackson vulnerability (#11619) add 987ebc0d0e In multi-stage aggregate, directly store column

[GitHub] [pinot] Jackie-Jiang merged pull request #11617: In multi-stage aggregate, directly store column index as identifier

2023-09-19 Thread via GitHub
Jackie-Jiang merged PR #11617: URL: https://github.com/apache/pinot/pull/11617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot] walterddr commented on issue #11627: [multistage][discuss] window agg with same partition but with/without order by

2023-09-19 Thread via GitHub
walterddr commented on issue #11627: URL: https://github.com/apache/pinot/issues/11627#issuecomment-1726580586 yes I thought so as well so i proposed the (2) and (3) optimization and see if that's possible --> simply being that it doesn't introduce additional exchange, which in most of the

[GitHub] [pinot] deemoliu commented on a diff in pull request #11346: Add hyperLogLogPlus aggregation function for distinct count

2023-09-19 Thread via GitHub
deemoliu commented on code in PR #11346: URL: https://github.com/apache/pinot/pull/11346#discussion_r1330687546 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountHLLPlusAggregationFunction.java: ## @@ -0,0 +1,473 @@ +/** + * Licensed to

[GitHub] [pinot-site] xiangfu0 commented on pull request #83: Update to the Features list, refresh of dependencies in package.json

2023-09-19 Thread via GitHub
xiangfu0 commented on PR #83: URL: https://github.com/apache/pinot-site/pull/83#issuecomment-1726415342 > @xiangfu0, do you have an email from [November, 2021 from Algolia/DocSearch](https://docusaurus.io/blog/2021/11/21/algolia-docsearch-migration) with a new appId and apiKey? The build

[GitHub] [pinot] somandal commented on issue #11627: [multistage][discuss] window agg with same partition but with/without order by

2023-09-19 Thread via GitHub
somandal commented on issue #11627: URL: https://github.com/apache/pinot/issues/11627#issuecomment-1726412553 One of the problems here is that the two window functions belong to different window groups. The default for ROW_NUMBER is ROWS type frame and for COUNT it’s RANGE type. I need to

[GitHub] [pinot] walterddr commented on issue #11627: [multistage][discuss] window agg with same partition but with/without order by

2023-09-19 Thread via GitHub
walterddr commented on issue #11627: URL: https://github.com/apache/pinot/issues/11627#issuecomment-1726388911 CC @sonam @snleee -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [pinot] walterddr opened a new issue, #11627: [multistage][discuss] window agg with same partition but with/without order by

2023-09-19 Thread via GitHub
walterddr opened a new issue, #11627: URL: https://github.com/apache/pinot/issues/11627 queries with the same partition but a missing or non-missing order-by clause: ``` SELECT ROW_NUMBER() OVER (PARTITION BY a ORDER BY b) , COUNT(*) OVER (PARTITION BY a) from tbl ```

[GitHub] [pinot] walterddr commented on a diff in pull request #11258: fix fromDateTime malform handling

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11258: URL: https://github.com/apache/pinot/pull/11258#discussion_r1330628461 ## pinot-common/src/main/java/org/apache/pinot/common/function/DateTimePatternHandler.java: ## @@ -35,15 +35,23 @@ private DateTimePatternHandler() { */

[GitHub] [pinot] walterddr commented on a diff in pull request #11258: fix fromDateTime malform handling

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11258: URL: https://github.com/apache/pinot/pull/11258#discussion_r1330627803 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java: ## @@ -491,7 +492,7 @@ public static String[] toDateTimeMV(long[]

[GitHub] [pinot] ankitsultana opened a new issue, #11626: Helix Incorrectly Reports Pending Messages (potentially)

2023-09-19 Thread via GitHub
ankitsultana opened a new issue, #11626: URL: https://github.com/apache/pinot/issues/11626 We use the following MBean to track Helix Pending messages in Pinot Servers: ``` CLMParticipantReport:MonitorType=ParticipantMessageMonitor,ParticipantName=some_instance_id: ``` We

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11617: In multi-stage aggregate, directly store column index as identifier

2023-09-19 Thread via GitHub
Jackie-Jiang commented on code in PR #11617: URL: https://github.com/apache/pinot/pull/11617#discussion_r1330605596 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java: ## @@ -205,90 +196,78 @@ private TransferableBlock

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11258: fix fromDateTime malform handling

2023-09-19 Thread via GitHub
Jackie-Jiang commented on code in PR #11258: URL: https://github.com/apache/pinot/pull/11258#discussion_r1330596741 ## pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java: ## @@ -491,7 +492,7 @@ public static String[] toDateTimeMV(long[]

[GitHub] [pinot] ArnavBalyan commented on pull request #11625: Force FileChannel to commit data to disk

2023-09-19 Thread via GitHub
ArnavBalyan commented on PR #11625: URL: https://github.com/apache/pinot/pull/11625#issuecomment-1726331022 cc @Jackie-Jiang and @ankitsultana if you could please review thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11258: fix fromDateTime malform handling

2023-09-19 Thread via GitHub
Jackie-Jiang commented on code in PR #11258: URL: https://github.com/apache/pinot/pull/11258#discussion_r1330594640 ## pinot-common/src/main/java/org/apache/pinot/common/function/DateTimePatternHandler.java: ## @@ -35,15 +35,23 @@ private DateTimePatternHandler() { */

[GitHub] [pinot] ArnavBalyan opened a new pull request, #11625: Force FileChannel to commit data to disk

2023-09-19 Thread via GitHub
ArnavBalyan opened a new pull request, #11625: URL: https://github.com/apache/pinot/pull/11625 - We have FileChannel in certain places but don't force which can cause data not to be written to disk. - Added force to make sure we don't have data remaining in OS I/O buffers. -

[GitHub] [pinot] codecov-commenter commented on pull request #11624: modify the error response on controller.

2023-09-19 Thread via GitHub
codecov-commenter commented on PR #11624: URL: https://github.com/apache/pinot/pull/11624#issuecomment-1726287225 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11624?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report > Merging

[pinot-site] branch dev updated: embedding vidoes (#86)

2023-09-19 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/pinot-site.git The following commit(s) were added to refs/heads/dev by this push: new 8345cebe embedding vidoes (#86) 8345cebe is

[GitHub] [pinot-site] xiangfu0 merged pull request #86: embedding videos

2023-09-19 Thread via GitHub
xiangfu0 merged PR #86: URL: https://github.com/apache/pinot-site/pull/86 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot-site] hdulay opened a new pull request, #86: embedding videos

2023-09-19 Thread via GitHub
hdulay opened a new pull request, #86: URL: https://github.com/apache/pinot-site/pull/86 Embedding video for Pinot 1.0 announcement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [pinot] walterddr opened a new pull request, #11624: modify the error response on controller.

2023-09-19 Thread via GitHub
walterddr opened a new pull request, #11624: URL: https://github.com/apache/pinot/pull/11624 - return will always be JSON format if process error occurred. - when there's web/network/app issue, strictly non 2xx error code should be there for http response this fixes: #11593 --

[GitHub] [pinot] jackjlli commented on a diff in pull request #11346: Add hyperLogLogPlus aggregation function for distinct count

2023-09-19 Thread via GitHub
jackjlli commented on code in PR #11346: URL: https://github.com/apache/pinot/pull/11346#discussion_r1330497084 ## pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountHLLPlusAggregationFunction.java: ## @@ -0,0 +1,473 @@ +/** + * Licensed to

[GitHub] [pinot] walterddr opened a new issue, #11623: [multistage] implement cancel and get running query

2023-09-19 Thread via GitHub
walterddr opened a new issue, #11623: URL: https://github.com/apache/pinot/issues/11623 implement the 2 capability in multi-stage enigne

[pinot] branch master updated (60abbcc7a5 -> fa51477350)

2023-09-19 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 60abbcc7a5 Refactor BasicAuthUtils from pinot-core to pinot-common and remove pinot-core dependency from

[GitHub] [pinot] xiangfu0 merged pull request #11619: fix jackson vulnerability

2023-09-19 Thread via GitHub
xiangfu0 merged PR #11619: URL: https://github.com/apache/pinot/pull/11619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[pinot] branch master updated (7990712fd9 -> 60abbcc7a5)

2023-09-19 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from 7990712fd9 Support several extensions on indexes (#11600) add 60abbcc7a5 Refactor BasicAuthUtils from pinot-core

[GitHub] [pinot] xiangfu0 merged pull request #11620: Refactor BasicAuthUtils from pinot-core to pinot-common

2023-09-19 Thread via GitHub
xiangfu0 merged PR #11620: URL: https://github.com/apache/pinot/pull/11620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[pinot-site] branch dev updated: apache pinot 1.0 blog and older blogs (#85)

2023-09-19 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/pinot-site.git The following commit(s) were added to refs/heads/dev by this push: new 704112a5 apache pinot 1.0 blog and older blogs

[GitHub] [pinot-site] xiangfu0 merged pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
xiangfu0 merged PR #85: URL: https://github.com/apache/pinot-site/pull/85 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot-site] walterddr commented on pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
walterddr commented on PR #85: URL: https://github.com/apache/pinot-site/pull/85#issuecomment-1726101349 Consider only do 1.0 blogs in the PR. We can back fill the rest separately -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [pinot-site] walterddr commented on pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
walterddr commented on PR #85: URL: https://github.com/apache/pinot-site/pull/85#issuecomment-1726100380 The following frontmatter: { "title": "Geospatial Indexing in Apache Pinot", "author": "Hubert Dulay", "author_title": "Hubert Dulay, Mayank Shrivastava, Neha Pawar",

[GitHub] [pinot-site] xiangfu0 commented on a diff in pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
xiangfu0 commented on code in PR #85: URL: https://github.com/apache/pinot-site/pull/85#discussion_r1330445558 ## website/blog/2023-09-19-Annoucing-Apache-Pinot-1-0.md: ## @@ -0,0 +1,289 @@ +--- +title: Geospatial Indexing in Apache Pinot Review Comment: Title should be

[GitHub] [pinot] rohityadav1993 commented on a diff in pull request #11584: custom partial upsert row merger

2023-09-19 Thread via GitHub
rohityadav1993 commented on code in PR #11584: URL: https://github.com/apache/pinot/pull/11584#discussion_r1330441934 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/readers/LazyRow.java: ## @@ -0,0 +1,94 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] rohityadav1993 commented on a diff in pull request #11584: custom partial upsert row merger

2023-09-19 Thread via GitHub
rohityadav1993 commented on code in PR #11584: URL: https://github.com/apache/pinot/pull/11584#discussion_r1330441934 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/readers/LazyRow.java: ## @@ -0,0 +1,94 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] ankitsultana commented on a diff in pull request #11561: Decouple metrics Key , client ID in stream consumer and add identifiers for kafka steam clientID to prevent mBean already exis

2023-09-19 Thread via GitHub
ankitsultana commented on code in PR #11561: URL: https://github.com/apache/pinot/pull/11561#discussion_r1330433141 ## pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaConsumerFactory.java: ## @@ -25,18 +25,22 @@

[pinot] branch master updated (e1236429a7 -> 7990712fd9)

2023-09-19 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git from e1236429a7 Fix the alias handling in single-stage engine (#11610) add 7990712fd9 Support several extensions on

[GitHub] [pinot] Jackie-Jiang merged pull request #11600: Support several extensions on indexes

2023-09-19 Thread via GitHub
Jackie-Jiang merged PR #11600: URL: https://github.com/apache/pinot/pull/11600 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [pinot] GSharayu commented on pull request #11591: Throw exception when schema name doesn't match table name during table creation

2023-09-19 Thread via GitHub
GSharayu commented on PR #11591: URL: https://github.com/apache/pinot/pull/11591#issuecomment-1726059351 > > Hi @xiangfu0 @Jackie-Jiang , > > As @mcvsubbu mentioned, We are working with partner team to resolve the table creation flow since this will error out for new tables. As I

[GitHub] [pinot-site] xiangfu0 commented on pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
xiangfu0 commented on PR #85: URL: https://github.com/apache/pinot-site/pull/85#issuecomment-1726058067 Any idea on the ci check failure? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [pinot] walterddr commented on a diff in pull request #11617: In multi-stage aggregate, directly store column index as identifier

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11617: URL: https://github.com/apache/pinot/pull/11617#discussion_r1330404415 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java: ## @@ -205,90 +196,78 @@ private TransferableBlock

[GitHub] [pinot] walterddr commented on a diff in pull request #11617: In multi-stage aggregate, directly store column index as identifier

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11617: URL: https://github.com/apache/pinot/pull/11617#discussion_r1330404415 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/AggregateOperator.java: ## @@ -205,90 +196,78 @@ private TransferableBlock

[GitHub] [pinot] walterddr commented on pull request #11577: [multistage] Add ROWS support for aggregation window functions with ORDER BY within OVER

2023-09-19 Thread via GitHub
walterddr commented on PR #11577: URL: https://github.com/apache/pinot/pull/11577#issuecomment-1725972870 > I usually let the test run and look at the difference in the plans and verify whether the planner side makes sense. At least during development of planner side code I've found

[GitHub] [pinot-site] hdulay commented on a diff in pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
hdulay commented on code in PR #85: URL: https://github.com/apache/pinot-site/pull/85#discussion_r1330365069 ## website/blog/2022-08-02-GapFill-Function-For-Time-Series-Datasets-In-Pinot.md: ## @@ -0,0 +1,224 @@ +--- Review Comment: piggy backing this to add old blog posts.

[GitHub] [pinot-site] walterddr commented on a diff in pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
walterddr commented on code in PR #85: URL: https://github.com/apache/pinot-site/pull/85#discussion_r1330363144 ## website/blog/2022-08-02-GapFill-Function-For-Time-Series-Datasets-In-Pinot.md: ## @@ -0,0 +1,224 @@ +--- Review Comment: are these relevant changes? seem like

[GitHub] [pinot] deemoliu commented on a diff in pull request #11346: Add hyperLogLogPlus aggregation function for distinct count

2023-09-19 Thread via GitHub
deemoliu commented on code in PR #11346: URL: https://github.com/apache/pinot/pull/11346#discussion_r1330357789 ## pinot-core/src/main/java/org/apache/pinot/core/operator/query/NonScanBasedAggregationOperator.java: ## @@ -233,6 +256,25 @@ private static HyperLogLog

[GitHub] [pinot] walterddr commented on a diff in pull request #11620: Refactor BasicAuthUtils from pinot-core to pinot-common

2023-09-19 Thread via GitHub
walterddr commented on code in PR #11620: URL: https://github.com/apache/pinot/pull/11620#discussion_r1330335754 ## pinot-core/src/main/java/org/apache/pinot/core/auth/BasicAuthUtils.java: ## @@ -64,21 +61,22 @@ private BasicAuthUtils() { * @return list of

[GitHub] [pinot] somandal commented on pull request #11577: [multistage] Add ROWS support for aggregation window functions with ORDER BY within OVER

2023-09-19 Thread via GitHub
somandal commented on PR #11577: URL: https://github.com/apache/pinot/pull/11577#issuecomment-1725840184 > Out of the scope of this PR, can you share how you usually generate the expected result for the query plan test? We have added so many hardcoded plan tests, and whenever we need to

[GitHub] [pinot-site] hdulay commented on pull request #85: apache pinot 1.0 blog and older blogs

2023-09-19 Thread via GitHub
hdulay commented on PR #85: URL: https://github.com/apache/pinot-site/pull/85#issuecomment-1725761017 @Jackie-Jiang, @xiangfu0 , @mayankshriv Please review and approve this blog. Thanks -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [pinot] ArnavBalyan commented on a diff in pull request #11608: Updated the response for reload API to be pretty printed

2023-09-19 Thread via GitHub
ArnavBalyan commented on code in PR #11608: URL: https://github.com/apache/pinot/pull/11608#discussion_r1330145418 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java: ## @@ -782,7 +782,7 @@ public SuccessResponse

[GitHub] [pinot] codecov-commenter commented on pull request #11621: Remove table state update operation in GET call

2023-09-19 Thread via GitHub
codecov-commenter commented on PR #11621: URL: https://github.com/apache/pinot/pull/11621#issuecomment-1725156593 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11621?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report > Merging

[GitHub] [pinot] pjpringle opened a new issue, #11622: datatables.net vulnerability in Pinot 1.0.0

2023-09-19 Thread via GitHub
pjpringle opened a new issue, #11622: URL: https://github.com/apache/pinot/issues/11622 Old version of datatables.net 1.10.7 has the following vulnerability preventing enterprise use. https://nvd.nist.gov/vuln/detail/CVE-2021-23445 Look to upgrade to latest version. -- This

[GitHub] [pinot] tibrewalpratik17 commented on a diff in pull request #11621: Remove table state update operation in GET call

2023-09-19 Thread via GitHub
tibrewalpratik17 commented on code in PR #11621: URL: https://github.com/apache/pinot/pull/11621#discussion_r1329800053 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java: ## @@ -379,87 +377,15 @@ private String

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11621: Remove table state update operation in GET call

2023-09-19 Thread via GitHub
xiangfu0 commented on code in PR #11621: URL: https://github.com/apache/pinot/pull/11621#discussion_r1329796739 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java: ## @@ -379,87 +377,15 @@ private String

[GitHub] [pinot] xiangfu0 commented on a diff in pull request #11621: Remove table state update operation in GET call

2023-09-19 Thread via GitHub
xiangfu0 commented on code in PR #11621: URL: https://github.com/apache/pinot/pull/11621#discussion_r1329796739 ## pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java: ## @@ -379,87 +377,15 @@ private String

[GitHub] [pinot] tibrewalpratik17 opened a new pull request, #11621: Remove table state update operation in GET call

2023-09-19 Thread via GitHub
tibrewalpratik17 opened a new pull request, #11621: URL: https://github.com/apache/pinot/pull/11621 label: bugfix In #11109 we added PUT api to enable / disable tables. The GET table config also does enabling / disabling which a GET operation shouldn't. I think we didn't remove this

[GitHub] [pinot] xiangfu0 commented on pull request #11574: Adding a controller task to update schema name

2023-09-19 Thread via GitHub
xiangfu0 commented on PR #11574: URL: https://github.com/apache/pinot/pull/11574#issuecomment-1725042006 > @xiangfu0 , I am not able to understand - Why are we needing a periodic task for this , as new tables creation with different schema name is anyways going to blocked . Any problems we

[GitHub] [pinot] xiangfu0 commented on pull request #11620: Refactor BasicAuthUtils from pinot-core to pinot-common

2023-09-19 Thread via GitHub
xiangfu0 commented on PR #11620: URL: https://github.com/apache/pinot/pull/11620#issuecomment-1725032777 > Seems we do have some scalar functions defined in pinot-core. Is it okay to not include them? It's ok if we exclude entire `pinot-core`, as the scalar functions won't even be

  1   2   >