[GitHub] [pinot] jtao15 commented on a change in pull request #7744: Add forceCleanup option for 'startReplaceSegments' API

2021-11-10 Thread GitBox
jtao15 commented on a change in pull request #7744: URL: https://github.com/apache/pinot/pull/7744#discussion_r747245432 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -2945,10 +2971,11 @@ public void re

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7744: Add forceCleanup option for 'startReplaceSegments' API

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7744: URL: https://github.com/apache/pinot/pull/7744#issuecomment-966016014 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7744?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7744: Add forceCleanup option for 'startReplaceSegments' API

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7744: URL: https://github.com/apache/pinot/pull/7744#issuecomment-966016014 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7744?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7744: Add forceCleanup option for 'startReplaceSegments' API

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7744: URL: https://github.com/apache/pinot/pull/7744#issuecomment-966016014 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7744?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] jackjlli commented on a change in pull request #7744: Add forceCleanup option for 'startReplaceSegments' API

2021-11-10 Thread GitBox
jackjlli commented on a change in pull request #7744: URL: https://github.com/apache/pinot/pull/7744#discussion_r747226615 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java ## @@ -2858,10 +2883,11 @@ public void

[GitHub] [pinot] codecov-commenter commented on pull request #7744: Add forceCleanup option for 'startReplaceSegments' API

2021-11-10 Thread GitBox
codecov-commenter commented on pull request #7744: URL: https://github.com/apache/pinot/pull/7744#issuecomment-966016014 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7744?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] snleee opened a new pull request #7744: Add forceCleanup option for 'startReplaceSegments' API

2021-11-10 Thread GitBox
snleee opened a new pull request #7744: URL: https://github.com/apache/pinot/pull/7744 The current API will not attempt to proactively clean up the lineage entry and throw the exception. When the offline flow fails in the middle of the replacement protocol, the second attempt will d

[pinot] branch master updated: Handle fields missing in the source in ParquetNativeRecordReader (#7742)

2021-11-10 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new af01aa5 Handle fields missing in the source in

[GitHub] [pinot] xiangfu0 merged pull request #7742: Handle fields missing in the source in ParquetNativeRecordReader

2021-11-10 Thread GitBox
xiangfu0 merged pull request #7742: URL: https://github.com/apache/pinot/pull/7742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7743: Always use SMALLEST offset criteria for new detected partitions

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7743: URL: https://github.com/apache/pinot/pull/7743#issuecomment-965947198 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7743?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] mcvsubbu commented on a change in pull request #7743: Always use SMALLEST offset criteria for new detected partitions

2021-11-10 Thread GitBox
mcvsubbu commented on a change in pull request #7743: URL: https://github.com/apache/pinot/pull/7743#discussion_r747175821 ## File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java ## @@ -872,8 +872,14 @@ p

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7743: Always use SMALLEST offset criteria for new detected partitions

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7743: URL: https://github.com/apache/pinot/pull/7743#issuecomment-965947198 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7743?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7742: Handle fields missing in the source in ParquetNativeRecordReader

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7742: URL: https://github.com/apache/pinot/pull/7742#issuecomment-965944787 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7742?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7742: Handle fields missing in the source in ParquetNativeRecordReader

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7742: URL: https://github.com/apache/pinot/pull/7742#issuecomment-965944787 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7742?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7743: Always use SMALLEST offset criteria for new detected partitions

2021-11-10 Thread GitBox
codecov-commenter commented on pull request #7743: URL: https://github.com/apache/pinot/pull/7743#issuecomment-965947198 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7743?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] codecov-commenter commented on pull request #7742: Handle fields missing in the source in ParquetNativeRecordReader

2021-11-10 Thread GitBox
codecov-commenter commented on pull request #7742: URL: https://github.com/apache/pinot/pull/7742#issuecomment-965944787 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7742?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] Jackie-Jiang opened a new pull request #7743: Always use SMALLEST offset criteria for new detected partitions

2021-11-10 Thread GitBox
Jackie-Jiang opened a new pull request #7743: URL: https://github.com/apache/pinot/pull/7743 Fix #7741 It is quite hard to add a test for this. Since the change is small and simple, didn't add a test for it. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [pinot] mqliang closed pull request #7724: Split thread cpu time into three metrics

2021-11-10 Thread GitBox
mqliang closed pull request #7724: URL: https://github.com/apache/pinot/pull/7724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [pinot] npawar opened a new pull request #7742: Handle fields missing in the source in ParquetNativeRecordReader

2021-11-10 Thread GitBox
npawar opened a new pull request #7742: URL: https://github.com/apache/pinot/pull/7742 ParquetNativeRecordReader was not handling fields which were missing at source. This is a common case when transform functions are applied, that the record reader gets a list of fields and not all of the

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7568: URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] Jackie-Jiang commented on issue #7740: Transient Status 305 error from recent master version (10/29/2021)

2021-11-10 Thread GitBox
Jackie-Jiang commented on issue #7740: URL: https://github.com/apache/pinot/issues/7740#issuecomment-965850770 I feel this issue might be related to the ZK callback lag/inconsistency. Can you please check the ZK server system usage and see if the ZK servers are overloaded when the exceptio

[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN

2021-11-10 Thread GitBox
amrishlal commented on a change in pull request #7568: URL: https://github.com/apache/pinot/pull/7568#discussion_r747069070 ## File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java ## @@ -0,0 +1,510 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] mcvsubbu commented on issue #7741: Realtime consumer loses data when new partitions are detected in stream

2021-11-10 Thread GitBox
mcvsubbu commented on issue #7741: URL: https://github.com/apache/pinot/issues/7741#issuecomment-965841009 cc: @npawar , @Jackie-Jiang , @sajjad-moradi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [pinot] mcvsubbu opened a new issue #7741: Realtime consumer loses data when new partitions are detected in stream

2021-11-10 Thread GitBox
mcvsubbu opened a new issue #7741: URL: https://github.com/apache/pinot/issues/7741 This issue happens on tables that are configured with an offset criteria of anything other than SMALLEST. Tables are often provisioned with offset criteria set to LARGEST (basically, ignore earlier

[GitHub] [pinot] Jackie-Jiang commented on issue #7546: Query with grouping columns but no aggregate ignores column aliases

2021-11-10 Thread GitBox
Jackie-Jiang commented on issue #7546: URL: https://github.com/apache/pinot/issues/7546#issuecomment-965785069 Fixed by #7678 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [pinot] Jackie-Jiang closed issue #7546: Query with grouping columns but no aggregate ignores column aliases

2021-11-10 Thread GitBox
Jackie-Jiang closed issue #7546: URL: https://github.com/apache/pinot/issues/7546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7729: Support Native FST As An Index Subtype for FST Indices

2021-11-10 Thread GitBox
Jackie-Jiang commented on a change in pull request #7729: URL: https://github.com/apache/pinot/pull/7729#discussion_r747004938 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java ## @@ -266,8 +268

[pinot] branch master updated (b66643a -> 928d4ef)

2021-11-10 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git. from b66643a remove unused Number2ObjectPair class and auxiliaries (#7736) add 928d4ef bump roaringbitmap version (#

[GitHub] [pinot] Jackie-Jiang merged pull request #7734: bump roaringbitmap version

2021-11-10 Thread GitBox
Jackie-Jiang merged pull request #7734: URL: https://github.com/apache/pinot/pull/7734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [pinot] dongxiaoman commented on issue #7740: Transient Status 305 error from recent master version (10/29/2021)

2021-11-10 Thread GitBox
dongxiaoman commented on issue #7740: URL: https://github.com/apache/pinot/issues/7740#issuecomment-965739323 Some extra logs in `pinotbroker` ``` Failed to find servers hosting segment: point_entry__108__1331__2020T1138Z for table: point_entry_REALTIME (all ONLINE/CONSUMING inst

[GitHub] [pinot] dongxiaoman opened a new issue #7740: Transient Status 305 error from recent master version (10/29/2021)

2021-11-10 Thread GitBox
dongxiaoman opened a new issue #7740: URL: https://github.com/apache/pinot/issues/7740 For the past a few days, our production cluster has seen several cases of this exception, causing queries to fail. ``` Caused by: org.apache.pinot.client.PinotClientException: Query had processing

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7739: Support default null value in data preprocessing job

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7739: URL: https://github.com/apache/pinot/pull/7739#issuecomment-965722902 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7739?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter commented on pull request #7739: Support default null value in data preprocessing job

2021-11-10 Thread GitBox
codecov-commenter commented on pull request #7739: URL: https://github.com/apache/pinot/pull/7739#issuecomment-965722902 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7739?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apac

[GitHub] [pinot] jackjlli opened a new pull request #7739: Support default null value in data preprocessing job

2021-11-10 Thread GitBox
jackjlli opened a new pull request #7739: URL: https://github.com/apache/pinot/pull/7739 ## Description This PR supports default null value for data preprocessing job. If the value of partitioning column is null, then evenly distribute the data to all the reducers. If the value of

[pinot] branch support-default-null-value-in-preprocessing created (now 87f83bf)

2021-11-10 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch support-default-null-value-in-preprocessing in repository https://gitbox.apache.org/repos/asf/pinot.git. at 87f83bf Support default null value in data preprocessing job This branch includes

[pinot] 01/01: Support default null value in data preprocessing job

2021-11-10 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch support-default-null-value-in-preprocessing in repository https://gitbox.apache.org/repos/asf/pinot.git commit 87f83bf548ef51b93bf96fba9a3aa6fa25741b67 Author: Jack Li(Analytics Engineering) Author

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7729: Support Native FST As An Index Subtype for FST Indices

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7729: URL: https://github.com/apache/pinot/pull/7729#issuecomment-964065900 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7729?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7729: Support Native FST As An Index Subtype for FST Indices

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7729: URL: https://github.com/apache/pinot/pull/7729#issuecomment-964065900 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7729?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[GitHub] [pinot] codecov-commenter edited a comment on pull request #7729: Support Native FST As An Index Subtype for FST Indices

2021-11-10 Thread GitBox
codecov-commenter edited a comment on pull request #7729: URL: https://github.com/apache/pinot/pull/7729#issuecomment-964065900 # [Codecov](https://codecov.io/gh/apache/pinot/pull/7729?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=T

[pinot] branch master updated: remove unused Number2ObjectPair class and auxiliaries (#7736)

2021-11-10 Thread kishoreg
This is an automated email from the ASF dual-hosted git repository. kishoreg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pinot.git The following commit(s) were added to refs/heads/master by this push: new b66643a remove unused Number2ObjectPair class

[GitHub] [pinot] kishoreg merged pull request #7736: remove unused Number2ObjectPair class and auxiliaries

2021-11-10 Thread GitBox
kishoreg merged pull request #7736: URL: https://github.com/apache/pinot/pull/7736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [pinot] atris commented on pull request #7729: Support Native FST As An Index Subtype for FST Indices

2021-11-10 Thread GitBox
atris commented on pull request #7729: URL: https://github.com/apache/pinot/pull/7729#issuecomment-964897922 @Jackie-Jiang Updated, please see -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [pinot] atris commented on a change in pull request #7729: Support Native FST As An Index Subtype for FST Indices

2021-11-10 Thread GitBox
atris commented on a change in pull request #7729: URL: https://github.com/apache/pinot/pull/7729#discussion_r746349341 ## File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/IndexLoadingConfig.java ## @@ -56,6 +57,7 @@ private S