[GitHub] [incubator-pinot] snleee commented on a change in pull request #5728: Add segment lineage based segment selector

snleee commented on a change in pull request #5728: URL: https://github.com/apache/incubator-pinot/pull/5728#discussion_r459846427 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentselector/SegmentSelectorFactory.java ## @@ -17,19 +17,25 @@ * un

[GitHub] [incubator-pinot] snleee commented on pull request #5712: Add startReplaceSegments, endReplaceSegments controller API

snleee commented on pull request #5712: URL: https://github.com/apache/incubator-pinot/pull/5712#issuecomment-663335958 #2715 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [incubator-pinot] snleee merged pull request #5712: Add startReplaceSegments, endReplaceSegments controller API

snleee merged pull request #5712: URL: https://github.com/apache/incubator-pinot/pull/5712 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[incubator-pinot] branch master updated: Add startReplaceSegments, endReplaceSegments controller API (#5712)

This is an automated email from the ASF dual-hosted git repository. snlee pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new ef3e3c3 Add startReplaceSegments, endRe

[GitHub] [incubator-pinot] apucher opened a new pull request #5750: [TE] support optional 'name' field in data source config

apucher opened a new pull request #5750: URL: https://github.com/apache/incubator-pinot/pull/5750 ## Description This PR adds an optional "name" property to data source configs in order to allow users to specify multiple datasources of the same type (e.g. two pinot clusters). If the "na

[GitHub] [incubator-pinot] apucher opened a new pull request #5749: Thirdeye docker log crash fix

apucher opened a new pull request #5749: URL: https://github.com/apache/incubator-pinot/pull/5749 ## Description Recent changes to ThirdEye logging aren't reflected in the config files shipped with the docker image and break the launch of apachepinot/thirdeye. This PR modifies the confi

[GitHub] [incubator-pinot] suvodeep-pyne commented on pull request #5740: [TE] Added a backfill start date for Anomaly Detection

suvodeep-pyne commented on pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#issuecomment-663334111 Hi @akshayrai I replaced the `backfillStart` with the already existing `lastTimestamp`. This value is used as a checkpoint and is already being persisted in

[incubator-pinot] 01/01: support optional 'name' field in data source config proeprties

This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch support-multiple-datasources-per-type in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 02d758332d0e7b8650bb3b20670f3f374c13c4f9 Author: Alexander Pucher AuthorDate: T

[incubator-pinot] branch support-multiple-datasources-per-type created (now 02d7583)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch support-multiple-datasources-per-type in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 02d7583 support optional 'name' field in data source config proeprties Thi

[incubator-pinot] branch thirdeye-docker-log-crash-fix updated: request log

This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch thirdeye-docker-log-crash-fix in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/thirdeye-docker-log-crash-fix by this push:

[GitHub] [incubator-pinot] ianvkoeppe commented on pull request #5745: Allow serving of offline segments immediately/inclusively.

ianvkoeppe commented on pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#issuecomment-663328375 There's also a new api being added in the controller that makes it a protocol. Perhaps you could piggy-back on it to update time-boundary? @snleee What do you think?

[GitHub] [incubator-pinot] ianvkoeppe edited a comment on pull request #5745: Allow serving of offline segments immediately/inclusively.

ianvkoeppe edited a comment on pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#issuecomment-663328375 > There's also a new api being added in the controller that makes it a protocol. Perhaps you could piggy-back on it to update time-boundary? @snleee What do yo

[GitHub] [incubator-pinot] npawar opened a new pull request #5748: GROOVY transform function UDF for queries

npawar opened a new pull request #5748: URL: https://github.com/apache/incubator-pinot/pull/5748 ## Description GROOVY transform function for query time. Syntax: ``` groovy('groovy expression', argument0, argument1, argument2..., 'return value metadata') ``` - 1st arg

[GitHub] [incubator-pinot] mayankshriv commented on pull request #5745: Allow serving of offline segments immediately/inclusively.

mayankshriv commented on pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#issuecomment-663326776 > @mayankshriv, thanks for reviewing. See answers inline below. > > > 1. The first offline segment should immediately push the high-water-mark to (maxTime-1).

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #5745: Allow serving of offline segments immediately/inclusively.

mayankshriv commented on a change in pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#discussion_r459830306 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/HelixBrokerStarter.java ## @@ -235,7 +234,7 @@ public void star

[GitHub] [incubator-pinot] ianvkoeppe commented on a change in pull request #5745: Allow serving of offline segments immediately/inclusively.

ianvkoeppe commented on a change in pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#discussion_r459822723 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -140,6 +141,8 @@ public Ba

[GitHub] [incubator-pinot] akshayrai commented on a change in pull request #5713: [TE] Setup a rest client pipeline for comms between ThirdEye services; added RCA highlights API as an example

akshayrai commented on a change in pull request #5713: URL: https://github.com/apache/incubator-pinot/pull/5713#discussion_r459815532 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/common/restclient/AbstractRestClient.java ## @@ -0,0 +1,269 @@ +/

[GitHub] [incubator-pinot] ianvkoeppe commented on pull request #5745: Allow serving of offline segments immediately/inclusively.

ianvkoeppe commented on pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#issuecomment-663311913 @mayankshriv, thanks for reviewing. See answers inline below. > 1. The first offline segment should immediately push the high-water-mark to (maxTime-1). Are you

[GitHub] [incubator-pinot] ianvkoeppe commented on a change in pull request #5745: Allow serving of offline segments immediately/inclusively.

ianvkoeppe commented on a change in pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#discussion_r459814064 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -140,6 +141,8 @@ public Ba

[GitHub] [incubator-pinot] fx19880617 commented on issue #5747: Check overlapping offline segments

fx19880617 commented on issue #5747: URL: https://github.com/apache/incubator-pinot/issues/5747#issuecomment-663308715 Since Pinot expected multiple segments for same day, could you describe more detailed situation of the issue? Basically how the data segments are organized and how

[GitHub] [incubator-pinot] jackjlli commented on pull request #5746: Fix code to correctly extract value of multi-value column from avro file

jackjlli commented on pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#issuecomment-663308658 > The fix should be in the `DataTypeTransformer` instead of here. > In `DataTypeTransformer` we should be able to handle map properly I've thought about that. B

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

siddharthteotia commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459806163 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @

[GitHub] [incubator-pinot] mayankshriv edited a comment on pull request #5745: Allow serving of offline segments immediately/inclusively.

mayankshriv edited a comment on pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#issuecomment-663299681 @ianvkoeppe I am unclear on the PR description: 1. The first offline segment should immediately push the high-water-mark to (maxTime-1). Are you referring t

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

jackjlli commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459805384 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @@ -353,

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

siddharthteotia commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459805263 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

siddharthteotia commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459805052 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @

[GitHub] [incubator-pinot] npawar commented on pull request #5746: Fix code to correctly extract value of multi-value column from avro file

npawar commented on pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#issuecomment-663302060 @jackjlli this doesn't look like the right fix for the issue. For GenericRecord, we should be looking at each field recursively. Also the AvroRecordExtractorComplex

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

siddharthteotia commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459804440 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

siddharthteotia commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459804013 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @

[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

siddharthteotia commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459804013 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @

[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #5745: Allow serving of offline segments immediately/inclusively.

kishoreg commented on a change in pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#discussion_r459803167 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -140,6 +141,8 @@ public Base

[GitHub] [incubator-pinot] kishoreg commented on pull request #5745: Allow serving of offline segments immediately/inclusively.

kishoreg commented on pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#issuecomment-663300262 This is a fantastic contribution! This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-pinot] mayankshriv commented on pull request #5745: Allow serving of offline segments immediately/inclusively.

mayankshriv commented on pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745#issuecomment-663299681 @ianvkoeppe I am unclear on the PR description: 1. The first offline segment should immediately push the high-water-mark. 2. Your requirement seems contrary to t

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

jackjlli commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459797366 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @@ -353,

[incubator-pinot] branch thirdeye-docker-log-crash-fix updated: external logger settings in docker configs

This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch thirdeye-docker-log-crash-fix in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/thirdeye-docker-log-crash-fix by this push:

[incubator-pinot] branch master updated (ee72faa -> dab1c2d)

This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from ee72faa update data generator schema json with dataTimeSpec (#5739) add dab1c2d [Deepstore by-passi

[GitHub] [incubator-pinot] snleee commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

snleee commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459795372 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @@ -353,16

[GitHub] [incubator-pinot] mcvsubbu merged pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment upload fails.

mcvsubbu merged pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

jackjlli commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459794468 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @@ -353,

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #5746: Fix code to correctly extract value of multi-value column from avro file

mayankshriv commented on a change in pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746#discussion_r459793932 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroUtils.java ## @@ -3

[GitHub] [incubator-pinot] yupeng9 opened a new issue #5747: Check overlapping offline segments

yupeng9 opened a new issue #5747: URL: https://github.com/apache/incubator-pinot/issues/5747 Similar to the missing offline segment checker, it will be useful to have an overlapping segment checker on the intervals to avoid duplicate records. --

[GitHub] [incubator-pinot] jackjlli opened a new pull request #5746: Fix code to correctly extract value of multi-value column from avro file

jackjlli opened a new pull request #5746: URL: https://github.com/apache/incubator-pinot/pull/5746 ## Description This PR fixed the issue introduced from this PR: https://github.com/apache/incubator-pinot/pull/5238 Basically the value of the multi-value column should be at the 1st

[incubator-pinot] 01/01: Fix code to correctly extract value of multi-value column from avro file

This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch fix-multi-value-column-from-avro in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 9469cec040216f22f3133a5177972e5b00ebcdc2 Author: Jack Li(Analytics Engineering) AuthorD

[incubator-pinot] branch fix-multi-value-column-from-avro updated (0747ebd -> 9469cec)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch fix-multi-value-column-from-avro in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 0747ebd Fix code to correctly extract value of multi-value column from avro file

[incubator-pinot] 01/01: Fix code to correctly extract value of multi-value column from avro file

This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch fix-multi-value-column-from-avro in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 0747ebd7f0acc5cbc77bfcf11020d838e4765054 Author: Jack Li(Analytics Engineering) AuthorD

[incubator-pinot] branch fix-multi-value-column-from-avro created (now 0747ebd)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch fix-multi-value-column-from-avro in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 0747ebd Fix code to correctly extract value of multi-value column from avro file

[incubator-pinot] branch thirdeye-docker-log-crash-fix created (now 6ad19ca)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a change to branch thirdeye-docker-log-crash-fix in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 6ad19ca add log conf to Dockerfile launch script This branch includes the followin

[incubator-pinot] 01/01: add log conf to Dockerfile launch script

This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch thirdeye-docker-log-crash-fix in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 6ad19ca137da7d3913fa39b2dae53a4de6b2bed4 Author: Alexander Pucher AuthorDate: Thu Jul 2

[GitHub] [incubator-pinot] ianvkoeppe opened a new pull request #5745: Allow serving of offline segments immediately/inclusively.

ianvkoeppe opened a new pull request #5745: URL: https://github.com/apache/incubator-pinot/pull/5745 ## Description Today, Pinot serves queries to Hybrid tables by applying a time filter based on data availability. For example, when the most recent offline segment is for 1/2/200

[GitHub] [incubator-pinot] fx19880617 commented on issue #5716: Disallow dot in table name and column name.

fx19880617 commented on issue #5716: URL: https://github.com/apache/incubator-pinot/issues/5716#issuecomment-663239120 cc @mayankshriv This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #5734: Adding column name rewrite for the identifiers in the format of [table].[column]

fx19880617 commented on a change in pull request #5734: URL: https://github.com/apache/incubator-pinot/pull/5734#discussion_r459735198 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -450,18 +448,22 @@ public

[incubator-pinot] branch master updated: update data generator schema json with dataTimeSpec (#5739)

This is an automated email from the ASF dual-hosted git repository. apucher pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new ee72faa update data generator schema

[GitHub] [incubator-pinot] apucher merged pull request #5739: update data generator schema json with dataTimeSpec

apucher merged pull request #5739: URL: https://github.com/apache/incubator-pinot/pull/5739 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #5734: Adding column name rewrite for the identifiers in the format of [table].[column]

mayankshriv commented on a change in pull request #5734: URL: https://github.com/apache/incubator-pinot/pull/5734#discussion_r459728321 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -450,18 +448,22 @@ public

[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

suvodeep-pyne commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459702739 ## File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js ## @@ -12,12 +12,11 @@ dataset: dataset_to_which_this_metric_belongs rules

[GitHub] [incubator-pinot] fx19880617 merged pull request #5741: Move lambda expression to inner function in pinot-spark

fx19880617 merged pull request #5741: URL: https://github.com/apache/incubator-pinot/pull/5741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[incubator-pinot] branch master updated (f95ef04 -> 48fb6f3)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from f95ef04 Adding support for ingestion job spec as JSON (#5729) add 48fb6f3 Move lambda expression to

[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

suvodeep-pyne commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459691900 ## File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js ## @@ -29,6 +28,20 @@ rules: type: DATA_SLA # Alert if da

[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment up

chenboat commented on a change in pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#discussion_r459690008 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -187,6 +187,7 @@ "pinot.server.instan

[GitHub] [incubator-pinot] harleyjj commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

harleyjj commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459688843 ## File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js ## @@ -12,12 +12,11 @@ dataset: dataset_to_which_this_metric_belongs rules: -

[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

suvodeep-pyne commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459686686 ## File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js ## @@ -29,6 +28,20 @@ rules: type: DATA_SLA # Alert if da

[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

suvodeep-pyne commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459684283 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/translator/DetectionConfigTranslator.java ## @

[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment up

chenboat commented on a change in pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#discussion_r459682187 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SegmentCommitterFactory.java ## @@ -29,18 +33,35 @@ public c

[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

suvodeep-pyne commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459681531 ## File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js ## @@ -12,12 +12,11 @@ dataset: dataset_to_which_this_metric_belongs rules

[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

suvodeep-pyne commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459680850 ## File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js ## @@ -29,6 +28,20 @@ rules: type: DATA_SLA # Alert if da

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #5734: Adding column name rewrite for the identifiers in the format of [table].[column]

fx19880617 commented on a change in pull request #5734: URL: https://github.com/apache/incubator-pinot/pull/5734#discussion_r459675209 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -450,18 +448,22 @@ public

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #5741: Move lambda expression to inner function in pinot-spark

fx19880617 commented on a change in pull request #5741: URL: https://github.com/apache/incubator-pinot/pull/5741#discussion_r459671619 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-spark/src/main/java/org/apache/pinot/plugin/ingestion/batch/spark/SparkSegment

[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment up

chenboat commented on a change in pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#discussion_r459670728 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SegmentCommitterFactory.java ## @@ -29,18 +33,35 @@ public c

[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment up

chenboat commented on a change in pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#discussion_r459670249 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SplitSegmentCommitter.java ## @@ -70,4 +68,15 @@ public Split

[incubator-pinot] branch pinot_spark_serialization_issue updated (aaf69d8 -> 9d1759a)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch pinot_spark_serialization_issue in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard aaf69d8 Move lambda expression to inner function in pinot-spark add 9d1759a

[GitHub] [incubator-pinot] akshayrai commented on a change in pull request #5740: [TE] Added a backfill start date for Anomaly Detection

akshayrai commented on a change in pull request #5740: URL: https://github.com/apache/incubator-pinot/pull/5740#discussion_r459665554 ## File path: thirdeye/thirdeye-frontend/app/utils/yaml-tools.js ## @@ -29,6 +28,20 @@ rules: type: DATA_SLA # Alert if data i

[incubator-pinot] branch master updated (174eb4b -> f95ef04)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 174eb4b Early termination for combining selection order-by results (#5686) add f95ef04 Adding suppo

[GitHub] [incubator-pinot] fx19880617 merged pull request #5729: Adding support for ingestion job spec as JSON

fx19880617 merged pull request #5729: URL: https://github.com/apache/incubator-pinot/pull/5729 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-pinot] yupeng9 opened a new pull request #5744: [Part 1] Add geo support - add a geo aggregate function st_union

yupeng9 opened a new pull request #5744: URL: https://github.com/apache/incubator-pinot/pull/5744 ## Description It returns a multi-geometry from a set of geometries. ## Upgrade Notes Does this PR otherwise need attention when creating release notes? Things to consider:

[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #5729: Adding support for ingestion job spec as JSON

KKcorps commented on a change in pull request #5729: URL: https://github.com/apache/incubator-pinot/pull/5729#discussion_r459649882 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/ingestion/batch/IngestionJobLauncher.java ## @@ -57,21 +61,33 @@ public static Segment

[GitHub] [incubator-pinot] mcvsubbu commented on pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment upload fails.

mcvsubbu commented on pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#issuecomment-663165120 @chenboat please add some tests in another follow-up commit. Here is what I would suggest: 1. You can start a thread that behaves like deepstore, failing the commit.

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment up

mcvsubbu commented on a change in pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#discussion_r459633576 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -187,6 +187,7 @@ "pinot.server.instan

[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment up

chenboat commented on a change in pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#discussion_r459631008 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/PeerSchemeSplitSegmentCommitter.java ## @@ -0,0 +1,51 @@ +/**

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #5741: Move lambda expression to inner function in pinot-spark

fx19880617 commented on a change in pull request #5741: URL: https://github.com/apache/incubator-pinot/pull/5741#discussion_r459630478 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-spark/src/main/java/org/apache/pinot/plugin/ingestion/batch/spark/SparkSegment

[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5700: [Deepstore by-passing]Introduce a subclasses SplitSegmentCommitter which will proceeds to commit even if the segment up

chenboat commented on a change in pull request #5700: URL: https://github.com/apache/incubator-pinot/pull/5700#discussion_r459630215 ## File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SplitSegmentCommitter.java ## @@ -70,4 +68,15 @@ public Split

[GitHub] [incubator-pinot] akshayrai opened a new pull request #5743: [TE] Make few internal documentation and mocks public

akshayrai opened a new pull request #5743: URL: https://github.com/apache/incubator-pinot/pull/5743 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[incubator-pinot] branch master updated: Early termination for combining selection order-by results (#5686)

This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 174eb4b Early termination for combinin

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #5686: Early termination for combining selection order-by results

Jackie-Jiang merged pull request #5686: URL: https://github.com/apache/incubator-pinot/pull/5686 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [incubator-pinot] mcvsubbu opened a new issue #5742: Add time column consistency checks during table/schema creation

mcvsubbu opened a new issue #5742: URL: https://github.com/apache/incubator-pinot/issues/5742 We need to ensure that when a table is created, the time column is the same as that in schema. We also need to verify that when a schema is created (as is the case for offline tables, since sch

[GitHub] [incubator-pinot] kishoreg commented on pull request #5712: Add startReplaceSegments, endReplaceSegments controller API

kishoreg commented on pull request #5712: URL: https://github.com/apache/incubator-pinot/pull/5712#issuecomment-663098410 @snleee are we good to go? can we add some docs on how to use this? This is an automated message fr

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #5711: [WIP] - Supporting inverted index drop from v3 format

mayankshriv commented on a change in pull request #5711: URL: https://github.com/apache/incubator-pinot/pull/5711#discussion_r459480938 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/store/SingleFileIndexDirectory.java ## @@ -339,9 +344,92 @@ public void

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #5734: Adding column name rewrite for the identifiers in the format of [table].[column]

mayankshriv commented on a change in pull request #5734: URL: https://github.com/apache/incubator-pinot/pull/5734#discussion_r459474270 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java ## @@ -450,18 +448,22 @@ public

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #5741: Move lambda expression to inner function in pinot-spark

mayankshriv commented on a change in pull request #5741: URL: https://github.com/apache/incubator-pinot/pull/5741#discussion_r459467397 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-spark/src/main/java/org/apache/pinot/plugin/ingestion/batch/spark/SparkSegmen

[incubator-pinot] branch pinot_spark_serialization_issue updated (4b6eb03 -> aaf69d8)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch pinot_spark_serialization_issue in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 4b6eb03 Move lambda expression to inner function in pinot-spark add aaf69d8

[incubator-pinot] branch pinot_spark_serialization_issue updated (514896e -> 4b6eb03)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch pinot_spark_serialization_issue in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 514896e Update SparkSegmentGenerationJobRunner.java omit 8f63792 Move lambda

[incubator-pinot] branch enable_table_cache_and_check_on_alias_table_column_name updated (7310a1c -> 1acea4b)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch enable_table_cache_and_check_on_alias_table_column_name in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 7310a1c Adding column name rewrite for the identifiers i

[incubator-pinot] branch pinot_spark_serialization_issue updated (8f63792 -> 514896e)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch pinot_spark_serialization_issue in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 8f63792 Move lambda expression to inner function in pinot-spark add 514896e

[GitHub] [incubator-pinot] fx19880617 opened a new pull request #5741: Move lambda expression to inner function in pinot-spark

fx19880617 opened a new pull request #5741: URL: https://github.com/apache/incubator-pinot/pull/5741 ## Description We observed user reporting issue that Spark cannot serialize lambda expressions. So need to explicitly use inner function for this. --

[GitHub] [incubator-pinot] kishoreg commented on pull request #5686: Early termination for combining selection order-by results

kishoreg commented on pull request #5686: URL: https://github.com/apache/incubator-pinot/pull/5686#issuecomment-66296 Lets create another issue to move common functionality across multiple combine operator to a basecombineoperator. -

[GitHub] [incubator-pinot] kishoreg commented on pull request #4901: Use FilterQuery instead of FilterQueryTree in optimizer.

kishoreg commented on pull request #4901: URL: https://github.com/apache/incubator-pinot/pull/4901#issuecomment-662899251 @siddharthteotia closing this since we have moved over to pinotquery This is an automated message from

[GitHub] [incubator-pinot] kishoreg closed pull request #4901: Use FilterQuery instead of FilterQueryTree in optimizer.

kishoreg closed pull request #4901: URL: https://github.com/apache/incubator-pinot/pull/4901 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] rush00121 commented on a change in pull request #5726: Adding Test coverage to NoOpPinotCrypt class

rush00121 commented on a change in pull request #5726: URL: https://github.com/apache/incubator-pinot/pull/5726#discussion_r459270656 ## File path: pinot-spi/src/test/java/org/apache/pinot/spi/crypt/NoOpPinotCryptTest.java ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache So

[incubator-pinot] branch master updated: updated cluster manage UI and added table details page and segment details page (#5732)

This is an automated email from the ASF dual-hosted git repository. kishoreg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 2e16aa4 updated cluster manage UI an

[GitHub] [incubator-pinot] kishoreg merged pull request #5732: updated cluster manage UI and added table details page and segment details page

kishoreg merged pull request #5732: URL: https://github.com/apache/incubator-pinot/pull/5732 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above