[GitHub] [incubator-pinot] suvodeep-pyne commented on pull request #6026: [TE] Creating a thirdeye-dashboard module to host the dashboard server
suvodeep-pyne commented on pull request #6026: URL: https://github.com/apache/incubator-pinot/pull/6026#issuecomment-693744923 I think IntelliJ did some magic when moving those files. It is fixing the line separators as per *nix standards. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] suvodeep-pyne opened a new pull request #6026: [TE] Moving the current dashboard server code to a new module
suvodeep-pyne opened a new pull request #6026: URL: https://github.com/apache/incubator-pinot/pull/6026 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh opened a new pull request #6025: [TE] UI - add severity level into the anomaly table
jihaozh opened a new pull request #6025: URL: https://github.com/apache/incubator-pinot/pull/6025 This PR adds severity level into the anomaly table in the alerts page and the preview page. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch master updated (7cd5221 -> 2da8fd4)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 7cd5221 [TE] frontend - harleyjj/error-msgs - focus on errors in preview and detection validation (#6016) add 2da8fd4 add `@types/react-router` dep (#6023) No new revisions were added by this update. Summary of changes: pinot-controller/src/main/resources/package.json | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 merged pull request #6023: add `@types/react-router` dep
fx19880617 merged pull request #6023: URL: https://github.com/apache/incubator-pinot/pull/6023 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch severity-level-ui created (now df72056)
This is an automated email from the ASF dual-hosted git repository. jihao pushed a change to branch severity-level-ui in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at df72056 [TE] UI - add severity level in the anomaly table This branch includes the following new commits: new df72056 [TE] UI - add severity level in the anomaly table The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 01/01: [TE] UI - add severity level in the anomaly table
This is an automated email from the ASF dual-hosted git repository.
jihao pushed a commit to branch severity-level-ui
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit df7205630bafb15908112a1e5102b838d3062c95
Author: Jihao Zhang
AuthorDate: Wed Sep 16 14:32:14 2020 -0700
[TE] UI - add severity level in the anomaly table
---
.../app/pods/components/alert-details/component.js| 7 +++
.../custom/anomalies-table/severity-level/template.hbs| 1 +
.../app/pods/services/api/anomalies/service.js| 7 ++-
thirdeye/thirdeye-frontend/app/utils/anomaly.js | 15 ++-
4 files changed, 28 insertions(+), 2 deletions(-)
diff --git
a/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
b/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
index ba665e4..e0aa2d4 100644
--- a/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
+++ b/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
@@ -646,6 +646,7 @@ export default Component.extend({
set(a, 'start', a.startTime);
set(a, 'end', a.endTime);
set(a, 'feedback', a.feedback ? a.feedback.feedbackType :
a.statusClassification);
+ set(a, 'severityLabel', a.severityLabel);
if (a.feedback === 'NONE') {
set(a, 'feedback', 'NO_FEEDBACK');
}
@@ -674,6 +675,8 @@ export default Component.extend({
set(a, 'start', a.startTime);
set(a, 'end', a.endTime);
set(a, 'feedback', a.feedback ? a.feedback.feedbackType :
a.statusClassification);
+ set(a, 'severityLabel', a.severityLabel);
+
if (a.feedback === 'NONE') {
set(a, 'feedback', 'NO_FEEDBACK');
}
@@ -747,6 +750,10 @@ export default Component.extend({
component: 'custom/anomalies-table/rule',
propertyName: 'rule',
title: 'Rule'
+ }, {
+component: 'custom/anomalies-table/severity-level',
+propertyName: 'severityLabel',
+title: 'Severity Level'
}];
const rightmostColumns = isPreviewMode ? [] : [{
component: 'custom/anomalies-table/resolution',
diff --git
a/thirdeye/thirdeye-frontend/app/pods/custom/anomalies-table/severity-level/template.hbs
b/thirdeye/thirdeye-frontend/app/pods/custom/anomalies-table/severity-level/template.hbs
new file mode 100644
index 000..6718b33
--- /dev/null
+++
b/thirdeye/thirdeye-frontend/app/pods/custom/anomalies-table/severity-level/template.hbs
@@ -0,0 +1 @@
+{{record.severityLabel}}
diff --git
a/thirdeye/thirdeye-frontend/app/pods/services/api/anomalies/service.js
b/thirdeye/thirdeye-frontend/app/pods/services/api/anomalies/service.js
index 224179c..6c4b3ed 100644
--- a/thirdeye/thirdeye-frontend/app/pods/services/api/anomalies/service.js
+++ b/thirdeye/thirdeye-frontend/app/pods/services/api/anomalies/service.js
@@ -6,7 +6,8 @@ import { humanizeFloat, humanizeChange } from
'thirdeye-frontend/utils/utils';
import floatToPercent from 'thirdeye-frontend/utils/float-to-percent';
import {
getFormattedDuration,
- anomalyResponseObjNew
+ anomalyResponseObjNew,
+ anomalySeverityLevelObj
} from 'thirdeye-frontend/utils/anomaly';
const HumanizedAnomaly = EmberObject.extend({// ex:
record.humanizedChangeDisplay (humanized), record.anomaly.start (raw)
@@ -48,6 +49,10 @@ const HumanizedAnomaly = EmberObject.extend({// ex:
record.humanizedChangeDispla
start: computed.alias('anomaly.start'),
settings: computed.alias('anomaly.settings'),
settingsNum: computed.alias('anomaly.settingsNum'),
+ severityLabel: computed('anomaly.severityLabel', function () {
+return get(this, 'anomaly.severityLabel') ? anomalySeverityLevelObj.find(
+ res => res.value == get(this, 'anomaly.severityLabel')).name : 'Default'
+ }),
anomalyFeedback: computed('anomaly.feedback', function() {
return get(this, 'anomaly.feedback') ? anomalyResponseObjNew.find(res =>
res.value === get(this, 'anomaly.feedback')).name : '';
}),
diff --git a/thirdeye/thirdeye-frontend/app/utils/anomaly.js
b/thirdeye/thirdeye-frontend/app/utils/anomaly.js
index 3a2542b..d50db0e 100644
--- a/thirdeye/thirdeye-frontend/app/utils/anomaly.js
+++ b/thirdeye/thirdeye-frontend/app/utils/anomaly.js
@@ -79,6 +79,18 @@ export const anomalyResponseObjNew = [
}
];
+export const anomalySeverityLevelObj = [{
+ name: 'Default', value: 'DEFAULT'
+}, {
+ name: 'Low', value: 'LOW'
+}, {
+ name: 'Medium', value: 'MEDIUM'
+}, {
+ name: 'High', value: 'HIGH'
+}, {
+ name: 'Critical', value: 'CRITICAL'
+}]
+
export const anomalyTypeMapping = {
"DEVIATION": "Metric Deviation", "TREND_CHANGE": "Trend Change", "DATA_SLA":
"SLA Violation"
}
@@ -305,5 +317,6 @@ export default {
getBounds,
searchAnomaly,
searchAnomalyWithFilters,
- anomalyTypeMapping
+ anomalyTypeMapping,
+ anomalySeverityLevelObj
};
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6020: Add Caching in Controller Broker API
fx19880617 commented on a change in pull request #6020:
URL: https://github.com/apache/incubator-pinot/pull/6020#discussion_r489717878
##
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/listener/ClusterInstanceConfigChangeListener.java
##
@@ -0,0 +1,32 @@
+package org.apache.pinot.controller.helix.core.listener;
+
+import org.apache.helix.HelixManager;
+import org.apache.helix.NotificationContext;
+import org.apache.helix.api.listeners.InstanceConfigChangeListener;
+import org.apache.helix.model.InstanceConfig;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.pinot.common.utils.helix.HelixHelper;
+
+
+public class ClusterInstanceConfigChangeListener implements
InstanceConfigChangeListener {
+private HelixManager _helixManager;
+private List _instanceConfigs = new ArrayList<>();
+
+public ClusterInstanceConfigChangeListener(HelixManager helixManager) {
+_helixManager = helixManager;
+}
+
+@Override
+public void onInstanceConfigChange(List instanceConfigs,
NotificationContext context) {
+_instanceConfigs = instanceConfigs;
+}
+
+public List getInstanceConfigs() {
+if(_instanceConfigs.isEmpty()){
Review comment:
don't set instanceConfig in get call
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6020: Add Caching in Controller Broker API
fx19880617 commented on a change in pull request #6020:
URL: https://github.com/apache/incubator-pinot/pull/6020#discussion_r489717662
##
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/listener/ClusterInstanceConfigChangeListener.java
##
@@ -0,0 +1,32 @@
+package org.apache.pinot.controller.helix.core.listener;
+
+import org.apache.helix.HelixManager;
+import org.apache.helix.NotificationContext;
+import org.apache.helix.api.listeners.InstanceConfigChangeListener;
+import org.apache.helix.model.InstanceConfig;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.pinot.common.utils.helix.HelixHelper;
+
+
+public class ClusterInstanceConfigChangeListener implements
InstanceConfigChangeListener {
+private HelixManager _helixManager;
+private List _instanceConfigs = new ArrayList<>();
+
+public ClusterInstanceConfigChangeListener(HelixManager helixManager) {
+_helixManager = helixManager;
+}
+
+@Override
+public void onInstanceConfigChange(List instanceConfigs,
NotificationContext context) {
+_instanceConfigs = instanceConfigs;
Review comment:
_instanceConfigs = HelixHelper.getInstanceConfigs(_helixManager);
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6020: Add Caching in Controller Broker API
fx19880617 commented on a change in pull request #6020:
URL: https://github.com/apache/incubator-pinot/pull/6020#discussion_r489718101
##
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/listener/ClusterInstanceConfigChangeListener.java
##
@@ -0,0 +1,32 @@
+package org.apache.pinot.controller.helix.core.listener;
+
+import org.apache.helix.HelixManager;
+import org.apache.helix.NotificationContext;
+import org.apache.helix.api.listeners.InstanceConfigChangeListener;
+import org.apache.helix.model.InstanceConfig;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.pinot.common.utils.helix.HelixHelper;
+
+
+public class ClusterInstanceConfigChangeListener implements
InstanceConfigChangeListener {
+private HelixManager _helixManager;
+private List _instanceConfigs = new ArrayList<>();
+
+public ClusterInstanceConfigChangeListener(HelixManager helixManager) {
+_helixManager = helixManager;
Review comment:
set _instanceConfigs here
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6020: Add Caching in Controller Broker API
fx19880617 commented on a change in pull request #6020:
URL: https://github.com/apache/incubator-pinot/pull/6020#discussion_r489717469
##
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/listener/ClusterLiveInstanceChangeListener.java
##
@@ -0,0 +1,34 @@
+package org.apache.pinot.controller.helix.core.listener;
+
+import java.util.ArrayList;
+import org.apache.helix.HelixDataAccessor;
+import org.apache.helix.NotificationContext;
+import org.apache.helix.api.listeners.LiveInstanceChangeListener;
+import org.apache.helix.model.LiveInstance;
+
+import java.util.List;
+import org.apache.helix.PropertyKey.Builder;
+
+
+public class ClusterLiveInstanceChangeListener implements
LiveInstanceChangeListener {
+ private HelixDataAccessor _helixDataAccessor;
+ private Builder _keyBuilder;
+ private List _liveInstances = new ArrayList<>();
+
+ public ClusterLiveInstanceChangeListener(HelixDataAccessor
helixDataAccessor, Builder keyBuilder) {
+_helixDataAccessor = helixDataAccessor;
+_keyBuilder = keyBuilder;
+ }
+
+ @Override
+ public void onLiveInstanceChange(List liveInstances,
NotificationContext changeContext) {
+_liveInstances = liveInstances;
Review comment:
_liveInstances =
_helixDataAccessor.getProperty(_keyBuilder.liveInstances());
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on a change in pull request #6021: List of partitioners in SegmentProcessorFramework
npawar commented on a change in pull request #6021:
URL: https://github.com/apache/incubator-pinot/pull/6021#discussion_r489672177
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java
##
@@ -100,8 +110,11 @@ public void map()
}
// Partitioning
- // TODO: 2 step partitioner. 1) Apply custom partitioner 2) Apply table
config partitioner. Combine both to get final partition.
- String partition = _partitioner.getPartition(reusableRow);
+ int p = 0;
+ for (Partitioner partitioner : _partitioners) {
+partitions[p++] = partitioner.getPartition(reusableRow);
+ }
+ String partition = StringUtil.join("_", partitions);
Review comment:
Practically, for the use case I described, it will be 2. But it need not
be (there could be more custom logic). Also the json config spec has List of
partitions, so I just continued it as List.
All these things are not set in stone as of now. We will be continuosly
re-evaluating, optimizing and editing this framework, as we begin using it (for
minion, and merge). It is difficult to predict right now and I prefer to not
introduce restrictions on number of partitioners.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch master updated: [TE] frontend - harleyjj/error-msgs - focus on errors in preview and detection validation (#6016)
This is an automated email from the ASF dual-hosted git repository.
jihao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 7cd5221 [TE] frontend - harleyjj/error-msgs - focus on errors in
preview and detection validation (#6016)
7cd5221 is described below
commit 7cd5221e2407abdac8fb4da423332753f94473a0
Author: Harley Jackson
AuthorDate: Wed Sep 16 10:07:48 2020 -0700
[TE] frontend - harleyjj/error-msgs - focus on errors in preview and
detection validation (#6016)
---
.../app/pods/components/alert-details/component.js | 3 ++-
.../pods/components/detection-yaml/component.js| 25 --
.../pods/components/detection-yaml/template.hbs| 4 ++--
.../app/pods/manage/yaml/controller.js | 14 +++-
.../app/pods/manage/yaml/template.hbs | 2 ++
.../app/pods/self-serve/create-alert/controller.js | 21 +++---
.../app/pods/self-serve/create-alert/template.hbs | 3 +++
7 files changed, 63 insertions(+), 9 deletions(-)
diff --git
a/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
b/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
index ba665e4..3caaea4 100644
--- a/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
+++ b/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
@@ -1166,7 +1166,6 @@ export default Component.extend({
})
.catch(error => {
if (error.name !== 'TaskCancelation') {
-this.get('notifications').error(error, 'Error', toastOptions);
set(this, 'getAnomaliesError', true);
if (this.get('isPreviewMode')) {
this.get('sendPreviewError')({
@@ -1174,6 +1173,8 @@ export default Component.extend({
previewErrorMsg: 'There was an error generating the preview.',
previewErrorInfo: error
});
+} else {
+ this.get('notifications').error(error, 'Error', toastOptions);
}
}
});
diff --git
a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
index 910e249..854c954 100644
--- a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
+++ b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
@@ -46,6 +46,7 @@ export default Component.extend({
detectionError: false,
detectionErrorMsg: null,
detectionErrorInfo: null,
+ detectionErrorScroll: false,
init() {
@@ -59,6 +60,24 @@ export default Component.extend({
}
},
+ didRender() {
+this._super(...arguments);
+if (this.get('detectionErrorScroll')) {
+ document.getElementById("detection-error").scrollIntoView();
+ const parentResetScroll = this.get('resetScroll');
+ // reset detectionErrorScroll in parent if present, then set in component
+ parentResetScroll ? parentResetScroll('detectionErrorScroll') : null;
+ set(this, 'detectionErrorScroll', false);
+}
+if (this.get('previewErrorScroll')) {
+ document.getElementById("preview-error").scrollIntoView();
+ const parentResetScroll = this.get('resetScroll');
+ // reset previewErrorScroll in parent if present, then set in component
+ parentResetScroll ? parentResetScroll('previewErrorScroll') : null;
+ set(this, 'previewErrorScroll', false);
+}
+ },
+
/**
* Calls api's for specific metric's autocomplete
* @method _loadAutocompleteById
@@ -288,7 +307,8 @@ export default Component.extend({
this.setProperties({
detectionError: true,
detectionErrorMsg: alert_json.message,
-detectionErrorInfo: alert_json["more-info"]
+detectionErrorInfo: alert_json["more-info"],
+detectionErrorScroll: true
});
} else {
notifications.success('Detection configuration saved successfully',
'Done', toastOptions);
@@ -298,7 +318,8 @@ export default Component.extend({
this.setProperties({
detectionError: true,
detectionErrorMsg: 'Error while saving detection config.',
- detectionErrorInfo: error
+ detectionErrorInfo: error,
+ detectionErrorScroll: true
});
}
}
diff --git
a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
index db88e48..deb2c16 100644
--- a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
+++ b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
@@ -21,7 +21,7 @@
{{#if previewError}}
-
+
{{material-design-icon
name='error-outline'
[incubator-pinot] branch master updated: [TE] frontend - harleyjj/error-msgs - focus on errors in preview and detection validation (#6016)
This is an automated email from the ASF dual-hosted git repository.
jihao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 7cd5221 [TE] frontend - harleyjj/error-msgs - focus on errors in
preview and detection validation (#6016)
7cd5221 is described below
commit 7cd5221e2407abdac8fb4da423332753f94473a0
Author: Harley Jackson
AuthorDate: Wed Sep 16 10:07:48 2020 -0700
[TE] frontend - harleyjj/error-msgs - focus on errors in preview and
detection validation (#6016)
---
.../app/pods/components/alert-details/component.js | 3 ++-
.../pods/components/detection-yaml/component.js| 25 --
.../pods/components/detection-yaml/template.hbs| 4 ++--
.../app/pods/manage/yaml/controller.js | 14 +++-
.../app/pods/manage/yaml/template.hbs | 2 ++
.../app/pods/self-serve/create-alert/controller.js | 21 +++---
.../app/pods/self-serve/create-alert/template.hbs | 3 +++
7 files changed, 63 insertions(+), 9 deletions(-)
diff --git
a/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
b/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
index ba665e4..3caaea4 100644
--- a/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
+++ b/thirdeye/thirdeye-frontend/app/pods/components/alert-details/component.js
@@ -1166,7 +1166,6 @@ export default Component.extend({
})
.catch(error => {
if (error.name !== 'TaskCancelation') {
-this.get('notifications').error(error, 'Error', toastOptions);
set(this, 'getAnomaliesError', true);
if (this.get('isPreviewMode')) {
this.get('sendPreviewError')({
@@ -1174,6 +1173,8 @@ export default Component.extend({
previewErrorMsg: 'There was an error generating the preview.',
previewErrorInfo: error
});
+} else {
+ this.get('notifications').error(error, 'Error', toastOptions);
}
}
});
diff --git
a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
index 910e249..854c954 100644
--- a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
+++ b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/component.js
@@ -46,6 +46,7 @@ export default Component.extend({
detectionError: false,
detectionErrorMsg: null,
detectionErrorInfo: null,
+ detectionErrorScroll: false,
init() {
@@ -59,6 +60,24 @@ export default Component.extend({
}
},
+ didRender() {
+this._super(...arguments);
+if (this.get('detectionErrorScroll')) {
+ document.getElementById("detection-error").scrollIntoView();
+ const parentResetScroll = this.get('resetScroll');
+ // reset detectionErrorScroll in parent if present, then set in component
+ parentResetScroll ? parentResetScroll('detectionErrorScroll') : null;
+ set(this, 'detectionErrorScroll', false);
+}
+if (this.get('previewErrorScroll')) {
+ document.getElementById("preview-error").scrollIntoView();
+ const parentResetScroll = this.get('resetScroll');
+ // reset previewErrorScroll in parent if present, then set in component
+ parentResetScroll ? parentResetScroll('previewErrorScroll') : null;
+ set(this, 'previewErrorScroll', false);
+}
+ },
+
/**
* Calls api's for specific metric's autocomplete
* @method _loadAutocompleteById
@@ -288,7 +307,8 @@ export default Component.extend({
this.setProperties({
detectionError: true,
detectionErrorMsg: alert_json.message,
-detectionErrorInfo: alert_json["more-info"]
+detectionErrorInfo: alert_json["more-info"],
+detectionErrorScroll: true
});
} else {
notifications.success('Detection configuration saved successfully',
'Done', toastOptions);
@@ -298,7 +318,8 @@ export default Component.extend({
this.setProperties({
detectionError: true,
detectionErrorMsg: 'Error while saving detection config.',
- detectionErrorInfo: error
+ detectionErrorInfo: error,
+ detectionErrorScroll: true
});
}
}
diff --git
a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
index db88e48..deb2c16 100644
--- a/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
+++ b/thirdeye/thirdeye-frontend/app/pods/components/detection-yaml/template.hbs
@@ -21,7 +21,7 @@
{{#if previewError}}
-
+
{{material-design-icon
name='error-outline'
[GitHub] [incubator-pinot] jihaozh merged pull request #6016: [TE] frontend - harleyjj/error-msgs - focus on errors in preview and …
jihaozh merged pull request #6016: URL: https://github.com/apache/incubator-pinot/pull/6016 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jackjlli merged pull request #6009: Adjust schema validation logic in AvroIngestionSchemaValidator
jackjlli merged pull request #6009: URL: https://github.com/apache/incubator-pinot/pull/6009 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch master updated: Adjust schema validation logic in AvroIngestionSchemaValidator (#6009)
This is an automated email from the ASF dual-hosted git repository.
jlli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new cf71498 Adjust schema validation logic in
AvroIngestionSchemaValidator (#6009)
cf71498 is described below
commit cf71498478f120fdc182c4aa34d156232076cd18
Author: Jialiang Li
AuthorDate: Wed Sep 16 09:44:39 2020 -0700
Adjust schema validation logic in AvroIngestionSchemaValidator (#6009)
* Adjust schema validation logic in AvroIngestionSchemaValidator
* Reduce test file sizes
Co-authored-by: Jack Li(Analytics Engineering)
---
.../hadoop/data/IngestionSchemaValidatorTest.java | 51 +--
.../src/test/resources/data/test_sample_data.avro | Bin 917973 -> 2315 bytes
.../data/test_sample_data_multi_value.avro | Bin 0 -> 5108 bytes
.../avro/AvroIngestionSchemaValidator.java | 54 +++--
4 files changed, 73 insertions(+), 32 deletions(-)
diff --git
a/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/src/test/java/org/apache/pinot/hadoop/data/IngestionSchemaValidatorTest.java
b/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/src/test/java/org/apache/pinot/hadoop/data/IngestionSchemaValidatorTest.java
index fec3583..8cd0912 100644
---
a/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/src/test/java/org/apache/pinot/hadoop/data/IngestionSchemaValidatorTest.java
+++
b/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/src/test/java/org/apache/pinot/hadoop/data/IngestionSchemaValidatorTest.java
@@ -29,16 +29,16 @@ import org.testng.annotations.Test;
public class IngestionSchemaValidatorTest {
+
@Test
- public void testAvroIngestionSchemaValidator()
+ public void testAvroIngestionSchemaValidatorForSingleValueColumns()
throws Exception {
-String inputFilePath = new File(
-
Preconditions.checkNotNull(IngestionSchemaValidatorTest.class.getClassLoader().getResource("data/test_sample_data.avro"))
-.getFile()).toString();
+String inputFilePath = new File(Preconditions
+
.checkNotNull(IngestionSchemaValidatorTest.class.getClassLoader().getResource("data/test_sample_data.avro"))
+.getFile()).toString();
String recordReaderClassName =
"org.apache.pinot.plugin.inputformat.avro.AvroRecordReader";
-Schema pinotSchema = new Schema.SchemaBuilder()
-.addSingleValueDimension("column1", FieldSpec.DataType.LONG)
+Schema pinotSchema = new
Schema.SchemaBuilder().addSingleValueDimension("column1",
FieldSpec.DataType.LONG)
.addSingleValueDimension("column2", FieldSpec.DataType.INT)
.addSingleValueDimension("column3", FieldSpec.DataType.STRING)
.addSingleValueDimension("column7", FieldSpec.DataType.STRING)
@@ -53,8 +53,7 @@ public class IngestionSchemaValidatorTest {
Assert.assertFalse(ingestionSchemaValidator.getMissingPinotColumnResult().isMismatchDetected());
// Adding one extra column
-pinotSchema = new Schema.SchemaBuilder()
-.addSingleValueDimension("column1", FieldSpec.DataType.LONG)
+pinotSchema = new
Schema.SchemaBuilder().addSingleValueDimension("column1",
FieldSpec.DataType.LONG)
.addSingleValueDimension("column2", FieldSpec.DataType.INT)
.addSingleValueDimension("column3", FieldSpec.DataType.STRING)
.addSingleValueDimension("extra_column", FieldSpec.DataType.STRING)
@@ -69,11 +68,9 @@ public class IngestionSchemaValidatorTest {
Assert.assertFalse(ingestionSchemaValidator.getMultiValueStructureMismatchResult().isMismatchDetected());
Assert.assertTrue(ingestionSchemaValidator.getMissingPinotColumnResult().isMismatchDetected());
Assert.assertNotNull(ingestionSchemaValidator.getMissingPinotColumnResult().getMismatchReason());
-
System.out.println(ingestionSchemaValidator.getMissingPinotColumnResult().getMismatchReason());
// Change the data type of column1 from LONG to STRING
-pinotSchema = new Schema.SchemaBuilder()
-.addSingleValueDimension("column1", FieldSpec.DataType.STRING)
+pinotSchema = new
Schema.SchemaBuilder().addSingleValueDimension("column1",
FieldSpec.DataType.STRING)
.addSingleValueDimension("column2", FieldSpec.DataType.INT)
.addSingleValueDimension("column3", FieldSpec.DataType.STRING)
.addSingleValueDimension("column7", FieldSpec.DataType.STRING)
@@ -83,14 +80,12 @@ public class IngestionSchemaValidatorTest {
Assert.assertNotNull(ingestionSchemaValidator);
Assert.assertTrue(ingestionSchemaValidator.getDataTypeMismatchResult().isMismatchDetected());
Assert.assertNotNull(ingestionSchemaValidator.getDataTypeMismatchResult().getMismatchReason());
-
System.out.println(ingestionSchemaValidator.getDataTypeMismatchResult().getMismatchReason());
[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6021: List of partitioners in SegmentProcessorFramework
mcvsubbu commented on a change in pull request #6021:
URL: https://github.com/apache/incubator-pinot/pull/6021#discussion_r489551646
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java
##
@@ -100,8 +110,11 @@ public void map()
}
// Partitioning
- // TODO: 2 step partitioner. 1) Apply custom partitioner 2) Apply table
config partitioner. Combine both to get final partition.
- String partition = _partitioner.getPartition(reusableRow);
+ int p = 0;
+ for (Partitioner partitioner : _partitioners) {
+partitions[p++] = partitioner.getPartition(reusableRow);
+ }
+ String partition = StringUtil.join("_", partitions);
Review comment:
Maybe I am missing something, I will try to find it in the design
document.
1. It seems to me that there may be at most 2 partitioners, so is making
that a Pair better? Or, there should be a comment and an assert some place that
the size is two.
2. The word partition confused me into thinking that we are somehow
respecting partitioning of data using a partition key with underscores (with
brokers constructing some partition keys). That is clearly not the case here.
Maybe rename this as a splitKey instead of partition?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on a change in pull request #6021: List of partitioners in SegmentProcessorFramework
npawar commented on a change in pull request #6021:
URL: https://github.com/apache/incubator-pinot/pull/6021#discussion_r489182763
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java
##
@@ -100,8 +110,11 @@ public void map()
}
// Partitioning
- // TODO: 2 step partitioner. 1) Apply custom partitioner 2) Apply table
config partitioner. Combine both to get final partition.
- String partition = _partitioner.getPartition(reusableRow);
+ int p = 0;
+ for (Partitioner partitioner : _partitioners) {
+partitions[p++] = partitioner.getPartition(reusableRow);
+ }
+ String partition = StringUtil.join("_", partitions);
Review comment:
Use case: say data in input segments is spread across 3 days. In the
resulting segments, we want to create a segment for each day. Additionally, we
want partitioning on some id column for query purposes.
Partitioning by time column is first step. This doesn't affect segment
metadata or broker routing. This is simply used by the framework, and it's
scope ends with the framework. It's merely helping create date aligned input
files for Segment generation stage.
Partitioning by id column is second step. This is for queries. This will be
whatever is in the table config. Only this partition will get set in the
segment metadata. And even that will happen during segment creation.
See this comment and
discussion:https://github.com/apache/incubator-pinot/pull/5934#discussion_r486006754
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
