[GitHub] drill pull request #742: DRILL-5242: The UI breaks when rendering profiles h...

[kkhatua]

Github user kkhatua closed the pull request at:

https://github.com/apache/drill/pull/742


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #742: DRILL-5242: The UI breaks when rendering profiles h...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/742#discussion_r101440816
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java
 ---
@@ -163,11 +165,18 @@ public String getMetricsTable() {
   null);
 
   final Number[] values = new Number[metricNames.length];
+  //Track new/Unknown Metrics
+  final Set unknownMetrics = new TreeSet();
   for (final MetricValue metric : op.getMetricList()) {
-if (metric.hasLongValue()) {
-  values[metric.getMetricId()] = metric.getLongValue();
-} else if (metric.hasDoubleValue()) {
-  values[metric.getMetricId()] = metric.getDoubleValue();
+if (metric.getMetricId() < metricNames.length) {
+  if (metric.hasLongValue()) {
+values[metric.getMetricId()] = metric.getLongValue();
+  } else if (metric.hasDoubleValue()) {
+values[metric.getMetricId()] = metric.getDoubleValue();
+  }
+} else {
+  //Tracking unknown metric IDs
+  unknownMetrics.add(metric.getMetricId());
--- End diff --

Sure. Still not convinced we need to build a map to throw away, but I guess 
doing so, while silly, is harmless.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #742: DRILL-5242: The UI breaks when rendering profiles h...

[kkhatua]

Github user kkhatua commented on a diff in the pull request:

https://github.com/apache/drill/pull/742#discussion_r100612470
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java
 ---
@@ -163,11 +165,18 @@ public String getMetricsTable() {
   null);
 
   final Number[] values = new Number[metricNames.length];
+  //Track new/Unknown Metrics
+  final Set unknownMetrics = new TreeSet();
   for (final MetricValue metric : op.getMetricList()) {
-if (metric.hasLongValue()) {
-  values[metric.getMetricId()] = metric.getLongValue();
-} else if (metric.hasDoubleValue()) {
-  values[metric.getMetricId()] = metric.getDoubleValue();
+if (metric.getMetricId() < metricNames.length) {
+  if (metric.hasLongValue()) {
+values[metric.getMetricId()] = metric.getLongValue();
+  } else if (metric.hasDoubleValue()) {
+values[metric.getMetricId()] = metric.getDoubleValue();
+  }
+} else {
+  //Tracking unknown metric IDs
+  unknownMetrics.add(metric.getMetricId());
--- End diff --

@paul-rogers Can we bless this? :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #742: DRILL-5242: The UI breaks when rendering profiles h...

[kkhatua]

Github user kkhatua commented on a diff in the pull request:

https://github.com/apache/drill/pull/742#discussion_r99975093
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java
 ---
@@ -163,11 +165,18 @@ public String getMetricsTable() {
   null);
 
   final Number[] values = new Number[metricNames.length];
+  //Track new/Unknown Metrics
+  final Set unknownMetrics = new TreeSet();
   for (final MetricValue metric : op.getMetricList()) {
-if (metric.hasLongValue()) {
-  values[metric.getMetricId()] = metric.getLongValue();
-} else if (metric.hasDoubleValue()) {
-  values[metric.getMetricId()] = metric.getDoubleValue();
+if (metric.getMetricId() < metricNames.length) {
+  if (metric.hasLongValue()) {
+values[metric.getMetricId()] = metric.getLongValue();
+  } else if (metric.hasDoubleValue()) {
+values[metric.getMetricId()] = metric.getDoubleValue();
+  }
+} else {
+  //Tracking unknown metric IDs
+  unknownMetrics.add(metric.getMetricId());
--- End diff --

The unknownMetrics set is built incase we want to use this list of metrics 
to do something. I tried logging by publishing the number of unknown metrics 
found, but the problem is that it floods the log. A fix could be to do 
something to publish it just once per rendering of the page. But either way, it 
has no real value. 
For me, the unknown metric IDs existed because the profile was created from 
a prototype/forked Drill version. It would be nice if the registry of these IDs 
was part of the profile, but that is a separate problem. 
The simplest suggestion from @parthchandra and @sudheeshkatkam was to 
ignore these metrics for now. The expectation is that elements of the profile 
which are unknown to the rendering version of Drill should never break the UI. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #742: DRILL-5242: The UI breaks when rendering profiles h...

[paul-rogers]

Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/742#discussion_r99684283
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java
 ---
@@ -163,11 +165,18 @@ public String getMetricsTable() {
   null);
 
   final Number[] values = new Number[metricNames.length];
+  //Track new/Unknown Metrics
+  final Set unknownMetrics = new TreeSet();
   for (final MetricValue metric : op.getMetricList()) {
-if (metric.hasLongValue()) {
-  values[metric.getMetricId()] = metric.getLongValue();
-} else if (metric.hasDoubleValue()) {
-  values[metric.getMetricId()] = metric.getDoubleValue();
+if (metric.getMetricId() < metricNames.length) {
+  if (metric.hasLongValue()) {
+values[metric.getMetricId()] = metric.getLongValue();
+  } else if (metric.hasDoubleValue()) {
+values[metric.getMetricId()] = metric.getDoubleValue();
+  }
+} else {
+  //Tracking unknown metric IDs
+  unknownMetrics.add(metric.getMetricId());
--- End diff --

Will this work? We leave the metric unset, then iterate over them in the 
following loop. Also, we build the unknownMetrics set, but never use it.

Suggestion: if the metric is not known, just set its value to 0, and log a 
message to the log file.

This situation occurs only when 1) opening an old profile with metrics that 
no longer exist, or 2) when adding a metric but not registering it properly.

Note that this code does not handle the case where the metric is not 
registered (the metric names is null.)

For the external sort, the problem occurred because we have two sets of 
metric enums (two implementations) but only one registry of names. The fix was 
to keep the old and new ones in sync.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #742: DRILL-5242: The UI breaks when rendering profiles h...

[kkhatua]

GitHub user kkhatua opened a pull request:

https://github.com/apache/drill/pull/742

DRILL-5242: The UI breaks when rendering profiles having unknown metrics

Skip any metrics whose metric ID is unknown, This prevents any 
ArrayIndexOutOfBoundsException from being thrown and breaking the UI rendering.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kkhatua/drill DRILL-5242

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/742.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #742


commit a949c99e099a46aad5462a115a30004a2712fa37
Author: Kunal Khatua 
Date:   2017-02-04T01:29:47Z

DRILL-5242: The UI breaks when trying to render profiles having unknown 
metrics

Skip any metrics whose metric ID is unknown, This prevents any 
ArrayIndexOutOfBoundsException from being thrown and breaking the UI rendering.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---