Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/756#discussion_r102820217
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/FragmentWrapper.java
---
@@ -49,58 +51,135 @@ public String getId() {
return String.format("fragment-%s", major.getMajorFragmentId());
}
- public static final String[] FRAGMENT_OVERVIEW_COLUMNS = {"Major
Fragment", "Minor Fragments Reporting",
- "First Start", "Last Start", "First End", "Last End", "Min Runtime",
"Avg Runtime", "Max Runtime", "Last Update",
- "Last Progress", "Max Peak Memory"};
+ public static final String[] ACTIVE_FRAGMENT_OVERVIEW_COLUMNS = {"Major
Fragment", "Minor Fragments Reporting",
+ "First Start", "Last Start", "First End", "Last End", "Min Runtime",
"Avg Runtime", "Max Runtime", "% Busy",
+ "Last Update", "Last Progress", "Max Peak Memory"};
+
+ public static final String[] ACTIVE_FRAGMENT_OVERVIEW_COLUMNS_TOOLTIP =
{null, "# Minor Fragments Spawned",
+ null, null, null, null, "Shortest duration of a fragment", "Avg
duration of a fragment", "Longest duration of a fragment", "%time Fragments
were Busy",
+ "Last time a running fragment's status was updated", "Last time we
heard from a running fragment", null};
// Not including Major Fragment ID and Minor Fragments Reporting
- public static final int NUM_NULLABLE_OVERVIEW_COLUMNS =
FRAGMENT_OVERVIEW_COLUMNS.length - 2;
+ public static final int NUM_NULLABLE_ACTIVE_OVERVIEW_COLUMNS =
ACTIVE_FRAGMENT_OVERVIEW_COLUMNS.length - 2;
public void addSummary(TableBuilder tb) {
// Use only minor fragments that have complete profiles
// Complete iff the fragment profile has at least one operator
profile, and start and end times.
final List<MinorFragmentProfile> complete = new ArrayList<>(
Collections2.filter(major.getMinorFragmentProfileList(),
Filters.hasOperatorsAndTimes));
- tb.appendCell(new OperatorPathBuilder().setMajor(major).build(), null);
- tb.appendCell(complete.size() + " / " +
major.getMinorFragmentProfileCount(), null);
+ tb.appendCell(new OperatorPathBuilder().setMajor(major).build(), null,
null);
+ tb.appendCell(complete.size() + " / " +
major.getMinorFragmentProfileCount(), null, null);
// If there are no stats to aggregate, create an empty row
if (complete.size() < 1) {
- tb.appendRepeated("", null, NUM_NULLABLE_OVERVIEW_COLUMNS);
+ tb.appendRepeated("", null, NUM_NULLABLE_ACTIVE_OVERVIEW_COLUMNS,
null);
return;
}
final MinorFragmentProfile firstStart = Collections.min(complete,
Comparators.startTime);
final MinorFragmentProfile lastStart = Collections.max(complete,
Comparators.startTime);
- tb.appendMillis(firstStart.getStartTime() - start, null);
- tb.appendMillis(lastStart.getStartTime() - start, null);
+ tb.appendMillis(firstStart.getStartTime() - start, null, null);
+ tb.appendMillis(lastStart.getStartTime() - start, null, null);
final MinorFragmentProfile firstEnd = Collections.min(complete,
Comparators.endTime);
final MinorFragmentProfile lastEnd = Collections.max(complete,
Comparators.endTime);
- tb.appendMillis(firstEnd.getEndTime() - start, null);
- tb.appendMillis(lastEnd.getEndTime() - start, null);
+ tb.appendMillis(firstEnd.getEndTime() - start, null, null);
+ tb.appendMillis(lastEnd.getEndTime() - start, null, null);
- long total = 0;
+ long totalDuration = 0L;
+ double totalProcessInMillis = 0.0d;
+ double totalWaitInMillis = 0.0d;
for (final MinorFragmentProfile p : complete) {
- total += p.getEndTime() - p.getStartTime();
+ totalDuration += p.getEndTime() - p.getStartTime();
+ //Capture Busy & Wait Time
+ List<OperatorProfile> opProfileList = p.getOperatorProfileList();
+ for (OperatorProfile operatorProfile : opProfileList) {
+ totalProcessInMillis += operatorProfile.getProcessNanos()/1E6;
+ totalWaitInMillis += operatorProfile.getWaitNanos()/1E6;
--- End diff --
Wouldn't Rounding actually reduce accuracy as I accumulate values?
I was worried of a possible overflow (in the event of a large number of
long running fragments) if I simply accumulated all the values in nanosec
instead of millisec. Hence the conversion.
But I see your point. Rounding wouldn't hurt because as we sum, the values
would be relatively small. I'll apply this suggestion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
