This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6213-98588bf2ab49aac65ebf002496eb057670e6fc07 in repository https://gitbox.apache.org/repos/asf/texera.git
commit 70c21145887920528d7d5540e3fb790b43e8b759 Author: Prateek Ganigi <[email protected]> AuthorDate: Sun Aug 30 22:31:48 2026 +0000 feat(frontend): add toggleable performance heat-map overlay (#6213) ### What changes were proposed in this PR? Adds a toggleable, canvas-level **performance heat-map overlay** (Layers -> Performance) that colors operators cold -> hot from the per-operator metrics captured in #5773. A read-only consumer of existing data and **no backend changes**. - **Three views** with bottleneck-oriented scoring: - **Runtime**: data + control processing time (matches the backend cost model) - **Time / row**: seconds per output row (slow producers / low throughput read hotter) - **I/O imbalance**: `|out − in| / (out + in)`, bounded to [0, 1] so an extreme amplifier can't dominate the scale - **Scoring/color helpers** (pure, unit-tested): `rawMetricForView`, `normalizeScores` (log1p + min-max), `scoreToColor` (colorblind-safe ramp) - **Overlay layer** modeled on the existing Grid/Regions/Workers pattern: a `BehaviorSubject` view stream on `JointGraphWrapper`, applied by the editor to the shared model (canvas + mini-map) - Heat-map owns **only the operator body fill**, so it coexists with the execution-status border - **Legend** showing the active view and its real min/max value range - **Hover tooltip** with the operator's metric value + heat score - Live recolor as stats stream in; restores default fills when toggled off Note on size: this is one cohesive sub-issue (#5774); roughly 40% of the diff is unit tests. ### Any related issues, documentation, discussions? Closes #5774. Part of umbrella #5772. Follows RFC discussion #5216. ### How was this PR tested? New Vitest specs (scoring, color, recolor methods, and the menu toggle): cd frontend && ng test --watch=false --include "**/heatmap-scoring.spec.ts" --include "**/heatmap-color.spec.ts" --include "**/heatmap-legend.component.spec.ts" --include "**/joint-ui.service.spec.ts" --include "**/menu.component.spec.ts" --include "**/workflow-editor.component.spec.ts" Also verified: `tsc --noEmit` (strict) clean, `eslint ./src` clean, Prettier clean, `ng build` succeeds, and the existing `workflow-editor`/`menu` specs still pass. Manually tested in the workspace: ran a workflow, toggled Performance on, switched all three views (recolor + legend range update), verified the hover tooltip, toggled off (fills restore), and confirmed the mini-map mirrors the colors. <!-- before/after screenshots + the three views go here --> UI screenshots Before: <img width="1469" height="837" alt="image" src="https://github.com/user-attachments/assets/cd612dac-934b-4803-ad2a-6deb311679d6"; /> After: <img width="1470" height="836" alt="image" src="https://github.com/user-attachments/assets/1c184de8-d5bd-4482-a8ae-c7f2e2d53f39"; /> ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Anthropic Claude Opus 4.7) --- .../heatmap-legend/heatmap-legend.component.html | 31 ++++ .../heatmap-legend/heatmap-legend.component.scss | 55 ++++++ .../heatmap-legend.component.spec.ts | 148 ++++++++++++++++ .../heatmap-legend/heatmap-legend.component.ts | 84 +++++++++ .../workspace/component/menu/menu.component.html | 33 ++++ .../workspace/component/menu/menu.component.scss | 20 +++ .../component/menu/menu.component.spec.ts | 38 ++++ .../app/workspace/component/menu/menu.component.ts | 20 +++ .../workflow-editor/workflow-editor.component.html | 10 ++ .../workflow-editor/workflow-editor.component.scss | 31 ++++ .../workflow-editor.component.spec.ts | 14 ++ .../workflow-editor/workflow-editor.component.ts | 170 +++++++++++++++++- .../service/heatmap/heatmap-color.spec.ts | 64 +++++++ .../app/workspace/service/heatmap/heatmap-color.ts | 66 +++++++ .../service/heatmap/heatmap-scoring.spec.ts | 192 +++++++++++++++++++++ .../workspace/service/heatmap/heatmap-scoring.ts | 176 +++++++++++++++++++ .../service/joint-ui/joint-ui.service.spec.ts | 41 +++++ .../workspace/service/joint-ui/joint-ui.service.ts | 21 +++ .../workflow-graph/model/joint-graph-wrapper.ts | 21 +++ 19 files changed, 1234 insertions(+), 1 deletion(-) diff --git a/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.html b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.html new file mode 100644 index 0000000000..9b0a2e34c8 --- /dev/null +++ b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.html @@ -0,0 +1,31 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +<div + class="heatmap-legend" + *ngIf="legend$ | async as legend"> + <div class="heatmap-legend__title">{{ legend.title }}</div> + <div + class="heatmap-legend__bar" + [style.background]="scaleGradient"></div> + <div class="heatmap-legend__labels"> + <span>{{ legend.minLabel }}</span> + <span>{{ legend.maxLabel }}</span> + </div> +</div> diff --git a/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.scss b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.scss new file mode 100644 index 0000000000..4c8f4fb2bd --- /dev/null +++ b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.scss @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +.heatmap-legend { + position: absolute; + // Bottom-left of the canvas; the mini-map and its toggle buttons sit in the bottom-right + // corner (see mini-map.component.scss). The bottom offset lifts the legend clear of the + // result panel docked along the lower edge (workspace.component.scss #result). + bottom: 72px; + left: 0px; + z-index: 10; + padding: 8px 10px; + font-size: 12px; + background: rgba(255, 255, 255, 0.95); + border: 1px solid #d9d9d9; + border-radius: 4px; + box-shadow: 0 1px 4px rgba(0, 0, 0, 0.15); + pointer-events: none; + + &__title { + font-weight: 600; + margin-bottom: 4px; + } + + &__bar { + width: 140px; + height: 10px; + border-radius: 2px; + // The gradient is bound in the template from scoreToColor(0/0.5/1) so it can't + // drift from the canvas ramp. + } + + &__labels { + display: flex; + justify-content: space-between; + margin-top: 2px; + color: #666; + } +} diff --git a/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.spec.ts b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.spec.ts new file mode 100644 index 0000000000..f86d969e85 --- /dev/null +++ b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.spec.ts @@ -0,0 +1,148 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { ComponentFixture, TestBed } from "@angular/core/testing"; +import { BehaviorSubject } from "rxjs"; +import { HeatmapLegendComponent } from "./heatmap-legend.component"; +import { WorkflowActionService } from "../../service/workflow-graph/model/workflow-action.service"; +import { WorkflowStatusService } from "../../service/workflow-status/workflow-status.service"; +import { OperatorPerformanceMetrics } from "../../service/workflow-status/performance-metrics"; +import { HeatmapView } from "../../service/heatmap/heatmap-scoring"; + +function makeMetrics(overrides: Partial<OperatorPerformanceMetrics>): OperatorPerformanceMetrics { + return { + dataProcessingTimeNs: 0, + controlProcessingTimeNs: 0, + idleTimeNs: 0, + inputRows: 0, + outputRows: 0, + inputSize: 0, + outputSize: 0, + numWorkers: 1, + ...overrides, + }; +} + +describe("HeatmapLegendComponent", () => { + let fixture: ComponentFixture<HeatmapLegendComponent>; + let viewSubject: BehaviorSubject<HeatmapView | null>; + let metricsSubject: BehaviorSubject<Record<string, OperatorPerformanceMetrics>>; + + const legendElement = (): HTMLElement | null => fixture.nativeElement.querySelector(".heatmap-legend"); + const textOf = (selector: string): string => + (fixture.nativeElement.querySelector(selector) as HTMLElement).textContent!.trim(); + const labels = (): string[] => + Array.from(fixture.nativeElement.querySelectorAll(".heatmap-legend__labels span") as NodeListOf<HTMLElement>).map( + el => el.textContent!.trim() + ); + + beforeEach(() => { + viewSubject = new BehaviorSubject<HeatmapView | null>(null); + metricsSubject = new BehaviorSubject<Record<string, OperatorPerformanceMetrics>>({}); + + const workflowActionServiceStub = { + getJointGraphWrapper: () => ({ getHeatmapViewStream: () => viewSubject.asObservable() }), + } as unknown as WorkflowActionService; + const workflowStatusServiceStub = { + getPerformanceMetricsStream: () => metricsSubject.asObservable(), + } as unknown as WorkflowStatusService; + + TestBed.configureTestingModule({ + imports: [HeatmapLegendComponent], + providers: [ + { provide: WorkflowActionService, useValue: workflowActionServiceStub }, + { provide: WorkflowStatusService, useValue: workflowStatusServiceStub }, + ], + }); + fixture = TestBed.createComponent(HeatmapLegendComponent); + fixture.detectChanges(); + }); + + it("is hidden while the overlay is off (view is null)", () => { + expect(legendElement()).toBeNull(); + }); + + it("shows the active view's title and the metric range once the overlay is on", () => { + metricsSubject.next({ + fast: makeMetrics({ dataProcessingTimeNs: 5_000_000 }), + slow: makeMetrics({ dataProcessingTimeNs: 8_620_000_000 }), + }); + viewSubject.next(HeatmapView.Runtime); + fixture.detectChanges(); + + expect(textOf(".heatmap-legend__title")).toBe("Runtime"); + expect(labels()).toEqual(["5 ms", "8.62 s"]); + }); + + it("shows — for both labels when there are no metrics yet", () => { + viewSubject.next(HeatmapView.Runtime); + fixture.detectChanges(); + + expect(labels()).toEqual(["—", "—"]); + }); + + it("keeps not-measurable operators out of the range so they don't anchor the low label", () => { + // The blocking operator (no output yet) has no time-per-row value; without + // the filter it would read as 0 and pin the min label to "0". + metricsSubject.next({ + blocking: makeMetrics({ dataProcessingTimeNs: 30_000_000_000, outputRows: 0 }), + producer: makeMetrics({ dataProcessingTimeNs: 2_000_000_000, outputRows: 4 }), + }); + viewSubject.next(HeatmapView.TimePerRow); + fixture.detectChanges(); + + // Only the producer is measurable: 0.5 s/row is both min and max. + expect(labels()).toEqual(["500.0 ms/row", "500.0 ms/row"]); + }); + + it("shows — when no operator is measurable for the view", () => { + // A lone source has no input rows, so I/O imbalance is undefined for it. + metricsSubject.next({ source: makeMetrics({ outputRows: 250 }) }); + viewSubject.next(HeatmapView.IoImbalance); + fixture.detectChanges(); + + expect(textOf(".heatmap-legend__title")).toBe("I/O imbalance"); + expect(labels()).toEqual(["—", "—"]); + }); + + it("updates the title and range when the view switches", () => { + metricsSubject.next({ + op: makeMetrics({ dataProcessingTimeNs: 5_000_000, inputRows: 1_000, outputRows: 250 }), + }); + viewSubject.next(HeatmapView.Runtime); + fixture.detectChanges(); + expect(textOf(".heatmap-legend__title")).toBe("Runtime"); + + viewSubject.next(HeatmapView.IoImbalance); + fixture.detectChanges(); + expect(textOf(".heatmap-legend__title")).toBe("I/O imbalance"); + // |250 - 1000| / (250 + 1000) = 0.6, the only operator anchors both ends. + expect(labels()).toEqual(["0.60", "0.60"]); + }); + + it("hides again when the overlay is turned off", () => { + viewSubject.next(HeatmapView.Runtime); + fixture.detectChanges(); + expect(legendElement()).not.toBeNull(); + + viewSubject.next(null); + fixture.detectChanges(); + expect(legendElement()).toBeNull(); + }); +}); diff --git a/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.ts b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.ts new file mode 100644 index 0000000000..bd57370868 --- /dev/null +++ b/frontend/src/app/workspace/component/heatmap-legend/heatmap-legend.component.ts @@ -0,0 +1,84 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { Component } from "@angular/core"; +import { AsyncPipe, NgIf } from "@angular/common"; +import { combineLatest, Observable } from "rxjs"; +import { map } from "rxjs/operators"; +import { WorkflowActionService } from "../../service/workflow-graph/model/workflow-action.service"; +import { WorkflowStatusService } from "../../service/workflow-status/workflow-status.service"; +import { OperatorPerformanceMetrics } from "../../service/workflow-status/performance-metrics"; +import { + formatMetricForView, + HeatmapView, + heatmapViewTitle, + rawMetricForView, +} from "../../service/heatmap/heatmap-scoring"; +import { scoreToColor } from "../../service/heatmap/heatmap-color"; + +interface HeatmapLegendState { + readonly title: string; + readonly minLabel: string; + readonly maxLabel: string; +} + +/** + * Presentational legend for the performance heat-map overlay. Shows the active view's name, the + * cold -> hot color scale, and the actual min/max metric values behind that scale. Hidden when the + * overlay is off (view is null). + */ +@Component({ + selector: "texera-heatmap-legend", + templateUrl: "./heatmap-legend.component.html", + styleUrls: ["./heatmap-legend.component.scss"], + imports: [NgIf, AsyncPipe], +}) +export class HeatmapLegendComponent { + public readonly legend$: Observable<HeatmapLegendState | null>; + + // Bind the scale bar's gradient from the same source as the operator fills (scoreToColor), + // so the legend can never drift from the canvas ramp. + public readonly scaleGradient = `linear-gradient(to right, ${scoreToColor(0)}, ${scoreToColor(0.5)}, ${scoreToColor(1)})`; + + constructor( + private workflowActionService: WorkflowActionService, + private workflowStatusService: WorkflowStatusService + ) { + this.legend$ = combineLatest([ + this.workflowActionService.getJointGraphWrapper().getHeatmapViewStream(), + this.workflowStatusService.getPerformanceMetricsStream(), + ]).pipe(map(([view, metrics]) => (view === null ? null : this.buildState(view, metrics)))); + } + + private buildState(view: HeatmapView, metrics: Record<string, OperatorPerformanceMetrics>): HeatmapLegendState { + // Undefined raws (view not measurable for the operator) stay out of the + // range, so they never anchor the legend's low label. + const raws = Object.values(metrics) + .map(m => rawMetricForView(m, view)) + .filter((v): v is number => v !== undefined && Number.isFinite(v)); + const hasData = raws.length > 0; + const min = hasData ? Math.min(...raws) : 0; + const max = hasData ? Math.max(...raws) : 0; + return { + title: heatmapViewTitle(view), + minLabel: hasData ? formatMetricForView(min, view) : "—", + maxLabel: hasData ? formatMetricForView(max, view) : "—", + }; + } +} diff --git a/frontend/src/app/workspace/component/menu/menu.component.html b/frontend/src/app/workspace/component/menu/menu.component.html index 34fa3dccc4..cfdeea7710 100644 --- a/frontend/src/app/workspace/component/menu/menu.component.html +++ b/frontend/src/app/workspace/component/menu/menu.component.html @@ -195,6 +195,39 @@ >Status</label > </li> + <li nz-menu-item> + <label + nz-checkbox + [(ngModel)]="showHeatmap" + (ngModelChange)="toggleHeatmap()" + >Performance</label + > + </li> + <li + nz-menu-item + *ngIf="showHeatmap" + class="heatmap-view-options"> + <nz-radio-group + [(ngModel)]="heatmapView" + (ngModelChange)="setHeatmapView($event)" + (click)="$event.stopPropagation()"> + <label + nz-radio + [nzValue]="HeatmapView.Runtime" + >Runtime</label + > + <label + nz-radio + [nzValue]="HeatmapView.TimePerRow" + >Time / row</label + > + <label + nz-radio + [nzValue]="HeatmapView.IoImbalance" + >I/O imbalance</label + > + </nz-radio-group> + </li> </ul> </nz-dropdown-menu> <button diff --git a/frontend/src/app/workspace/component/menu/menu.component.scss b/frontend/src/app/workspace/component/menu/menu.component.scss index cefda1cd79..deb31c0258 100644 --- a/frontend/src/app/workspace/component/menu/menu.component.scss +++ b/frontend/src/app/workspace/component/menu/menu.component.scss @@ -188,6 +188,26 @@ texera-coeditor-user-icon { ::ng-deep .layers-dropdown { user-select: none; + + // Present the performance views as a sub-group nested under the "Performance" toggle: + // indented, stacked vertically, with a tree-style left rule. + .heatmap-view-options { + cursor: default; + + .ant-radio-group { + display: flex; + flex-direction: column; + row-gap: 2px; + margin-left: 24px; + padding-left: 10px; + border-left: 2px solid #f0f0f0; + } + + .ant-radio-wrapper { + margin-left: 0; + color: rgba(0, 0, 0, 0.85); + } + } } .jupyter-notebook-icon { diff --git a/frontend/src/app/workspace/component/menu/menu.component.spec.ts b/frontend/src/app/workspace/component/menu/menu.component.spec.ts index 8601e7d38c..c726bfec65 100644 --- a/frontend/src/app/workspace/component/menu/menu.component.spec.ts +++ b/frontend/src/app/workspace/component/menu/menu.component.spec.ts @@ -42,6 +42,7 @@ import { WorkflowVersionService } from "../../../dashboard/service/user/workflow import { WorkflowPersistService } from "../../../common/service/workflow-persist/workflow-persist.service"; import { NotificationService } from "../../../common/service/notification/notification.service"; import { ExecutionState } from "../../types/execute-workflow.interface"; +import { HeatmapView } from "../../service/heatmap/heatmap-scoring"; import { ComputingUnitState } from "../../../common/type/computing-unit-connection.interface"; import { mockPoint, mockScanPredicate } from "../../service/workflow-graph/model/mock-workflow-data"; import { saveAs } from "file-saver"; @@ -753,6 +754,43 @@ describe("MenuComponent", () => { }); }); + describe("toggleHeatmap / setHeatmapView", () => { + it("publishes the selected view to the joint graph wrapper when enabled", () => { + const setSpy = vi.spyOn(workflowActionService.getJointGraphWrapper(), "setHeatmapView"); + + component.showHeatmap = true; + component.heatmapView = HeatmapView.TimePerRow; + component.toggleHeatmap(); + + expect(setSpy).toHaveBeenCalledWith(HeatmapView.TimePerRow); + }); + + it("publishes null to the joint graph wrapper when disabled", () => { + const setSpy = vi.spyOn(workflowActionService.getJointGraphWrapper(), "setHeatmapView"); + + component.showHeatmap = false; + component.toggleHeatmap(); + + expect(setSpy).toHaveBeenCalledWith(null); + }); + + it("pushes a newly selected view only while the overlay is enabled", () => { + const setSpy = vi.spyOn(workflowActionService.getJointGraphWrapper(), "setHeatmapView"); + + component.showHeatmap = true; + component.setHeatmapView(HeatmapView.IoImbalance); + expect(component.heatmapView).toBe(HeatmapView.IoImbalance); + expect(setSpy).toHaveBeenCalledWith(HeatmapView.IoImbalance); + + setSpy.mockClear(); + component.showHeatmap = false; + component.setHeatmapView(HeatmapView.Runtime); + // View selection is remembered, but nothing is pushed while the overlay is off. + expect(component.heatmapView).toBe(HeatmapView.Runtime); + expect(setSpy).not.toHaveBeenCalled(); + }); + }); + describe("toggleStatus", () => { it("removes hide-operator-status when enabled and repositions the status label", () => { const operator = fakeElement("operator"); diff --git a/frontend/src/app/workspace/component/menu/menu.component.ts b/frontend/src/app/workspace/component/menu/menu.component.ts index 4f88930335..b5b209b3eb 100644 --- a/frontend/src/app/workspace/component/menu/menu.component.ts +++ b/frontend/src/app/workspace/component/menu/menu.component.ts @@ -28,6 +28,7 @@ import { UndoRedoService } from "../../service/undo-redo/undo-redo.service"; import { ValidationWorkflowService } from "../../service/validation/validation-workflow.service"; import { WorkflowActionService } from "../../service/workflow-graph/model/workflow-action.service"; import { ExecutionState } from "../../types/execute-workflow.interface"; +import { HeatmapView } from "../../service/heatmap/heatmap-scoring"; import { WorkflowWebsocketService } from "../../service/workflow-websocket/workflow-websocket.service"; import { WorkflowResultExportService } from "../../service/workflow-result-export/workflow-result-export.service"; import { catchError, debounceTime, switchMap, tap } from "rxjs/operators"; @@ -64,6 +65,7 @@ import { UserIconComponent } from "../../../dashboard/component/user/user-icon/u import { NzDropdownDirective, NzDropdownMenuComponent } from "ng-zorro-antd/dropdown"; import { NzMenuDirective, NzMenuItemComponent } from "ng-zorro-antd/menu"; import { NzCheckboxComponent } from "ng-zorro-antd/checkbox"; +import { NzRadioComponent, NzRadioGroupComponent } from "ng-zorro-antd/radio"; import { NzPopoverDirective } from "ng-zorro-antd/popover"; import { NzSwitchComponent } from "ng-zorro-antd/switch"; import { NzBadgeComponent } from "ng-zorro-antd/badge"; @@ -108,6 +110,8 @@ import { JupyterPanelService } from "../../service/jupyter-panel/jupyter-panel.s NzMenuDirective, NzMenuItemComponent, NzCheckboxComponent, + NzRadioComponent, + NzRadioGroupComponent, NgTemplateOutlet, ComputingUnitSelectionComponent, NzPopoverDirective, @@ -133,6 +137,9 @@ export class MenuComponent implements OnInit, OnDestroy { public showGrid: boolean = false; public showNumWorkers: boolean = false; public showStatus: boolean = false; + public showHeatmap: boolean = false; + public heatmapView: HeatmapView = HeatmapView.Runtime; + public HeatmapView = HeatmapView; // make Angular HTML access enum definition protected readonly USER_WORKFLOW = USER_WORKFLOW; @Input() public writeAccess: boolean = false; @@ -533,6 +540,19 @@ export class MenuComponent implements OnInit, OnDestroy { this.workflowActionService.getJointGraphWrapper().setRegionsDisplayed(this.showRegion); } + public toggleHeatmap(): void { + // The editor subscribes to this stream and colors operator fills (canvas + mini-map). + // A null view turns the overlay off; a view enables it. + this.workflowActionService.getJointGraphWrapper().setHeatmapView(this.showHeatmap ? this.heatmapView : null); + } + + public setHeatmapView(view: HeatmapView): void { + this.heatmapView = view; + if (this.showHeatmap) { + this.workflowActionService.getJointGraphWrapper().setHeatmapView(view); + } + } + /** * This method will run the autoLayout function * diff --git a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.html b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.html index ed0b7cc748..0b652d1a76 100644 --- a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.html +++ b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.html @@ -21,6 +21,16 @@ <div id="workflow-editor" (contextmenu)="nzContextMenu.create($event, menu)"></div> + <texera-heatmap-legend></texera-heatmap-legend> + <div + class="heatmap-tooltip" + *ngIf="heatmapTooltip" + [style.left.px]="heatmapTooltip.x" + [style.top.px]="heatmapTooltip.y"> + <div class="heatmap-tooltip__title">{{ heatmapTooltip.title }}</div> + <div class="heatmap-tooltip__row">{{ heatmapTooltip.metricLabel }}</div> + <div class="heatmap-tooltip__row">Heat: {{ heatmapTooltip.heatLabel }}</div> + </div> <nz-dropdown-menu nzNoAnimation #menu="nzDropdownMenu"> diff --git a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.scss b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.scss index 482dac56a2..2d2928c002 100644 --- a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.scss +++ b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.scss @@ -19,12 +19,43 @@ #workflow-editor-wrapper { height: 100%; + // Anchors absolutely-positioned canvas overlays (e.g. the heat-map legend). + position: relative; } #workflow-editor { height: 100%; } +// Smoothly animate operator body fill changes for the heat-map recolor (view switch / toggle). +// Scoped to the .heatmap-active class the overlay toggles, so it never animates the +// disable/enable or validation fill paths when the overlay is off. +::ng-deep #workflow-editor.heatmap-active .body { + transition: fill 0.25s ease; +} + +.heatmap-tooltip { + position: absolute; + z-index: 11; + padding: 6px 8px; + font-size: 12px; + line-height: 1.4; + color: #fff; + background: rgba(0, 0, 0, 0.8); + border-radius: 4px; + pointer-events: none; + white-space: nowrap; + + &__title { + font-weight: 600; + margin-bottom: 2px; + } + + &__row { + color: #eee; + } +} + ::ng-deep .agent-action { // Agent action highlights - temporary 5-second visual indicators // Styles are defined inline in JointJS element, but this provides a hook for customization diff --git a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.spec.ts b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.spec.ts index 0dbb5d9f38..9597229482 100644 --- a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.spec.ts +++ b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.spec.ts @@ -18,6 +18,7 @@ */ import { WorkflowActionService } from "../../service/workflow-graph/model/workflow-action.service"; +import { HeatmapView } from "../../service/heatmap/heatmap-scoring"; import { UndoRedoService } from "../../service/undo-redo/undo-redo.service"; import { DragDropService } from "../../service/drag-drop/drag-drop.service"; import { WorkflowUtilService } from "../../service/workflow-graph/util/workflow-util.service"; @@ -120,6 +121,19 @@ describe("WorkflowEditorComponent", () => { expect(component).toBeTruthy(); }); + it("should reset the heat-map view on destroy so a re-entered workspace starts with the overlay off", () => { + // The wrapper is root-provided and outlives the editor, while the menu's + // checkbox re-initializes to off on every workspace entry; without the + // reset the stale view repaints no-data colors and the first checkbox + // click re-publishes the view instead of clearing it. + const wrapper = TestBed.inject(WorkflowActionService).getJointGraphWrapper(); + wrapper.setHeatmapView(HeatmapView.Runtime); + + fixture.destroy(); + + expect(wrapper.getHeatmapView()).toBeNull(); + }); + it("should hide operator status on the canvas by default", () => { // keeps the Status toggle off until the user enables it const editor = (component as any).editor as HTMLElement; diff --git a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.ts b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.ts index f84ed89e88..99a30acdd1 100644 --- a/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.ts +++ b/frontend/src/app/workspace/component/workflow-editor/workflow-editor.component.ts @@ -28,6 +28,13 @@ import { fromJointPaperEvent, JointUIService, linkPathStrokeColor } from "../../ import { Validation, ValidationWorkflowService } from "../../service/validation/validation-workflow.service"; import { WorkflowActionService } from "../../service/workflow-graph/model/workflow-action.service"; import { WorkflowStatusService } from "../../service/workflow-status/workflow-status.service"; +import { + formatMetricForView, + HeatmapView, + heatmapViewTitle, + normalizeScores, + rawMetricForView, +} from "../../service/heatmap/heatmap-scoring"; import { ExecutionState, OperatorState } from "../../types/execute-workflow.interface"; import { LogicalPort, OperatorLink, OperatorPredicate } from "../../types/workflow-common.interface"; import { auditTime, filter, map, takeUntil, withLatestFrom } from "rxjs/operators"; @@ -48,6 +55,7 @@ import { NzNoAnimationDirective } from "ng-zorro-antd/core/animation"; import { ContextMenuComponent } from "./context-menu/context-menu/context-menu.component"; import { NgIf } from "@angular/common"; import { AgentInteractionComponent } from "../agent/agent-interaction/agent-interaction.component"; +import { HeatmapLegendComponent } from "../heatmap-legend/heatmap-legend.component"; import { JupyterPanelService } from "../../service/jupyter-panel/jupyter-panel.service"; // jointjs interactive options for enabling and disabling interactivity @@ -89,12 +97,27 @@ export const MAIN_CANVAS = { selector: "texera-workflow-editor", templateUrl: "workflow-editor.component.html", styleUrls: ["workflow-editor.component.scss"], - imports: [NzDropdownMenuComponent, NzNoAnimationDirective, ContextMenuComponent, NgIf, AgentInteractionComponent], + imports: [ + NzDropdownMenuComponent, + NzNoAnimationDirective, + ContextMenuComponent, + NgIf, + AgentInteractionComponent, + HeatmapLegendComponent, + ], }) export class WorkflowEditorComponent implements OnInit, AfterViewInit, OnDestroy { editor!: HTMLElement; editorWrapper!: HTMLElement; paper!: joint.dia.Paper; + // Heat-map hover tooltip (shown while the Performance overlay is on). Null when hidden. + public heatmapTooltip: { + x: number; + y: number; + title: string; + metricLabel: string; + heatLabel: string; + } | null = null; private interactive: boolean = true; private _onProcessKeyboardActionObservable: Subject<void> = new Subject(); private wrapper; @@ -190,6 +213,8 @@ export class WorkflowEditorComponent implements OnInit, AfterViewInit, OnDestroy this.handlePortHighlightEvent(); this.registerPortDisplayNameChangeHandler(); this.handleOperatorStatisticsUpdate(); + this.handleHeatmapOverlay(); + this.handleHeatmapHover(); this.handleRegionEvents(); this.handleOperatorSuggestionHighlightEvent(); this.handleAgentHoverHighlight(); @@ -211,6 +236,12 @@ export class WorkflowEditorComponent implements OnInit, AfterViewInit, OnDestroy ngOnDestroy(): void { document.removeEventListener("keydown", this._handleKeyboardAction.bind(this)); + // The overlay belongs to the canvas being viewed, but the wrapper holding + // the view is root-provided and outlives this component, while the menu's + // checkbox re-initializes to off and the metrics behind the overlay are + // cleared on workspace teardown. Reset the view here so all three agree + // when a workspace is re-entered. + this.workflowActionService.getJointGraphWrapper().setHeatmapView(null); } private _handleKeyboardAction(event: any) { @@ -391,6 +422,143 @@ export class WorkflowEditorComponent implements OnInit, AfterViewInit, OnDestroy }); } + /** + * Drives the performance heat-map overlay (Layers > Performance). The overlay colors only the + * operator body fill, so it coexists with the execution-status border. Colors are derived from + * the WorkflowStatusService performance metrics via the pure heatmap-scoring helpers, so both the + * main canvas and the mini-map (shared model) update. + */ + private handleHeatmapOverlay(): void { + // Repaint whenever the active view or the metrics change (only while a view is active). + combineLatest([this.wrapper.getHeatmapViewStream(), this.workflowStatusService.getPerformanceMetricsStream()]) + .pipe(untilDestroyed(this)) + .subscribe(([view]) => { + if (view !== null) { + this.repaintHeatmapColors(view); + } + }); + + // Restore default fills when the overlay is turned off, and toggle the .heatmap-active + // class that scopes the body-fill transition, so the animation only applies while the + // overlay is on (not to the disable/enable or validation fill paths). + this.wrapper + .getHeatmapViewStream() + .pipe(untilDestroyed(this)) + .subscribe(view => { + this.editor.classList.toggle("heatmap-active", view !== null); + if (view === null) { + this.restoreAllOperatorFills(); + this.heatmapTooltip = null; + } + }); + + // Paint newly (re)added operators when the overlay is active (e.g. after reload). + this.workflowActionService + .getTexeraGraph() + .getOperatorAddStream() + .pipe(untilDestroyed(this)) + .subscribe(() => { + const view = this.wrapper.getHeatmapView(); + if (view !== null) { + this.repaintHeatmapColors(view); + } + }); + + // Enabling/disabling an operator rewrites its body fill via changeOperatorDisableStatus + // (#FFFFFF / #E0E0E0) — colors this overlay owns. Repaint so the heat-map survives a + // disable/enable while active, even on an otherwise idle workflow. This subscription is + // registered after handleDisableOperator (called earlier in ngAfterViewInit), so the + // repaint runs last and wins the fill. + this.workflowActionService + .getTexeraGraph() + .getDisabledOperatorsChangedStream() + .pipe(untilDestroyed(this)) + .subscribe(() => { + const view = this.wrapper.getHeatmapView(); + if (view !== null) { + this.repaintHeatmapColors(view); + } + }); + } + + private heatmapScores(view: HeatmapView): Record<string, number> { + const metrics = this.workflowStatusService.getCurrentPerformanceMetrics(); + const rawById: Record<string, number> = {}; + for (const operatorId of Object.keys(metrics)) { + const raw = rawMetricForView(metrics[operatorId], view); + // Not measurable for this view: leave the operator out so it gets an + // undefined score (painted as no-data) instead of anchoring the scale + // minimum as the coldest operator. + if (raw !== undefined) { + rawById[operatorId] = raw; + } + } + return normalizeScores(rawById); + } + + private repaintHeatmapColors(view: HeatmapView): void { + const scores = this.heatmapScores(view); + this.workflowActionService + .getTexeraGraph() + .getAllOperators() + .forEach(op => this.jointUIService.applyHeatmapColor(this.paper, op.operatorID, scores[op.operatorID])); + } + + private restoreAllOperatorFills(): void { + this.workflowActionService + .getTexeraGraph() + .getAllOperators() + .forEach(op => this.jointUIService.restoreOperatorFill(this.paper, op)); + } + + /** + * Shows a small tooltip with the hovered operator's metric value and heat score for the active + * view. Only active while the heat-map overlay is on. + */ + private handleHeatmapHover(): void { + fromJointPaperEvent(this.paper, "element:mouseenter") + .pipe(untilDestroyed(this)) + .subscribe(([elementView, evt]) => { + const view = this.wrapper.getHeatmapView(); + if (view === null) { + return; + } + const operatorId = elementView.model.id.toString(); + if (!this.workflowActionService.getTexeraGraph().hasOperator(operatorId)) { + return; + } + const metrics = this.workflowStatusService.getCurrentPerformanceMetrics()[operatorId]; + const score = this.heatmapScores(view)[operatorId]; + const rect = this.editor.getBoundingClientRect(); + const mouseEvent = evt as unknown as MouseEvent; + // "—" marks the absence of a value — missing metrics for both fields, or a + // view that is not measurable for the operator (metricLabel) — so neither is + // confused with a genuine zero value. + this.heatmapTooltip = { + x: mouseEvent.clientX - rect.left + 12, + y: mouseEvent.clientY - rect.top + 12, + title: heatmapViewTitle(view), + metricLabel: metrics ? formatMetricForView(rawMetricForView(metrics, view), view) : "—", + heatLabel: score === undefined ? "—" : `${Math.round(score * 100)}%`, + }; + // JointJS paper events fire outside Angular's zone, so trigger change detection + // for the tooltip to render (mirrors the chat-popover handling). + this.changeDetectorRef.detectChanges(); + }); + + fromJointPaperEvent(this.paper, "element:mouseleave") + .pipe(untilDestroyed(this)) + .subscribe(() => { + // Stay inert when the overlay is off (no tooltip is ever shown then), and skip the + // change-detection pass when there is nothing to clear. + if (this.wrapper.getHeatmapView() === null || this.heatmapTooltip === null) { + return; + } + this.heatmapTooltip = null; + this.changeDetectorRef.detectChanges(); + }); + } + /** * Single source of truth for the operator's border color. Both the * validation stream and the operator-add stream route through here so diff --git a/frontend/src/app/workspace/service/heatmap/heatmap-color.spec.ts b/frontend/src/app/workspace/service/heatmap/heatmap-color.spec.ts new file mode 100644 index 0000000000..41345e068b --- /dev/null +++ b/frontend/src/app/workspace/service/heatmap/heatmap-color.spec.ts @@ -0,0 +1,64 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { HEATMAP_NO_DATA_COLOR, scoreToColor } from "./heatmap-color"; + +describe("scoreToColor", () => { + it("maps 0 to the cold stop (blue)", () => { + expect(scoreToColor(0)).toBe("#5b9bd5"); + }); + + it("maps 0.5 to the mid stop (pale yellow)", () => { + expect(scoreToColor(0.5)).toBe("#ffffbf"); + }); + + it("maps 1 to the hot stop (red)", () => { + expect(scoreToColor(1)).toBe("#e05a52"); + }); + + it("clamps values below 0 to the cold stop", () => { + expect(scoreToColor(-0.5)).toBe(scoreToColor(0)); + }); + + it("clamps values above 1 to the hot stop", () => { + expect(scoreToColor(2)).toBe(scoreToColor(1)); + }); + + it("maps non-finite scores to the ramp endpoints (NaN -> cold, Infinity -> hot)", () => { + expect(scoreToColor(Number.NaN)).toBe(scoreToColor(0)); + expect(scoreToColor(Number.POSITIVE_INFINITY)).toBe(scoreToColor(1)); + }); + + it("returns a valid hex color for interpolated values", () => { + for (const score of [0.1, 0.25, 0.5, 0.75, 0.9]) { + expect(scoreToColor(score)).toMatch(/^#[0-9a-f]{6}$/); + } + }); + + it("interpolates between stops rather than snapping to an endpoint", () => { + const mid = scoreToColor(0.25); + expect(mid).not.toBe(scoreToColor(0)); + expect(mid).not.toBe(scoreToColor(0.5)); + }); + + it("exposes a distinct neutral no-data color", () => { + expect(HEATMAP_NO_DATA_COLOR).toMatch(/^#[0-9a-f]{6}$/); + expect(HEATMAP_NO_DATA_COLOR).not.toBe(scoreToColor(0)); + }); +}); diff --git a/frontend/src/app/workspace/service/heatmap/heatmap-color.ts b/frontend/src/app/workspace/service/heatmap/heatmap-color.ts new file mode 100644 index 0000000000..726a1d0cd2 --- /dev/null +++ b/frontend/src/app/workspace/service/heatmap/heatmap-color.ts @@ -0,0 +1,66 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +type Rgb = readonly [number, number, number]; + +/** + * Cold -> hot ramp (ColorBrewer RdYlBu, reversed): blue -> pale yellow -> red. + * Chosen because it is colorblind-safer than a rainbow ramp. + */ +const COLD: Rgb = [91, 155, 213]; // #5b9bd5 — bright blue (still light enough for readable labels) +const MID: Rgb = [255, 255, 191]; // #ffffbf — pale yellow +const HOT: Rgb = [224, 90, 82]; // #e05a52 — bright red + +/** + * Neutral fill for an operator with no score — either no metrics captured yet, or the active + * view is not measurable for it. + */ +export const HEATMAP_NO_DATA_COLOR = "#eeeeee"; + +function clamp01(value: number): number { + // Guard NaN (which would otherwise produce "#NaNNaNNaN"); it maps to 0 (cold). + // Infinity still clamps to 1 (hot) via the min/max below. + const safe = Number.isNaN(value) ? 0 : value; + return Math.min(1, Math.max(0, safe)); +} + +function lerpChannel(from: number, to: number, t: number): number { + return Math.round(from + (to - from) * t); +} + +function toHex(channel: number): string { + return channel.toString(16).padStart(2, "0"); +} + +function mix(from: Rgb, to: Rgb, t: number): string { + const r = lerpChannel(from[0], to[0], t); + const g = lerpChannel(from[1], to[1], t); + const b = lerpChannel(from[2], to[2], t); + return `#${toHex(r)}${toHex(g)}${toHex(b)}`; +} + +/** + * Map a normalized [0, 1] heat score to a hex color on the cold -> hot ramp. + * Values outside [0, 1] are clamped. 0 -> cold (blue), 0.5 -> pale yellow, + * 1 -> hot (red). + */ +export function scoreToColor(score: number): string { + const t = clamp01(score); + return t <= 0.5 ? mix(COLD, MID, t / 0.5) : mix(MID, HOT, (t - 0.5) / 0.5); +} diff --git a/frontend/src/app/workspace/service/heatmap/heatmap-scoring.spec.ts b/frontend/src/app/workspace/service/heatmap/heatmap-scoring.spec.ts new file mode 100644 index 0000000000..18ecd41047 --- /dev/null +++ b/frontend/src/app/workspace/service/heatmap/heatmap-scoring.spec.ts @@ -0,0 +1,192 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { + formatMetricForView, + HeatmapView, + heatmapViewTitle, + normalizeScores, + rawMetricForView, +} from "./heatmap-scoring"; +import { OperatorPerformanceMetrics } from "../workflow-status/performance-metrics"; + +function makeMetrics(overrides: Partial<OperatorPerformanceMetrics>): OperatorPerformanceMetrics { + return { + dataProcessingTimeNs: 0, + controlProcessingTimeNs: 0, + idleTimeNs: 0, + inputRows: 0, + outputRows: 0, + inputSize: 0, + outputSize: 0, + numWorkers: 1, + ...overrides, + }; +} + +describe("rawMetricForView", () => { + it("Runtime sums data and control processing time", () => { + const m = makeMetrics({ dataProcessingTimeNs: 5_000_000, controlProcessingTimeNs: 2_000_000 }); + expect(rawMetricForView(m, HeatmapView.Runtime)).toBe(7_000_000); + }); + + it("Time-per-row returns seconds per output tuple (slow producers are hotter)", () => { + // 2s of processing over 4 rows -> 0.5s per tuple + const m = makeMetrics({ dataProcessingTimeNs: 2_000_000_000, outputRows: 4 }); + expect(rawMetricForView(m, HeatmapView.TimePerRow)).toBe(0.5); + }); + + it("Time-per-row is not measurable (undefined) before the operator emits a row", () => { + // A blocking operator mid-run has burned time but produced nothing; folding + // this into 0 would paint it coldest and anchor the scale minimum. + const m = makeMetrics({ dataProcessingTimeNs: 2_000_000_000, outputRows: 0 }); + expect(rawMetricForView(m, HeatmapView.TimePerRow)).toBeUndefined(); + }); + + it("Time-per-row returns 0 when there is no processing time (infinitely fast -> cold)", () => { + const m = makeMetrics({ dataProcessingTimeNs: 0, controlProcessingTimeNs: 0, outputRows: 10 }); + expect(rawMetricForView(m, HeatmapView.TimePerRow)).toBe(0); + }); + + it("IoImbalance scores a row-dropping operator (out < in)", () => { + // |250 - 1000| / (250 + 1000) = 0.6 + const m = makeMetrics({ inputRows: 1_000, outputRows: 250 }); + expect(rawMetricForView(m, HeatmapView.IoImbalance)).toBe(0.6); + }); + + it("IoImbalance scores an amplifying operator (out > in)", () => { + // |300 - 100| / (300 + 100) = 0.5 + const m = makeMetrics({ inputRows: 100, outputRows: 300 }); + expect(rawMetricForView(m, HeatmapView.IoImbalance)).toBe(0.5); + }); + + it("IoImbalance scores a total drop (out = 0) as maximally imbalanced", () => { + const m = makeMetrics({ inputRows: 1_000, outputRows: 0 }); + expect(rawMetricForView(m, HeatmapView.IoImbalance)).toBe(1); + }); + + it("IoImbalance stays within [0, 1] even for an extreme amplifier", () => { + const m = makeMetrics({ inputRows: 1, outputRows: 1_000_000 }); + const score = rawMetricForView(m, HeatmapView.IoImbalance); + expect(score).toBeGreaterThan(0.99); + expect(score).toBeLessThanOrEqual(1); + }); + + it("IoImbalance is 0 for a balanced operator (out == in)", () => { + const m = makeMetrics({ inputRows: 1_000, outputRows: 1_000 }); + expect(rawMetricForView(m, HeatmapView.IoImbalance)).toBe(0); + }); + + it("IoImbalance is not measurable (undefined) without input rows", () => { + // Sources (and operators that have consumed nothing yet) have no in/out + // ratio; 0 would collide with a genuinely balanced operator. + const source = makeMetrics({ inputRows: 0, outputRows: 250 }); + expect(rawMetricForView(source, HeatmapView.IoImbalance)).toBeUndefined(); + + const idle = makeMetrics({ inputRows: 0, outputRows: 0 }); + expect(rawMetricForView(idle, HeatmapView.IoImbalance)).toBeUndefined(); + }); +}); + +describe("normalizeScores", () => { + it("returns an empty object for empty input", () => { + expect(normalizeScores({})).toEqual({}); + }); + + it("scores a single operator that did work as 1", () => { + expect(normalizeScores({ a: 42 })).toEqual({ a: 1 }); + }); + + it("scores a single operator that did no work as 0.5", () => { + expect(normalizeScores({ a: 0 })).toEqual({ a: 0.5 }); + }); + + it("scores all-equal values as 0.5 (avoids divide-by-zero)", () => { + expect(normalizeScores({ a: 5, b: 5, c: 5 })).toEqual({ a: 0.5, b: 0.5, c: 0.5 }); + }); + + it("scores all-zero values as 0.5", () => { + expect(normalizeScores({ a: 0, b: 0 })).toEqual({ a: 0.5, b: 0.5 }); + }); + + it("maps the min to 0 and the max to 1 for two distinct values", () => { + const scores = normalizeScores({ low: 1, high: 100 }); + expect(scores["low"]).toBe(0); + expect(scores["high"]).toBe(1); + }); + + it("keeps all scores within [0, 1]", () => { + const scores = normalizeScores({ a: 3, b: 50, c: 900, d: 12 }); + for (const s of Object.values(scores)) { + expect(s).toBeGreaterThanOrEqual(0); + expect(s).toBeLessThanOrEqual(1); + } + }); + + it("compresses heavy-tailed values so the middle is not flattened to ~0", () => { + // Linear min-max would map 100 to ~0.1; log scaling lifts it above 0.5. + const scores = normalizeScores({ small: 1, mid: 100, big: 1000 }); + expect(scores["small"]).toBe(0); + expect(scores["big"]).toBe(1); + expect(scores["mid"]).toBeGreaterThan(0.5); + }); +}); + +describe("formatMetricForView", () => { + it("formats Runtime nanoseconds as a human duration", () => { + expect(formatMetricForView(8_620_000_000, HeatmapView.Runtime)).toBe("8.62 s"); + expect(formatMetricForView(5_000_000, HeatmapView.Runtime)).toBe("5 ms"); + expect(formatMetricForView(2_000, HeatmapView.Runtime)).toBe("2 µs"); + expect(formatMetricForView(500, HeatmapView.Runtime)).toBe("500 ns"); + }); + + it("formats Time-per-row as time-per-row", () => { + expect(formatMetricForView(2, HeatmapView.TimePerRow)).toBe("2.00 s/row"); + expect(formatMetricForView(0.0015, HeatmapView.TimePerRow)).toBe("1.5 ms/row"); + expect(formatMetricForView(0.0005, HeatmapView.TimePerRow)).toBe("500 µs/row"); + }); + + it("formats I/O imbalance as a 2-decimal ratio", () => { + expect(formatMetricForView(0.75, HeatmapView.IoImbalance)).toBe("0.75"); + expect(formatMetricForView(4, HeatmapView.IoImbalance)).toBe("4.00"); + }); + + it("renders 0 for non-positive or non-finite values", () => { + expect(formatMetricForView(0, HeatmapView.Runtime)).toBe("0"); + expect(formatMetricForView(Number.NaN, HeatmapView.TimePerRow)).toBe("0"); + expect(formatMetricForView(Number.POSITIVE_INFINITY, HeatmapView.IoImbalance)).toBe("0"); + }); + + it("renders a not-measurable (undefined) value as — so it is distinct from a genuine zero", () => { + expect(formatMetricForView(undefined, HeatmapView.TimePerRow)).toBe("—"); + expect(formatMetricForView(undefined, HeatmapView.IoImbalance)).toBe("—"); + }); +}); + +describe("heatmapViewTitle", () => { + it("returns a human-readable title for each view", () => { + expect(heatmapViewTitle(HeatmapView.Runtime)).toBe("Runtime"); + expect(heatmapViewTitle(HeatmapView.TimePerRow)).toBe("Time / row"); + expect(heatmapViewTitle(HeatmapView.IoImbalance)).toBe("I/O imbalance"); + }); + + it("falls back to a generic title for an unexpected value", () => { + expect(heatmapViewTitle("bogus" as HeatmapView)).toBe("Performance"); + }); +}); diff --git a/frontend/src/app/workspace/service/heatmap/heatmap-scoring.ts b/frontend/src/app/workspace/service/heatmap/heatmap-scoring.ts new file mode 100644 index 0000000000..8a62c68032 --- /dev/null +++ b/frontend/src/app/workspace/service/heatmap/heatmap-scoring.ts @@ -0,0 +1,176 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { OperatorPerformanceMetrics } from "../workflow-status/performance-metrics"; + +/** + * The three heat-map views. Each answers a different "where should I look?" + * question; see {@link rawMetricForView} for the per-operator cost each uses. + * String-valued so the selection serializes readably (e.g. to localStorage). + */ +export enum HeatmapView { + Runtime = "runtime", + // Seconds per output row — the reciprocal of throughput, so it grows (hotter) as throughput + // falls. Named directionally rather than "throughput" to match how the ramp reads. + TimePerRow = "time-per-row", + IoImbalance = "io-imbalance", +} + +/** + * Per-operator raw cost for a view, BEFORE normalization. Higher = hotter (more + * worth a look). + * + * Returns undefined when the view's metric is NOT MEASURABLE for the operator, + * as opposed to measured-as-zero. Callers must exclude undefined from + * normalization and the legend range and render it as no-data; folding it into + * 0 would paint the operator coldest and anchor the scale minimum — e.g. a + * blocking operator mid-run (rows consumed, none emitted yet) would read as + * the fastest thing on the canvas. + * + * - Runtime: data + control processing time — slower operators are hotter. + * Always measurable (0 = genuinely idle). + * - TimePerRow: seconds per output tuple — slow producers (low throughput) are + * hotter; undefined until the operator has emitted a row. + * - IoImbalance: |out - in| / (out + in) — operators that drop OR amplify rows + * are hotter; a balanced operator -> 0 (cold); undefined without + * input rows (sources, or nothing consumed yet). Bounded to + * [0, 1] so an extreme amplifier can't dominate the scale. + */ +export function rawMetricForView(metrics: OperatorPerformanceMetrics, view: HeatmapView): number | undefined { + switch (view) { + case HeatmapView.Runtime: + return metrics.dataProcessingTimeNs + metrics.controlProcessingTimeNs; + case HeatmapView.TimePerRow: { + if (metrics.outputRows === 0) { + return undefined; + } + const timeSec = (metrics.dataProcessingTimeNs + metrics.controlProcessingTimeNs) / 1e9; + return timeSec / metrics.outputRows; + } + case HeatmapView.IoImbalance: + return metrics.inputRows > 0 + ? Math.abs(metrics.outputRows - metrics.inputRows) / (metrics.outputRows + metrics.inputRows) + : undefined; + default: + return undefined; + } +} + +/** Human-readable title for a view, shown in the legend and hover tooltip. */ +export function heatmapViewTitle(view: HeatmapView): string { + switch (view) { + case HeatmapView.Runtime: + return "Runtime"; + case HeatmapView.TimePerRow: + return "Time / row"; + case HeatmapView.IoImbalance: + return "I/O imbalance"; + default: + // Total function: guards against an unexpected (e.g. persisted) value slipping through. + return "Performance"; + } +} + +function formatNanos(ns: number): string { + if (ns >= 1e9) return `${(ns / 1e9).toFixed(2)} s`; + if (ns >= 1e6) return `${(ns / 1e6).toFixed(0)} ms`; + if (ns >= 1e3) return `${(ns / 1e3).toFixed(0)} µs`; + return `${Math.round(ns)} ns`; +} + +function formatSeconds(seconds: number): string { + if (seconds >= 1) return `${seconds.toFixed(2)} s`; + if (seconds >= 1e-3) return `${(seconds * 1e3).toFixed(1)} ms`; + return `${(seconds * 1e6).toFixed(0)} µs`; +} + +/** + * Human-readable label for a raw view metric, used by the legend to show the actual value range + * behind the color scale and by the hover tooltip. Units match each view: Runtime is a duration, + * Time/row is seconds per row, I/O imbalance is a unitless ratio. An undefined value (the view is + * not measurable for the operator, see {@link rawMetricForView}) renders as "—" so it cannot be + * confused with a genuine zero. + */ +export function formatMetricForView(value: number | undefined, view: HeatmapView): string { + if (value === undefined) { + return "—"; + } + if (!Number.isFinite(value) || value <= 0) { + return "0"; + } + switch (view) { + case HeatmapView.Runtime: + return formatNanos(value); + case HeatmapView.TimePerRow: + return `${formatSeconds(value)}/row`; + case HeatmapView.IoImbalance: + return value.toFixed(2); + default: + return String(value); + } +} + +/** + * Normalize per-operator raw costs into [0, 1] heat scores. + * + * Uses log1p compression then min-max across operators, so a single dominant + * operator does not flatten everyone else toward 0. Rules: + * - empty input -> {} + * - single operator -> 1 if it did measurable work, else 0.5 (neutral) + * - all values equal -> 0.5 for everyone (no spread to show; avoids /0) + * - otherwise -> min maps to 0, max maps to 1, rest interpolated + */ +export function normalizeScores(rawById: Record<string, number>): Record<string, number> { + const ids = Object.keys(rawById); + if (ids.length === 0) { + return {}; + } + + // Log-compress every raw cost so a heavy tail doesn't flatten everyone else. + const compressed: Record<string, number> = {}; + for (const id of ids) { + compressed[id] = Math.log1p(rawById[id]); + } + + // A single operator is trivially the hottest, unless it did no measurable work. + if (ids.length === 1) { + return { [ids[0]]: compressed[ids[0]] > 0 ? 1 : 0.5 }; + } + + const values = Object.values(compressed); + const min = Math.min(...values); + const max = Math.max(...values); + + const scores: Record<string, number> = {}; + + // Everything equal (covers all-zero): no spread to show, and avoids /0. + if (max === min) { + for (const id of ids) { + scores[id] = 0.5; + } + return scores; + } + + // Linear min-max into [0, 1]. + const range = max - min; + for (const id of ids) { + scores[id] = (compressed[id] - min) / range; + } + return scores; +} diff --git a/frontend/src/app/workspace/service/joint-ui/joint-ui.service.spec.ts b/frontend/src/app/workspace/service/joint-ui/joint-ui.service.spec.ts index c383e0963d..39e294b27d 100644 --- a/frontend/src/app/workspace/service/joint-ui/joint-ui.service.spec.ts +++ b/frontend/src/app/workspace/service/joint-ui/joint-ui.service.spec.ts @@ -23,6 +23,7 @@ import { JointUIService, operatorNameClass, operatorStateClass, operatorPortMetr import { CommentBox, OperatorPredicate } from "../../types/workflow-common.interface"; import { OperatorState } from "../../types/execute-workflow.interface"; import { Coeditor } from "../../../common/type/user"; +import { HEATMAP_NO_DATA_COLOR, scoreToColor } from "../heatmap/heatmap-color"; // Minimal mock of OperatorMetadataService — the constructor subscribes to // getOperatorMetadata() but the schemas list isn't needed for the methods @@ -631,6 +632,46 @@ describe("JointUIService", () => { }); }); + describe("applyHeatmapColor", () => { + it("paints the body fill with the ramp color for a given score", () => { + const { paper, attrSpy } = makePaperWithModel(); + const service = new JointUIService(emptyMetadataStub as never); + service.applyHeatmapColor(paper, "op-1", 1); + expect(attrSpy).toHaveBeenCalledWith("rect.body/fill", scoreToColor(1)); + }); + it("paints the neutral no-data color when the score is undefined", () => { + const { paper, attrSpy } = makePaperWithModel(); + const service = new JointUIService(emptyMetadataStub as never); + service.applyHeatmapColor(paper, "op-1", undefined); + expect(attrSpy).toHaveBeenCalledWith("rect.body/fill", HEATMAP_NO_DATA_COLOR); + }); + it("no-ops when the model is missing", () => { + const paper = { getModelById: vi.fn(() => null) } as unknown as joint.dia.Paper; + const service = new JointUIService(emptyMetadataStub as never); + expect(() => service.applyHeatmapColor(paper, "missing-op", 0.5)).not.toThrow(); + }); + }); + + describe("restoreOperatorFill", () => { + it("restores the default white fill for an enabled operator", () => { + const { paper, attrSpy } = makePaperWithModel(); + const service = new JointUIService(emptyMetadataStub as never); + service.restoreOperatorFill(paper, { operatorID: "op-1" } as OperatorPredicate); + expect(attrSpy).toHaveBeenCalledWith("rect.body/fill", "#FFFFFF"); + }); + it("restores the disabled grey fill for a disabled operator", () => { + const { paper, attrSpy } = makePaperWithModel(); + const service = new JointUIService(emptyMetadataStub as never); + service.restoreOperatorFill(paper, { operatorID: "op-1", isDisabled: true } as OperatorPredicate); + expect(attrSpy).toHaveBeenCalledWith("rect.body/fill", "#E0E0E0"); + }); + it("no-ops when the model is missing", () => { + const paper = { getModelById: vi.fn(() => null) } as unknown as joint.dia.Paper; + const service = new JointUIService(emptyMetadataStub as never); + expect(() => service.restoreOperatorFill(paper, { operatorID: "missing-op" } as OperatorPredicate)).not.toThrow(); + }); + }); + describe("changeOperatorViewResultStatus", () => { it("writes the view-result asset path when viewResult is true", () => { const { paper, attrSpy } = makePaperWithModel(); diff --git a/frontend/src/app/workspace/service/joint-ui/joint-ui.service.ts b/frontend/src/app/workspace/service/joint-ui/joint-ui.service.ts index bb6fbff0f0..3afbccde87 100644 --- a/frontend/src/app/workspace/service/joint-ui/joint-ui.service.ts +++ b/frontend/src/app/workspace/service/joint-ui/joint-ui.service.ts @@ -26,6 +26,7 @@ import * as joint from "jointjs"; import { fromEventPattern, Observable } from "rxjs"; import { Coeditor } from "../../../common/type/user"; import { OperatorResultCacheStatus } from "../../types/workflow-websocket.interface"; +import { HEATMAP_NO_DATA_COLOR, scoreToColor } from "../heatmap/heatmap-color"; /** * Defines the SVG path for the delete button @@ -515,6 +516,26 @@ export class JointUIService { jointPaper.getModelById(operator.operatorID).attr("rect.body/fill", JointUIService.getOperatorFillColor(operator)); } + /** + * Paints an operator's body fill for the performance heat-map overlay. The heat-map owns only + * `rect.body/fill`, so it coexists with the execution-status border (`rect.body/stroke`). + * The method paints a neutral color when `score` is undefined, which means no heat is known + * for the operator — either no metrics captured yet, or the active view is not measurable + * for it. + */ + public applyHeatmapColor(jointPaper: joint.dia.Paper, operatorID: string, score: number | undefined): void { + const fill = score === undefined ? HEATMAP_NO_DATA_COLOR : scoreToColor(score); + jointPaper.getModelById(operatorID)?.attr("rect.body/fill", fill); + } + + /** + * Restores an operator's default body fill (used when the heat-map overlay is turned off), + * reusing the same source as the normal enabled/disabled coloring. + */ + public restoreOperatorFill(jointPaper: joint.dia.Paper, operator: OperatorPredicate): void { + jointPaper.getModelById(operator.operatorID)?.attr("rect.body/fill", JointUIService.getOperatorFillColor(operator)); + } + public changeOperatorViewResultStatus( jointPaper: joint.dia.Paper, operator: OperatorPredicate, diff --git a/frontend/src/app/workspace/service/workflow-graph/model/joint-graph-wrapper.ts b/frontend/src/app/workspace/service/workflow-graph/model/joint-graph-wrapper.ts index b7a2bf05b5..66ddc42cdd 100644 --- a/frontend/src/app/workspace/service/workflow-graph/model/joint-graph-wrapper.ts +++ b/frontend/src/app/workspace/service/workflow-graph/model/joint-graph-wrapper.ts @@ -26,6 +26,7 @@ import * as graphlib from "graphlib"; import { ObservableContextManager } from "src/app/common/util/context"; import { Coeditor, User } from "../../../../common/type/user"; import { operatorCoeditorChangedPropertyClass, operatorCoeditorEditingClass } from "../../joint-ui/joint-ui.service"; +import { HeatmapView } from "../../heatmap/heatmap-scoring"; import { dia } from "jointjs/types/joint"; import * as _ from "lodash"; import Selectors = dia.Cell.Selectors; @@ -108,6 +109,11 @@ export class JointGraphWrapper { // reapply it to the shared model (covering both the main canvas and the mini-map). private regionsDisplayedStream = new BehaviorSubject<boolean>(false); + // The active performance heat-map view, or null when the overlay is off (Layers > Performance). + // Kept here so the editor can (re)apply operator colors on the shared model, covering both the + // main canvas and the mini-map. + private heatmapViewStream = new BehaviorSubject<HeatmapView | null>(null); + private elementPositions: Map<string, PositionInfo> = new Map<string, PositionInfo>(); private listenPositionChange: boolean = true; @@ -229,6 +235,21 @@ export class JointGraphWrapper { return this.regionsDisplayedStream.asObservable(); } + /** + * Sets the active performance heat-map view, or null to turn the overlay off. + */ + public setHeatmapView(view: HeatmapView | null): void { + this.heatmapViewStream.next(view); + } + + public getHeatmapView(): HeatmapView | null { + return this.heatmapViewStream.value; + } + + public getHeatmapViewStream(): Observable<HeatmapView | null> { + return this.heatmapViewStream.asObservable(); + } + /** * This method is used to toggle the multiselect mode. * @param multiSelect
