Rangsh commented on PR #12173: URL: https://github.com/apache/seatunnel/pull/12173#issuecomment-5610606054
### Normal Benchmarks before/after (no profiler) Following @nzw921rx — these are from the normal **Benchmarks** workflow (`benchmarks.yml`), not Diagnostics. | | run | ref / commit | |---|---|---| | **before** | https://github.com/Rangsh/seatunnel/actions/runs/34354309911 | `dev` @ `75fd4ed` | | **after** | https://github.com/Rangsh/seatunnel/actions/runs/34354315647 | PR @ `c0d59da` | | selector | `IMapJobStorageBenchmark.(runningJobGrowth\|completedJobHistoryGrowth)$` | | #### Java 11 Score / Error / CV | Benchmark | Count | Before Score | Before Err% | Before CV% | After Score | After Err% | After CV% | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | `runningJobGrowth` | 0 | 301.294 | 16.32% | 15.27% | 435.115 | 8.76% | 8.19% | | `runningJobGrowth` | 1000 | 263.751 | 13.04% | 12.19% | 369.463 | 20.45% | 19.13% | | `completedJobHistoryGrowth` | 0 | 1,870.43 | 9.30% | 8.70% | 2,102.04 | 9.55% | 8.94% | | `completedJobHistoryGrowth` | 1000 | 1,715.04 | 4.43% | 4.14% | 1,903.08 | 5.35% | 5.01% | #### Caveat (important) These two fork dispatches landed on **different GitHub-hosted CPUs** (`AMD EPYC 9V74` before vs `AMD EPYC 7763` after), so absolute Score is not comparable across the two runs, and CV deltas are weakened by host noise. The workflow report itself labels this as observational. What this still shows on the **after** head alone (same JVM settings / no profiler): - `completedJobHistoryGrowth` CV stays in a low single-digit / ~9% band - `runningJobGrowth` @0 CV improved vs this before run (15.27% → 8.19%) - `runningJobGrowth` @1000 is still noisy on this after host (19.13%) For a same-worker comparison that the docs recommend, please run **Benchmarks** on `apache/seatunnel` with `pr_number=12173` (`baseline -> PR -> PR -> baseline`). I cannot dispatch there (403). Diagnostics wall stacks remain causal-only evidence (fixture `FileMapStore.loadAll` / WAL reload removed from the measured-adjacent path) and are not used for Score/CV comparison. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
