Aman-Mittal opened a new pull request, #550: URL: https://github.com/apache/fineract-backoffice-ui/pull/550
## What changed `scripts/e2e-summary.mjs` now reconciles what a test *did* with what it was *asked* to do, instead of reading `result.status` alone. - an asserted failure (`test.fail()`) no longer counts toward the failure total - it gets its own **๐ Known failures** section and a column in the per-file table - a `test.fail()` case that has started passing is now reported as a failure ## Why A `test.fail()` case documents a known bug: Playwright runs it, requires it to fail, and exits zero when it does. The summary counted it as a failure anyway, so a completely green run rendered as: > โ 1 failing โ 358 passed ยท 1 failed ยท 1 skipped while all 31 checks, every E2E shard included, were green. Seen on #549. A report that cries wolf on a passing suite is one people stop reading, which defeats the point of the summary existing. The inverse matters as much. When such a test starts passing the marker is stale and Playwright fails the run โ the report has to agree with the exit code rather than quietly counting it as a pass. Known failures are kept visible rather than folded into green: each one is a bug someone chose to document instead of fix, and an unnamed list of them is how a `test.fail()` becomes permanent. The per-file table gets its own column for the same reason โ a known failure is neither a broken run nor a test that did not run, and the counts should still add up. ## Testing Verified against a **real** Playwright JSON report containing an expected failure, produced by running the #549 spec locally, rather than a synthetic fixture. The report confirms the shape the fix keys on: `expectedStatus: "failed"`, `results[].status: "failed"`, run exit code 0. Before โ after on that report: ``` โ 1 failing โ 0 passed ยท 1 failed ยท 0 skipped โ All green โ 1 passed ยท 0 failed ยท 0 skipped ยท 1 known-failing ``` With the same report's result flipped to `passed` โ the stale-marker case โ it correctly reads `โ 1 failing`. Also run: `npm run test:scripts` (31 passing, including three new cases for `classifyStatus`), `npm run lint`, `prettier --check`. Found while landing #549, which adds the repository's first `test.fail()` case. Kept separate from that PR rather than folded in, since this is tooling that any future expected-failure test would hit. Related: #549, #548 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
