tkobayas opened a new issue, #4018:
URL: https://github.com/apache/incubator-kie-tools/issues/4018
### Description
During CI runs on `ubuntu-latest` (workflow run `94969099687`), we observed
intermittent and non-deterministic Playwright E2E test failures across multiple
consecutive retries. On each run, the build, TypeScript compilation, and unit
tests passed cleanly, but different downstream E2E test suites failed with
distinct flakiness patterns.
### Environment
- **OS**: Ubuntu (GitHub Actions runner / Linux Container)
- **Node**: v20 / v24
- **Test Framework**: Playwright (via containerized runner)
- **Browsers**: WebKit, Chromium, Google Chrome
---
### Summary of Observed Failures Across Consecutive Runs
| Run # | Package | Failed Spec / Test Case | Browser | Failure Mode |
| :--- | :--- | :--- | :--- | :--- |
| **Initial** | `@kie-tools/dmn-editor` |
`tests-e2e/drgElements/addBkm.spec.ts`<br>`tests-e2e/drgElements/addDecision.spec.ts`<br>`tests-e2e/drgRequirements/moveAuthorityRequirementWaypoint.spec.ts`
| WebKit | Drag-and-drop / node interaction timeout & assertions |
| **Retry 1** | `@kie-tools/dmn-editor` |
`tests-e2e/drgElements/modelDecisionService.spec.ts` | WebKit | Signature input
reordering assertion |
| **Retry 2** | `@kie-tools/boxed-expression-component` |
`tests-e2e/boxedExpressions/invocation/populate.spec.ts` | Chromium |
`toHaveScreenshot` failed (541px / ~1% pixel diff) |
| **Retry 3** | `@kie-tools/dmn-editor-standalone` |
`tests-e2e/api/getContent.spec.ts` | Chrome | XML `xmlns` attribute
serialization ordering mismatch |
| **Retry 4** | `@kie-tools/dmn-editor` |
`tests-e2e/drgRequirements/moveAuthorityRequirementWaypoint.spec.ts` | WebKit |
`toHaveScreenshot` failed (1081px / ~1% pixel diff) |
---
### Root Cause Categories
1. **Visual Regression / Screenshot Flakiness (`toHaveScreenshot`)**:
- `boxed-expression-component`
(`boxed-invocation-affordability-calculation.png` on Chromium) and `dmn-editor`
(`move-authority-requirement-ending-waypoint.png` on WebKit) failed due to ~1%
pixel differences.
- Likely caused by minor font antialiasing / subpixel rendering
differences or timing before the container screenshot is taken.
2. **WebKit Drag-and-Drop / Interaction Timing in Linux Containers**:
- WebKit in headless containerized Linux shows intermittent drag/drop and
waypoint move action instability in `dmn-editor`.
3. **XML Attribute Ordering Assertion in Standalone DMN Editor**:
- `getContent.spec.ts` in `dmn-editor-standalone` does a string equality
check (`toEqual`) between `editor.getFormattedContent()` and a static file
fixture.
- The ordering of `xmlns:*` attributes in the serialized XML root element
is non-deterministic, causing string comparison failures even when the XML is
semantically identical.
---
### Suggested Fixes / Improvements
- [ ] **Visual Testing**: Adjust `maxDiffPixelRatio` / `threshold` tolerance
for `toHaveScreenshot` assertions, or ensure stable font rendering /
animation-disable in test fixtures.
- [ ] **XML Comparison**: Parse XML to DOM/AST or compare semantically
(canonical XML / attribute-agnostic comparison) in `getContent.spec.ts` rather
than raw string comparison.
- [ ] **WebKit Stability**: Investigate whether additional wait states /
element stability checks are needed before drag-and-drop operations in
`dmn-editor` fixtures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
