The GitHub Actions job "Required Checks" on texera.git/main has failed. Run started by GitHub user github-merge-queue[bot] (triggered by github-merge-queue[bot]).
Head commit for run: f9cfaceed3b6dfff6a07929443eba840422e2ea5 / Yicong Huang <[email protected]> test(workflow-core): add @NonParallelTest to de-flake shared-MinIO suites (#7051) ### What changes were proposed in this PR? De-flake `S3StorageClientSpec`, which intermittently fails **4 tests** in the `build / amber` CI job (scope `WorkflowCore / Test / test`). **Root cause.** Four suites mix `S3StorageTestBase` and share a JVM-wide `S3StorageClient.s3Client` plus a single `StorageConfig.s3Endpoint`, all pointed at one shared MinIO container: `S3StorageClientSpec`, `LargeBinaryManagerSpec`, `LargeBinaryInputStreamSpec`, `LargeBinaryOutputStreamSpec`. ScalaTest's parallel suite distributor runs them concurrently within a single in-process test task, so they contend for that one container and intermittently time out. The existing `Global / concurrentRestrictions += Tags.limit(Tags.Test, 1)` only bounds sbt **task** concurrency, not ScalaTest's in-JVM distributor (CI logs show `pool-N-thread-M-ScalaTest-running-...Spec`). **Fix — a reusable `@NonParallelTest` tag.** - New `org.apache.texera.common.tags.NonParallelTest` — a Java `@TagAnnotation` mirroring amber's `IntegrationTest`. Suites carrying it must not run concurrently with one another. - Tag the four MinIO-backed suites. - `common/workflow-core/build.sbt`: `Test / testGrouping` gives each tagged suite its own forked JVM. sbt runs forked groups one at a time (`Tags.ForkedTestGroup` limit), so tagged suites never overlap; every untagged suite stays in a single in-process group and keeps running in parallel. The forked working directory is pinned to the repo root to match the in-process cwd. - This mirrors the FileService approach for testcontainer suites, but keyed on an **opt-in annotation** instead of a hard-coded name pattern, so future container-backed suites just add `@NonParallelTest`. - Detection note: sbt discovers ScalaTest suites by subclass fingerprint, not by annotation, so the tag is not visible in `definedTests`; it is detected reflectively through the test class loader (`loadClass` does not run static initializers; a load failure falls back to the parallel group). - Also bound the concurrent-upload burst in the *>1000 objects* test from 16 to 4 threads, so a single suite no longer floods the container. **Why not `@IntegrationTest`?** That tag/lane lives only in the `amber` module (`amber/src/test/integration`). `amber` depends on `workflow-core`, so a workflow-core suite cannot reference it. The established convention for common-module container-backed tests (e.g. file-service's LakeFS/MinIO suites) is to keep them in normal `Test` scope and isolate them via forked JVMs — which is what this PR does. ### Any related issues, documentation, discussions? Closes #7049 ### How was this PR tested? - `sbt reload` loads the build cleanly and type-checks the reflective `testGrouping` (the `testLoader`/`Tests.InProcess`/`Tests.SubProcess` usage); `WorkflowCore/Test/forkOptions` confirms the forked working directory resolves to the repo root. - Full test compile + a live `testGrouping` run require the JOOQ codegen prerequisite (a database) and Docker (MinIO), neither available in the authoring environment, so the runtime grouping and de-flaking are validated by CI. Since the failure is flaky, reviewers may want to re-run the `amber` job a couple of times to confirm stability. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8 [1M context]) Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/30498942090 With regards, GitHub Actions via GitBox
