This is an automated email from the ASF dual-hosted git repository. jungm pushed a commit to branch ee11 in repository https://gitbox.apache.org/repos/asf/tomee-tck.git
commit 848420b6610444fa5dc22dfb38e7697a6e90d8cf Author: Markus Jung <[email protected]> AuthorDate: Sun Jul 19 10:53:10 2026 +0200 Document the security-old baseline and its one product-gap exclusion --- KNOWN_ISSUES.md | 37 +++++++++++++++++++-------- runner-standalone/README.md | 10 +++++--- runner-standalone/exclusions/README.md | 6 ++--- runner-standalone/exclusions/security-old.txt | 18 +++++++++++++ 4 files changed, 53 insertions(+), 18 deletions(-) diff --git a/KNOWN_ISSUES.md b/KNOWN_ISSUES.md index f0e2d51..2f120fe 100644 --- a/KNOWN_ISSUES.md +++ b/KNOWN_ISSUES.md @@ -38,6 +38,7 @@ Detail lives next to each runner: | jsonb | 295 tests, 1 F + 1 E | 2 tests | 2 Johnzon 2.1.0 gaps | | debugging | passes (4 SMAPs validated) | — | — | | security | 132 tests, 5 F + 2 E; signature test passes | 7 tests | TomEE Jakarta Security | +| security-old (JavaTest) | 83 tests, 82 pass, 1 F | 1 test | TomEE `SecurityContext.hasAccessToWebResource()` returns false for an authorized caller | | authentication | 106 tests, 1 F; signature test passes | 1 method (spi `CheckMsgInfoKey`) | TCK challenge #219 (hard-codes a JACC requirement) | | websocket | 737 tests, 3 E | 3 methods | Client container advertises permessage-deflate in the negotiated extension lists | | faces (modern modules) | 327 tests, 45 F + 6 E (failsafe reruns inflate the counts; 10 distinct failing classes) | 10 classes | 1 TomEE faces-config gap + Mojarra/TomEE CDI-injection, whole-bean/method validation, and one Chrome ajax quirk | @@ -114,20 +115,30 @@ Fixes belong in Apache TomEE (or Tomcat); each removes exclusion entries. 5. **Jakarta Security** — the BASIC mechanism answers 401 for valid credentials in the decorated/custom-handler variants, and both OpenID default modules fail token validation. - [security.txt](runner-standalone/exclusions/security.txt). The runner drives - the reactor's modern Arquillian app modules plus the signature test; the - reactor's `old-tck` module (~68 legacy JavaTest tests) is not wired into - the runner, so the recorded totals cover the modern suite only. -6. **Persistence integration** — undeploy calls `close()` on an + [security.txt](runner-standalone/exclusions/security.txt). The `security` + runner drives the reactor's modern Arquillian app modules plus the signature + test; the reactor's `old-tck` module (legacy JavaTest suite, + `com.sun.ts.tests.securityapi`) is run by the separate `security-old` runner. +6. **Jakarta Security `SecurityContext.hasAccessToWebResource()`** — the + programmatic access check returns `false` for a caller that is authorized + for the resource. The old-tck `securitycontext/callerdata` servlet reports + the correct caller and role membership, but `hasAccessToWebResource( + "/protectedServlet", "GET")` answers `false` where the spec requires `true` + for user `tom` (Manager role) against the `@HttpMethodConstraint("GET")` + resource; TomEE's `SecurityContext` is not wired to the servlet + authorization/`Policy` layer for this call. Excluded in + [security-old.txt](runner-standalone/exclusions/security-old.txt) + (`securitycontext/callerdata/Client.java#testSecurityContextHasAccessToWebResource`). +7. **Persistence integration** — undeploy calls `close()` on an already-closed `EntityManagerFactory` (fails the `entityManagerFactoryCloseExceptions` vehicles), and the Jakarta Persistence 3.2 CDI qualifier beans (`EntityManagerFactory`/ `EntityManager` etc. from `persistence.xml`) are not registered (`ServletEMLookupTest`). Affects Plume and webprofile alike. -7. **Jakarta Tags TLD registration** — the `jakarta.tags.*` URIs of the +8. **Jakarta Tags TLD registration** — the `jakarta.tags.*` URIs of the replacement Jakarta Tags 3.0 jar are not exposed to applications; all 50 Tags classes plus the EJB-Lite JSP vehicles fail as collateral. -8. **Transactions — cross-request `UserTransaction` state leakage across +9. **Transactions — cross-request `UserTransaction` state leakage across pooled servlet requests.** A `UserTransaction` a servlet/jsp request leaves in a non-clean state poisons the next request served on the same pooled Tomcat exec thread; the victim sees an `IllegalStateException` that is not @@ -148,14 +159,14 @@ Fixes belong in Apache TomEE (or Tomcat); each removes exclusion entries. [transactions.txt](runner-standalone/exclusions/transactions.txt). The Platform catalog additionally shows CDI `@Transactional` interceptors failing propagation, rollback-rule, and `TransactionScoped` assertions. -9. **Enterprise Beans** — timer callbacks expose incomplete/not-retried +10. **Enterprise Beans** — timer callbacks expose incomplete/not-retried transactions, `java:comp` is mutable where the spec requires `OperationNotSupportedException`, and failed CDI/EJB deployments leak deployment IDs (`DuplicateDeploymentIdException` in later apps). -10. **webprofile ZIP signature leak** — the combined `jakartaee-api` jar +11. **webprofile ZIP signature leak** — the combined `jakartaee-api` jar exposes Jakarta Batch and Messaging packages although `javaee.level=web` does not declare them; strip them or declare and certify them. -11. **WebSocket 2.2 extension advertising (Tomcat)** — the server-side +12. **WebSocket 2.2 extension advertising (Tomcat)** — the server-side configurator reports the extensions the client requested and negotiated. TomEE's client-side WebSocket container (Tomcat's `tomcat-websocket`) always advertises its built-in `permessage-deflate` extension in the @@ -168,7 +179,7 @@ Fixes belong in Apache TomEE (or Tomcat); each removes exclusion entries. and pass: the runner's Arquillian extension tolerates the deployment failure so each client probe still runs. [websocket.txt](runner-standalone/exclusions/websocket.txt). -12. **Bean Validation XML config broken on stock Plume** — the Plume +13. **Bean Validation XML config broken on stock Plume** — the Plume distribution ships EclipseLink MOXy (`eclipselink-5.0.1.jar`) and the JAXB RI (`jaxb-runtime-4.0.4.jar`) side by side. EclipseLink registers a `jakarta.xml.bind.JAXBContextFactory` service and wins ServiceLoader @@ -256,6 +267,10 @@ Not product bugs — gaps in this repository's coverage. ships an `old-tck-selenium` module (22 tests) that neither the `faces` runner (which drives faces22/23/40/41 + faces-signaturetest) nor the `faces-old` JavaTest runner executes; wire it into one of them. +- **Security `security-old` not in CI**: the `security-old` runner (the + reactor's legacy JavaTest `com.sun.ts.tests.securityapi` suite) runs + locally via `run-standalone-suite.sh` and is green with the reviewed + exclusion; it joins the Jenkins branch list once its baseline has settled. ## What CI runs diff --git a/runner-standalone/README.md b/runner-standalone/README.md index 79f7fd7..243b163 100644 --- a/runner-standalone/README.md +++ b/runner-standalone/README.md @@ -13,10 +13,11 @@ runner-standalone/run-standalone-suite.sh concurrency The Derby port is overridable everywhere through `-Dtck.derby.port`; the `concurrency`, `data`, `servlet`, `pages`, `rest`, `websocket`, -`transactions`, and `faces-old` runners additionally accept +`transactions`, `faces-old`, and `security-old` runners additionally accept `-Dtomee.http.port`, `-Dtomee.https.port`, and `-Dtomee.shutdown.port` -(`transactions` and `faces-old` also `-Dtck.harness.log.port` for their -JavaTest listeners), so they can run next to another harness instance. The +(`transactions`, `faces-old`, and `security-old` also `-Dtck.harness.log.port` +for their JavaTest listeners), so they can run next to another harness +instance. The `validation`, `cdi`, and `cdi-ee` runners parameterize only the HTTP and shutdown ports (`-Dtomee.http.port`/`-Dtomee.shutdown.port`); their arquillian.xml pins no `httpsPort`. The source-reactor runners security, @@ -59,7 +60,8 @@ derived from; with the default exclusions applied these suites run green. | Pages 4.0 | EFTL zip (installed as `jakarta.tck:jakarta-pages-tck:4.0.0`) | `pages` | **Passes 682/682 (0 failures, 0 errors, signature test included, no exclusions)** (2026-07-18). Needs the Central `jakarta.tck:common`/`signaturetest` support line at 11.1.1 with sigtest 2.6 (the TCK pom's own pins depend on the unpublished JavaTest harness), and the overlay removes TomEE's global UTF-8 default encodings from conf/web.xml because the TCK asserts the spec default ISO-8859-1. Its TomEE/ [...] | Validation 3.1 | EFTL zip (installed as `jakarta.validation:validation-tck-tests:3.1.1`) | `validation` | **Runs: 1,049 tests, 0 failures, no exclusions; the mandatory API signature test runs and passes** against the `jakarta.validation` packages the Plume distribution ships. Needs AssertJ 3.7.0 pinned (the published TCK jar is compiled against its covariant signatures) and the JAXB RI pinned as `jakarta.xml.bind.JAXBContextFactory` in the server JVM (Plume ships both MOXy and the RI; [...] | WebSocket 2.2 | EFTL zip (installed as `jakarta.tck:websocket-tck-spec-tests:2.2.0`) | `websocket` | **Runs: 737 tests, 0 failures, 3 errors** (2026-07-18) against Tomcat's WebSocket implementation, driven through the `tomcat-websocket` client container; the signature test passes. The negative-deployment classes run and pass: an invalid server endpoint aborts the whole webapp deployment as the spec requires, and the runner's Arquillian extension (`NegativeDeploymentToleranceObserver`) [...] -| Security 4.0 | Source reactor zip 4.0.1 | `security` (Maven module) | **Runs: 26 app modules plus the signature test, 132 tests, 5 failures, 2 errors at baseline; green with the reviewed exclusions (all 27 invoker projects pass, verified end-to-end 2026-07-18)**. The runner downloads and patches the reactor, injects a tomee-remote profile (including the `trustStore.path`/`trustStore.password` properties the OpenID modules' keytool steps read), and drives every module through the Maven [...] +| Security 4.0 | Source reactor zip 4.0.1 | `security` (Maven module) | **Runs: 26 app modules plus the signature test, 132 tests, 5 failures, 2 errors at baseline; green with the reviewed exclusions (all 27 invoker projects pass, verified end-to-end 2026-07-18)**. The runner downloads and patches the reactor, injects a tomee-remote profile (including the `trustStore.path`/`trustStore.password` properties the OpenID modules' keytool steps read), and drives every module through the Maven [...] +| Security 4.0 (legacy old-tck) | Source reactor zip 4.0.1 (old-tck built from bundled sources) | `security-old` (Maven module) | **Baseline 83 tests, 82 pass, 1 failure (Plume, JDK 21, 2026-07-19); green with the reviewed exclusion.** The legacy JavaTest half exercises ~65 `com.sun.ts.tests.securityapi` Client classes across the idstore (in-war, database, LDAP), ham (HTTP authentication mechanism) and securitycontext trees. It deploys through the TCK's own `tomcat` handler (WAR copy int [...] | Authentication 3.1 | Source reactor zip 3.1.2 | `authentication` (Maven module) | **Runs: 12 Web Profile modules plus the signature test; green with the reviewed exclusions (all 13 invoker projects pass, 0 failures, verified end-to-end 2026-07-18)**. The spi module registers the TCK's test `AuthConfigProvider` under Tomcat's JASPIC app-context naming (`Catalina/localhost /spitests_servlet_web`), so `ServletProfileSPITest` runs against Tomcat's `AuthConfigFactory` and passes 56 of its 5 [...] | Faces 4.1 | Source reactor zip 4.1.2 | `faces` (Maven module) | **Runs: modern Arquillian modules (faces22/23/40/41) — 327 tests, 45 failures + 6 errors** (2026-07-18) on Plume's Mojarra; the failsafe rerun mechanism inflates those totals, covering 10 distinct failing classes, all in `exclusions/faces.txt`. One is a TomEE product gap (faces-config.xml unmarshaller rejecting the nonstandard *https* `xsi:schemaLocation`, `Spec1760IT`); the rest are Mojarra/TomEE integration behaviors — F [...] | Faces 4.1 (legacy old-tck) | Source reactor zip 4.1.2 (old-tck built from bundled sources) | `faces-old` (Maven module) | **Passes 5,391/5,391 (standalone mode, no exclusions)** (2026-07-18). The recorded full run shows 5 failures in `htmloutcometargetbutton`, all from a foreign server answering port 8080 mid-run; they pass on re-run against TomEE. The JavaTest half deploys through the TCK's own `tomcat` handler (WAR copy into `webapps/`) against a TomEE instance the runner provisions [...] diff --git a/runner-standalone/exclusions/README.md b/runner-standalone/exclusions/README.md index f71ee3e..64644cc 100644 --- a/runner-standalone/exclusions/README.md +++ b/runner-standalone/exclusions/README.md @@ -24,9 +24,9 @@ Two formats, depending on how the runner executes tests: both the declaring class and the concrete runtime class — several TCK base classes contribute inherited test methods that only the subclass name identifies. -- **JavaTest-based runners** (`transactions`, `faces-old`): JavaTest jtx - exclusion lines (`test/path/File.java#testid`), appended by the runner to - the harness `ts.jtx` before the run. +- **JavaTest-based runners** (`transactions`, `faces-old`, `security-old`): + JavaTest jtx exclusion lines (`test/path/File.java#testid`), appended by the + runner to the harness `ts.jtx` before the run. To collect a full compatibility baseline without exclusions, run a suite with `-Dtck.exclusions.file=$(pwd)/runner-standalone/exclusions/none.txt` (the diff --git a/runner-standalone/exclusions/security-old.txt b/runner-standalone/exclusions/security-old.txt new file mode 100644 index 0000000..50bb8a6 --- /dev/null +++ b/runner-standalone/exclusions/security-old.txt @@ -0,0 +1,18 @@ +# Reviewed known-gap exclusions for the legacy Jakarta Security 4.0 old-tck +# (JavaTest) run against TomEE. Lines use the JavaTest jtx format +# (test/dir/path#testid) and are appended to the mode's ts.jtx by the +# runner-standalone-security-old module. See KNOWN_ISSUES.md for the +# rationale behind each entry. +# +# Baseline 2026-07-19 (Plume, JDK 21): 83 tests, 82 passed, 1 failed, +# 0 errors. The single failure below is a TomEE product gap. + +# TomEE product gap: SecurityContext.hasAccessToWebResource() returns false +# for a caller that is authorized for the resource. The servlet reports the +# correct caller ("context username: tom") and the authentication path works, +# but the programmatic access check answers "has GET access ... : false" where +# the spec requires true for user tom (Manager role) against the +# @HttpMethodConstraint("GET") /protectedServlet. TomEE's SecurityContext is +# not wired to the servlet authorization/Policy layer for +# hasAccessToWebResource(). Assertions Security:JAVADOC:7/8, Security:SPEC:4.3-1. +com/sun/ts/tests/securityapi/securitycontext/callerdata/Client.java#testSecurityContextHasAccessToWebResource
