Hi, First of all: It is really appreciated, that you jumped into exploring the TCK setup :)
I think, it might depend on the way you are running those tests. Typically, it it not sufficient to just run the *.sh file of https://github.com/apache/tomee-tck/ (at least from my experience). On my German localized system, I wasn’t able to get some tests working, which were fine under the https://tck.work/tomee/projects environment. A typical thing for some JAX-RS tests with plume looked like: rm -rf target/ && ./runtests -Dhttps.protocols=TLSv1.1,TLSv1.2 --ee91 -j --env -nc -c -U -ol -w tomee-plume com.sun.ts.tests.jaxrs.api.rs.ext.interceptor.reader.readerinterceptorcontext.JAXRSClient \ com.sun.ts.tests.jaxrs.ee.rs.core.responsebuilder.JAXRSClient \ com.sun.ts.tests.jaxrs.ee.rs.ext.interceptor.containerreader.readerinterceptorcontext.JAXRSClient \ com.sun.ts.tests.jaxrs.ee.rs.pathparam.locator.JAXRSLocatorClient \ com.sun.ts.tests.jaxrs.spec.client.typedentities.JAXRSClient \ com.sun.ts.tests.jaxrs.spec.context.server.JAXRSClient \ com.sun.ts.tests.jaxrs.spec.filter.interceptor.JAXRSClient \ com.sun.ts.tests.jaxrs.spec.filter.lastvalue.JAXRSClient \ com.sun.ts.tests.jaxrs.spec.provider.reader.JAXRSClient \ com.sun.ts.tests.jaxrs.spec.provider.standardnotnull.JAXRSClient \ com.sun.ts.tests.jaxrs.spec.resource.requestmatching.JAXRSClient \ com.sun.ts.tests.jaxrs.spec.resourceconstructor.JAXRSClient I guess that we might need to update the „—ee91“ flag to respect EE10, but didn’t test it myself at the moment. Maybe Markus has some more insights as (I think) he was able to run some EE10 TCK test lately. The POM (spec versions) and README might need an update too, so more people can have the possibility to set things up. To try to your questions: (1) Share your actual run configuration. How do you start the tests? Which tests? For a first try, maybe focus on one single test first. (2) Tomitribe provided tck.work to run the TCK on AWS. I don’t think, that the current setting is runnable. Maybe JL or Jon have insights here? Are we able to do a run with a M3-SNAPSHOT? (3) I would suggest to use a gist to share the logs of a single TCK test. From a quick search in the TCK setup, it seems, that openejb.xml uses javax.jms instead of jakarta.jms (and we are on amq6, so should be jakarta, I guess - maybe try a search / replace for it?). Might be worth a try, if the connection factory and related services come up afterwards? Hope it helps. Richard > Am 05.08.2024 um 14:14 schrieb Paulo Araújo <cris...@vikeo.com.br.INVALID>: > > I've began to run the tck-tests on tomee-plume and it seems really bad, so > I've switched to test tomee-web and still waiting results. It seems there is > a problem with JMS configuration on tomee test project (all tests fail after > 10 retries, each one timing-out). > > 1) Can someone with some experience guide me how to fix such issues on tests? > > 2) Is there any automated environment where these tests take place and are > reported and, if so, how can I access them? > > 3) How can I share the log results (~200k/test group) in the list? > > > > Thanks in advance, > > Paulo > > > > > > > > > > ---- Em qui, 25 jul 2024 11:12:31 -0300 Richard Zowalla <rich...@zowalla.com> > escreveu --- > > > > Standalone is here: https://github.com/apache/tomee/tree/main/tck > > As long as it is arquillian/junit/testng based and can be executed via Maven. > For some stuff, Jiras may exist. > > For the tomee-tck itself you don't need to create a Jira imho. We may need > them for standalone and fixing tests. > > Gruß > Richard > > Am 25. Juli 2024 16:05:53 MESZ schrieb "Paulo Araújo" > <mailto:cris...@vikeo.com.br.INVALID>: >> Hi, >> >> >> >> Considering the report of glassfish-7 web profile certification, tomee-10 >> should pass the following groups of tests: >> >> appclient >> >> assembly >> >> connector >> >> ejb >> >> ejb30.assembly >> >> ejb30.bb >> >> ejb30.lite.appexception >> >> ejb30.lite.async >> >> ejb30.lite.basic >> >> ejb30.lite.ejbcontext >> >> ejb30.lite.enventry >> >> ejb30.lite.interceptor >> >> ejb30.lite.lookup >> >> ejb30.lite.naming >> >> ejb30.lite.nointerface >> >> ejb30.lite.packaging >> >> ejb30.lite.singleton >> >> ejb30.lite.stateful >> >> ejb30.lite.tx >> >> ejb30.lite.view >> >> ejb30.lite.xmloverride >> >> ejb30.misc >> >> ejb30.sec >> >> ejb30.timer >> >> ejb30.webservice >> >> ejb30.zombie >> >> ejb32 >> >> el >> >> integration >> >> jacc >> >> javaee >> >> javamail >> >> jaxrs >> >> jaxrs >> >> jdbc.ee.batchUpdate >> >> jdbc.ee.callStmt >> >> jdbc.ee.connection >> >> jdbc.ee.dateTime >> >> jdbc.ee.dbMeta >> >> jdbc.ee.escapeSyntax >> >> jdbc.ee.exception >> >> jdbc.ee.prepStmt >> >> jdbc.ee.resultSet >> >> jdbc.ee.rsMeta >> >> jdbc.ee.stmt >> >> jms >> >> jsp >> >> jstl >> >> jta >> >> samples >> >> servlet >> >> signaturetest >> >> webservices12 >> >> webservices13 >> >> websocket >> >> xa >> >> >> >> I've updated the webprofile.sh of tomee-tck project to reflect these current >> required tests (pull request of changes will be submitted later, if my >> assumptions here are confirmed). >> >> >> >> As I understand from glassfish results, jsf tests were dropped from TCK-10 >> in favor of individual Faces TCK[1], situation in which are also concurrency >> [2], json-p [3], json-b [4], jax-rs [5], security [6], authentication [7], >> CDI [8], validation [9], dependency injection [10] and debugging support for >> other languages [11]. >> >> >> >> Previous web profiile certification of tomee [12] had already used this >> model concerting DI, CDI, validation, XML binding and debugging. >> >> The new specs to be covered individually where added by the jakartaee-10 >> webprofile spec [13]. >> >> >> >> Does the tomme-tck project is also responsible for running or generating >> tests results for the individual project? Do I need to open a new Jira Issue >> to submit the proposed changes to tomee-tck project? >> >> >> >> Best regards, >> >> Paulo >> >> >> >> >> >> [1] >> https://download.eclipse.org/jakartaee/faces/4.0/jakarta-faces-tck-4.0.1.zip >> >> [2] >> https://download.eclipse.org/jakartaee/concurrency/3.0/concurrency-tck-3.0.2.zip >> >> >> [3] >> https://download.eclipse.org/jakartaee/jsonp/2.1/jakarta-jsonp-tck-2.1.0.zip >> >> [4] >> https://download.eclipse.org/jakartaee/jsonp/2.1/jakarta-jsonp-tck-2.1.0.zip >> >> [5] >> https://download.eclipse.org/jakartaee/restful-ws/3.1/jakarta-restful-ws-tck-3.1.0.zip >> >> >> [6] >> https://download.eclipse.org/jakartaee/security/3.0/jakarta-security-tck-3.0.0.zip >> >> >> [7] >> https://download.eclipse.org/jakartaee/authentication/3.0/jakarta-authentication-tck-3.0.2.zip >> >> >> [8] https://download.eclipse.org/jakartaee/cdi/4.0/cdi-tck-4.0.5-dist.zip >> >> [9] >> https://download.eclipse.org/jakartaee/bean-validation/3.0/beanvalidation-tck-dist-3.0.1.zip >> >> >> [10] >> https://download.eclipse.org/jakartaee/dependency-injection/2.0/jakarta.inject-tck-2.0.2-bin.zip >> >> >> [11] >> https://download.eclipse.org/jakartaee/debugging/2.0/jakarta-debugging-tck-2.0.0.zip >> >> >> [12] https://tomee.apache.org/9.0.0-M7/plume/webprofile-9.1.html >> >> [13] >> https://jakarta.ee/specifications/webprofile/10/jakarta-webprofile-spec-10.0#a43 >> >> >> >> >> >> >> ---- Em qui, 25 jul 2024 02:38:32 -0300 Richard Zowalla >> <mailto:r...@apache.org> escreveu --- >> >> >> >> Hi David, >> >> Thanks for clarification! >> >> Gruß >> Richard >> >>> Am 25.07.2024 um 00:39 schrieb David Blevins >>> <mailto:mailto:david.blev...@gmail.com>: >>> >>> Hey All, >>> >>> We do need to still run and pass the Web Profile tests of the Platform TCK. >>> More tests have been broken out in the same fashion CDI and Validation >>> were always broken out, but there are still a great deal of tests that are >>> not broken out and unlikely to ever be -- connector, ejb and certain >>> integration tests in particular. >>> >>> Here is a certification result that includes tests from the Platform TCK >>> and the name, version and SHA of the TCK binary used: >>> >>> - >>> https://glassfish.org/certifications/jakarta-platform/10/TCK-Results-7.0.0-M8 >>> >>> >>> Those can be found from this page: >>> >>> - https://jakarta.ee/compatibility/certification/10/ >>> >>> >>> -David >>> >>>> On Jul 23, 2024, at 12:05 PM, Richard Zowalla >>>> <mailto:mailto:r...@apache.org> wrote: >>>> >>>> Hey guys, >>>> >>>> do we actually need to run the platform-tck? >>>> >>>> Markus and myself were looking into it and for example Payara and Wildly >>>> are doing standalone setups only: >>>> >>>> - https://github.com/payara/jakartaee-10-tck-runners >>>> - https://github.com/wildfly/wildfly-tck-runners/tree/wildfly-27 >>>> >>>> So do we actually need the javatest based setup at all(aka platform-tck) >>>> ?? >>>> >>>> Or can we also just do a bunch of standalone tck setups (as we already to >>>> in /tck in the TomEE repo) and claim to be compliant if we pass the set >>>> needed for the web profile? >>>> >>>> Hoping for someone, who is more familiar with the certification process >>>> (David, maybe??) to jump in to give some guidance? >>>> >>>> Gruß >>>> Richard >>>> >>>> >>>>> Am 23.07.2024 um 12:53 schrieb Jonathan Gallimore >>>>> <mailto:mailto:jonathan.gallim...@gmail.com>: >>>>> >>>>>> Also I'm far away from doing a complete run, I only ran a couple dozen >>>>>> tests so far. Ideally once we figure out these setup issues someone from >>>>>> tomitribe could trigger a full run on tck.work so we can get some actual >>>>>> numbers. >>>>> >>>>> I'm happy to help wire that up. >>>>> >>>>> Jon >>>>> >>>>> On Tue, Jul 23, 2024 at 11:32 AM Markus Jung >>>>> <mailto:mailto:ju...@apache.org> wrote: >>>>> >>>>>> Hey Richard, >>>>>> >>>>>> >>>>>> I did some initial work on running a few TCK tests in the platform-tck >>>>>> the past days and have been able to gain some quick wins with a couple >>>>>> of setup issues. [1] >>>>>> >>>>>> >>>>>> This is where I'm at: >>>>>> Main pain points are that LegacyCookieProcessor is gone in Tomcat 10.1 >>>>>> which causes a bunch of tests to fail and tssv.jar is seemingly missing >>>>>> from the platform-tck, which makes the ProviderConfiguration [2] invalid >>>>>> as this .jar contains the class >>>>>> com.sun.ts.tests.jaspic.tssv.config.TSAuthConfigProvider. Updating the >>>>>> Context.xml configs to not reference LegacyCookieProcesor anymore did >>>>>> the trick but I'm still trying to figure out what needs to be done to >>>>>> get the tssv.jar file. It was in the EE 9.1 TCK but is seemingly missing >>>>>> from EE 10. One solution that works is just extracting it from the >>>>>> Authentication 3.0 TCK, but I doubt this is the right solution. Maybe >>>>>> someone who is more involved knows more? >>>>>> >>>>>> Also I'm far away from doing a complete run, I only ran a couple dozen >>>>>> tests so far. Ideally once we figure out these setup issues someone from >>>>>> tomitribe could trigger a full run on tck.work so we can get some actual >>>>>> numbers. >>>>>> >>>>>> Thanks >>>>>> Markus >>>>>> >>>>>> [1] https://github.com/jungm/tomee-tck/tree/ee10work >>>>>> [2] >>>>>> >>>>>> https://github.com/jungm/tomee-tck/blob/main/src/test/tomee-plume/conf/ProviderConfiguration.xml >>>>>> >>>>>> >>>>>> On 23.07.24 11:06, Richard Zowalla wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> TomEE 10.0.0-M2 is out and it seems, that we have quite a few people >>>>>> already testing it, which is great. >>>>>>> >>>>>>> Since we resolved most of the open dependency updates related to EE10, >>>>>> we can now start to work on TCK setup / compliance or at least to get >>>>>> some >>>>>> numbers. >>>>>>> Aside from the standalone TCKs here [1], we will need to check on how to >>>>>> run https://github.com/jakartaee/platform-tck/tree/10.0.x with TomEE 10 >>>>>> to (at least) get some numbers. >>>>>>> Some of the required tweaks / config options (etc) can be found here: >>>>>> https://github.com/apache/tomee-tck >>>>>>> >>>>>>> I think, that Markus J. did a quick check on the platform-tck, so maybe >>>>>> you can add your findings in this thread, so we can (hopefully) create >>>>>> an >>>>>> environment for further discussion on how to proceed with the TCK itself. >>>>>>> >>>>>>> From what I remember on the standalone TCKs: >>>>>>> >>>>>>> - We need to correctly setup the concurrency TCK and fix our >>>>>> implementation if needed. >>>>>>> - We need to check on the CDI TCK. >>>>>>> >>>>>>> All TCKs can be enabled via a specific Maven profile. >>>>>>> >>>>>>> There are still some other open things: >>>>>>> >>>>>>> - MP 6.0 requires us to implement OpenTelemetry, which hangs because of >>>>>> https://issues.apache.org/jira/browse/OWB-1441 atm. >>>>>>> - CXF 4.1.0 needs to be final ;-) >>>>>>> >>>>>>> So if anyone wants to have a look into one of these areas, feel free to >>>>>> drop a message on dev@ >>>>>>> >>>>>>> Cheers, >>>>>>> Richard >>>>>>> >>>>>>> >>>>>>> [1] https://github.com/apache/tomee/tree/main/tck >>>>>> >>>>
