Hi, you actually need to choose the right branch on that repository. Some of these tests require specific settings to be present (only available on some branches) and assume certain locale / enviroment conditions (most of them aren't documented in the README). I had a lot of pain running TCK tests for 8.x and 9.1.x on my de_DE Ubuntu machine ;-) (and yes, I also had the bsdtar issue). Another thing is, that a run polutes your TCK installation, so I ended up putting it under a local git and clean it after each run ;-)
Getting the old TCK to run, is a bit of a pain. Even if you are doing everything right, it sometimes also depends on the JDK version you have on your machine (for 9.1.1, I had some pain because of a bug in the JDK HTTP client resulting in TCK tests to fail due to a change in CXF). Long story short: Given that most TCK/specs migrated towards arquillian/junit/testng, setting up and running the TCK for TomEE 10, is now a totally different thing than before. Some of the quirks and properties / configs (and hacks) can be copied over or need to be added. A good example for that procedere is [1], in which Benedict and myself did a trial and error approach to get the JAX-RS E10 TCK set up. We would need to do that for the other relevant spec TCKs inside of TomEE too, but this is "Neuland" (new area) for everyone involved. Some of these modern TCKs are straightforward (eg. BatchEE TCK), but others require a more sophisticated setup. It involves looking into the TCK guide, looking into other projects (how they do it), copying / checking quirks required for TomEE inside the old TCK (system properties, etc.) and trying to get it setup and run it. It is really a try and error process. The signature tests are more or less "quick" wins, but sometimes also require some hackery :/ I guess, that the place to ask questions is the dev@ list. If it requires a more sync way of communicating, Slack might also be an opton. A synchronous meeting might work too, but is limited to timezone, etc. Gruß Richard [1] https://github.com/apache/tomee/pull/1063 Am Freitag, dem 22.03.2024 um 13:59 +0100 schrieb Jens Zurawski: > Hi David, > > thanks for your reply. It's a difficult situation right now, feels a > bit > like a "deadlock": No relief for the active rest of the committers > without new and "educated" team members, but no new team members > without > additional time invest of the current committers. As with technical > deadlocks, it has to be broken up at some point (even with some small > loss somewhere if there is no other way). So, I'm looking forward if > it's possible to re-activate one of the retired committers. At least > to > be available as a kind of "big brother" who can be asked some > questions > once in a while when the newbies get stuck somewhere. And, of course, > a > short introduction into what/where/when/how. > > Just out of curiosity I've started a quick and naive attempt to check > the TCKs on my machine. I actually don't have the time slices to dig > deeper into this, but I wanted to get a feeling of what's coming up > if I > decide to get into it. > > So I just installed a new WSL ubuntu 22.04 instance (if that won't > work, > I would have continued with a dedicated linux VM), installed git and > maven, and followed the instructions on > https://github.com/apache/tomee-tck/ > > It doesn't work out of the box on a fresh ubuntu. "bsdtar" is not > installed in default dist setup. Why bsdtar? Maybe could be changed > to > just use GNU tar? > Now, with bsdtar installed, after setup.sh has finished (there is no > setup-tck9.sh, the README.adoc should be adapted), there were no > error > messages or warnings, but glassfish-7.0.0-M8 download/unzip wasn't > successful. Don't know why, I've done it manually then. When I have > some > more time, I may try to find out where the problem is. > > Now, everything seems to be here and I started a first try with the > sample from the README: > ./runtests --web tomee-plume > com.sun.ts.tests.ejb30.bb.localaccess.statelessclient > > Nice, all tests passed :-) > > Then, cheered up from this first success, tried a more comprehensive > test with: > ./runtests --web tomee-plume -c com.sun.ts.tests.ejb30 > > Could run in background during I was doing my daily stuff. Ok, seems > this will take quite a while.... It's already running since 2 hours > or > so and I think I will stop it here. Until now, 795 tests passed and > 1807 > tests failed. So what I learn from this quick shot: Something is not > working correct by now or I have not understood correctly how it > works, > because a failed/passed ratio of 2:1 can't be correct. On the other > hand, passing 795 tests with success also means I'm not totally off > track ;-) > > Yeah, well, without some educated hints it will take much more time > to > correctly get into this. I guess it's only some glitches in my setup, > or > I have forgotten something, or I'm just doing it wrong. For me to > find > out what's wrong, this could be a long journey and maybe cost me > hours > and hours and hours. At this point it would be perfect if there is > one > or two persons who can take a look and out of experience say "of > course > it's not working, you have to do this and that" or even if they are > only > pointing me in the right direction of where to search would already > be > helpful. > > cu > Jens > > > > Am 21.03.2024 um 18:47 schrieb David Blevins: > > > On Mar 21, 2024, at 5:54 AM, Jens Zurawski <[email protected]> > > > wrote: > > > > > > Another problem is: I'm afraid, I will have to learn quite a > > > bunch of things before I will be of any real help to the project. > > > Although I'm already doing some JSF Appliactions my knowledge of > > > the EE universe/specifications is limited. Up to now it's more > > > cherry picking the stuff I need for my projects. > > There is a learning curve that is steep and it does involve an up > > front investment of time before you're productive. It can be done, > > though. There were four of us who did most the work on the TomEE > > 1.0 TCK compliance and I was the only one who had any experience > > with it due to my time in Geronimo. The others had to learn after > > hours from scratch. They did it and were able to help get tests to > > pass. > > > > More directly, if only full-time people were able to help there > > would never have been a TomEE in the first place. > > > > You absolutely do need someone, however, helping you that is > > familiar with the TCKs. We'd need to find someone with that time > > for anything to be successful, hence the other conversation. In > > terms of how much time it takes to get tests to pass, it's hit and > > miss. There are usually many easy wins when you're under 80% > > passing. As you approach the 95% it's harder. Those last few > > tests at 98% passing are very hard and you can spend two weeks > > trying to get one test to pass. > > > > In all cases it involves you running the test with remote > > debugging, attaching with a debugger and stepping through the code > > over and over and reading through the specification document to > > better understand the requirements the test claims it is > > asserting. You quickly become deeply familiar with parts of the > > code and parts of the spec. That knowledge expands the more tests > > you work on. For sanity sake, it's usually best to focus on one > > section for a while (say JSF), otherwise it's too much context > > shifting. > > > > The volume of tests makes it very easy for many people to work in > > parallel. We also have a system that can run all the tests in > > parallel (https://tck.work/tomee/builds), which is useful only for > > finding failing tests that you then can run and debug locally. > > > > The TCK setups themselves are getting far simpler than they were. > > The legacy Jakarta EE TCK is a 500mb beast that uses a formerly > > proprietary testing framework created in the Sun days. It's not > > much fun to work with. The setup for that is here: > > > > - https://github.com/apache/tomee-tck/ > > > > Several of the Jakarta EE spec teams have been taking their tests > > out of the legacy TCK and have been converting them to JUnit and > > TestNG, sometimes using Arquillian. These are far easier to get > > started with as it's what everyone is familiar with. > > > > > > > So.... I really would like to contribute to the project, but it > > > may turn out that eventually I will not have enough time or will > > > fail in understanding all the EE specifications and their > > > dependencies. And because I know that it will be additional work > > > for you and the other active contributors to educate me (or at > > > least give helping hands here and there), I'm hesitating. > > Then you'd be in the same boat as everyone else, including me. > > That's just the way things go. And it works in reverse as well, > > it's hard to encourage people when you know they'll need you and > > you know you're not available and they'll probably fail and have > > their time wasted. Hence the idea of finding a person who used to > > contribute and became a committer and see if we can use some brief > > sponsorship to get them back on the project in their evening time > > for a while to help all of you. We have 34 committers. I bet we > > can find at least one to help bootstrap this if we throw something > > their way. > > > > Short of that idea, we're kind of stuck for the same reasons. > > > > > > -David > > >
