Yes, but it feels a bit wasteful to create a new module for three tests. Note that one of the tests is just testing that the test rig in this file works correctly. So there's really only two tests here.
Also note that we can't change the working directory when forked per test, so each test would need to share the same Catalog.properties file, which I don't think would work. I think the best way to test things work is via a CLI test that can change directory, create files, have a custom Catalog.properties file, etc. Though, these tests don't even use valid DFDL schemas. It really just tests a couple specific things about the XMLLoader. So it might be a bit of work to get it working end ensuring it's testing the right thing, if that's even possible. On 10/26/20 10:40 AM, Beckerle, Mike wrote: > Is there any way to have a separate test module where tests are all run with > fork = true? > ________________________________ > From: Steve Lawrence <slawre...@apache.org> > Sent: Monday, October 26, 2020 9:54 AM > To: dev@daffodil.apache.org <dev@daffodil.apache.org> > Subject: TestDaffodilXMLLoader non-thread safe tests > > We have three tests in TestDaffodilXMLLoader that had a random failure > in Github Actions for a pull request I created that updates SBT to > 1.4.1. Looking at these tests, I believe they are not thread-safe and > that's likely causing the failure I'm seeing. > > The way they work is each tests creates a schema, test data, and XML > Catalog file, and a Catalog.properties file. > > The first issue is each test writes to the exact same file location, so > if these individual tests are run in parallel things will break. One > option to resolve this would be to create files in random temp > directories. This would work except for the next issue. > > Which is that the test creates a CatalogManager.properties file at the > root the classpath. When Daffodil does XML resolution, it searches for > this file to change specify XML catalogs. This file is temporarily added > during these tests to reference the created XML Catalog file > > This means that while these tests are running, the > CatalogManager.properties file could affect any other tests, not just > those in this TestDaffodilXMLLoader file. I'm wondering if this was the > actual cause of some of the random failures we used to see fairly > regularly and thought it was just non-thread safe code in the TDML > Runner. Perhaps it was actually just this test changing > CatalogManager.properties and breaking XML resolution. > > I can't think of a safe way to deal with testing this capability, except > maybe CLI tests where we actually fork processes and could do so in temp > directories with custom properties files. But these tests seem to be a > bit more unit-testy. Any thoughts on how to test this capability? Should > we convert to CLI test? Is there an alternative? Or should we just > remove them completely? >
