Is there any way to have a separate test module where tests are all run with fork = true? ________________________________ From: Steve Lawrence <slawre...@apache.org> Sent: Monday, October 26, 2020 9:54 AM To: dev@daffodil.apache.org <dev@daffodil.apache.org> Subject: TestDaffodilXMLLoader non-thread safe tests
We have three tests in TestDaffodilXMLLoader that had a random failure in Github Actions for a pull request I created that updates SBT to 1.4.1. Looking at these tests, I believe they are not thread-safe and that's likely causing the failure I'm seeing. The way they work is each tests creates a schema, test data, and XML Catalog file, and a Catalog.properties file. The first issue is each test writes to the exact same file location, so if these individual tests are run in parallel things will break. One option to resolve this would be to create files in random temp directories. This would work except for the next issue. Which is that the test creates a CatalogManager.properties file at the root the classpath. When Daffodil does XML resolution, it searches for this file to change specify XML catalogs. This file is temporarily added during these tests to reference the created XML Catalog file This means that while these tests are running, the CatalogManager.properties file could affect any other tests, not just those in this TestDaffodilXMLLoader file. I'm wondering if this was the actual cause of some of the random failures we used to see fairly regularly and thought it was just non-thread safe code in the TDML Runner. Perhaps it was actually just this test changing CatalogManager.properties and breaking XML resolution. I can't think of a safe way to deal with testing this capability, except maybe CLI tests where we actually fork processes and could do so in temp directories with custom properties files. But these tests seem to be a bit more unit-testy. Any thoughts on how to test this capability? Should we convert to CLI test? Is there an alternative? Or should we just remove them completely?
