dependabot[bot] opened a new pull request, #3524: URL: https://github.com/apache/maven-surefire/pull/3524
Bumps [org.testng:testng](https://github.com/cbeust/testng) from 7.5 to 7.5.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/cbeust/testng/releases";>org.testng:testng's releases</a>.</em></p> <blockquote> <h2>TestNG v7.5.1</h2> <h2>What's Changed</h2> <ul> <li>Cherrypick Zip Slip Vulnerability to 7.5 release by <a href="https://github.com/prashil-g";><code>@prashil-g</code></a> in <a href="https://redirect.github.com/cbeust/testng/pull/2899";>cbeust/testng#2899</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/prashil-g";><code>@prashil-g</code></a> made their first contribution in <a href="https://redirect.github.com/cbeust/testng/pull/2899";>cbeust/testng#2899</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/cbeust/testng/compare/7.5...7.5.1";>https://github.com/cbeust/testng/compare/7.5...7.5.1</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/testng-team/testng/blob/master/CHANGES.txt";>org.testng:testng's changelog</a>.</em></p> <blockquote> <p>Current (7.13.0) Fixed: GITHUB-3566: TestNG.run() records status 1 when it throws a TestNGException, including unreadable suites, missing classes or listeners, and duplicate test names. The CLI prints diagnostics, causes and stack traces to stderr. A run with no tests still reports status 8 when a configuration method fails (Krishnan Mahadevan and Arthur031221) Fixed: GITHUB-3528: A resolver that throws while TestNG builds the arguments of one sequential data-provider row now fails that row and runs the later rows. The failure carries the row. The sequential path used to let the exception leave the row loop, so the later rows produced no result and the failure named no row. A row that does not fit the method still stops the sequential provider, as before. A parallel provider and a retry already failed only the affected row (Burak Kalaycı) Fixed: GITHUB-3533: A method-level configuration failure on one parallel data-provider row no longer skips a sibling row. The failure was keyed by the instance, the current invocation count and parameterInvocationCount. Parallel workers share the ITestNGMethod, and the iterator sets parameterInvocationCount to the last row before any worker runs, so every first attempt used one token. A <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> failure on row 0 then skipped row 1 under configFailurePolicy=CONTINUE, after row 1's own setup had passed. The token now uses the row index the worker was given. The invocation count in that token is captured when the invocation starts, not read again at lookup, because sibling workers increment the shared counter. A firstTimeOnly failure is stored on a shared key so every parallel row sees it under CONTINUE. The same index is what an <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a> failure is reco rded under. org.testng.conffailure.issue3533.Issue3533Test covers two parallel rows (Burak Kalaycı) Fixed: GITHUB-3539: A pooled method whose instance id is not an index key no longer runs firstTimeOnly and lastTimeOnly configuration for every <a href="https://github.com/Factory";><code>@Factory</code></a> instance. invokeTimeOnlyConfigurations skipped the instance filter when configInstanceId returned null. A WrappedTestNGMethod around a non-BaseTestMethod invents an id for the graph, and that id is not a key, so each pool walked every factory instance. One instance then ran its firstTimeOnly method once per pool, and another never ran it. lastTimeOnly ran for every instance on every pool. When the index id is missing, the invoker now keeps the object whose embedded instance matches the method. The comparison is equals, the same one method-configuration filtering uses. The index id still wins when the method has one (Burak Kalaycı) Fixed: GITHUB-3537: Dry run reports tests with expectedExceptions as passed without invoking them. The missing-exception check still applies to normal execution (Lev Shagalov) Fixed: GITHUB-3435: A <a href="https://github.com/Factory";><code>@Factory</code></a> that declares <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> or <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a> no longer scans every instance's configuration on each test invocation. TestClass indexes those methods by instance id, the same way it already indexes <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a> and <a href="https://github.com/AfterClass";><code>@AfterClass</code></a>. The invoker reads that index. The public flat accessors still return every instance (Burak Kalaycı) Fixed: GITHUB-3530: A later configuration failure for the same class instance no longer erases earlier records. ConfigInvoker kept one RecordedFailure per instance, so a parallel worker's teardown could replace another worker's setup failure. A retry on the later thread then ignored that replacement as its own and ran even though the earlier failure still applied. Each key now keeps the latest record per thread, which is what the retry filter needs, rather than appending every skipped configuration onto a CopyOnWriteArrayList. org.testng.conffailure.RetriedInvocationTest covers two parallel data-provider rows that force that order (Burak Kalaycı) Fixed: GITHUB-1622: alwaysRun on a <a href="https://github.com/Before";><code>@Before</code></a> method no longer runs it after a previous configuration failed. The attribute is documented to mean two different things -- on a <a href="https://github.com/Before";><code>@Before</code></a> method it lets the method run regardless of the groups it belongs to, on an <a href="https://github.com/After";><code>@After</code></a> method it lets the method run even if a method invoked before it failed or was skipped -- and since 6.9.5 a <a href="https://github.com/Before";><code>@Before</code></a> method had been getting both. The fix for GITHUB-420 had added the four getBefore*() terms to MethodHelper.isAlwaysRun, and ConfigInvoker read that one predicate for two independent decisions: whether a configuration of a <a href="https://github.com/Test";><code>@Test</code></a>(enabled = false) class may still run, which is what GITHUB-420 needed, and whether it may run past a failure, w hich it did not. So a failing <a href="https://github.com/BeforeSuite";><code>@BeforeSuite</code></a> -- which marks the whole suite failed -- was followed by every alwaysRun <a href="https://github.com/BeforeTest";><code>@BeforeTest</code></a>, <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a>, <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> and <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a> of the run, and by every other alwaysRun <a href="https://github.com/BeforeSuite";><code>@BeforeSuite</code></a>: the reporter's four-method class reported 4 configuration failures and 0 skips where 6.9.4 had reported 1 and 3. The failure bypass is now read from a separate predicate, MethodHelper.canBypassConfigurationFailure, which answers true for the after role only -- <a href="https://github.com/AfterSuite";><code>@AfterSuite</code></a>, <a href="https://github.com/AfterTest";><code>@AfterTest</code>< /a>, <a href="https://github.com/AfterClass";><code>@AfterClass</code></a>, <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a> and <a href="https://github.com/AfterGroups";><code>@AfterGroups</code></a> -- so those are unchanged and keep running after a failure. The role is the one TestNG runs the method in, not the merged annotation, so a method that carries both a <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> and an <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a>(alwaysRun = true) is skipped as a setup and run as a teardown. A skipped setup still marks what it would have set up, so the tests that need it are skipped too, as they are when the setup does not say alwaysRun. The group meaning of alwaysRun on a <a href="https://github.com/Before";><code>@Before</code></a> method is untouched, and so is the enabled decision GITHUB-420 covers. This is deliberately not tied to configFailurePolicy: an alwa ysRun <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> is skipped after a failed <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a> under SKIP and under CONTINUE alike. It is listed under Possible backward incompatible changes below (Julien Herr) Fixed: A configuration method skipped because an earlier one failed no longer moves the level flags that decide where ignoreFailure is looked up. The skip went through the same bookkeeping as a failure, flags included, so a skipped <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> in one class made a later class's <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a>(ignoreFailure = true) stop protecting that class: the lookup read the later class's <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> list instead of its <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a> list, found no ignoreFailure there, and skipped the test. A skip still marks what the setup would have set up, so the tests that need it are skipped; only the flags are left to a failure. GITHUB-1622 above makes such skips far more frequent, since an alwaysRun <a href="https://github.com/Before";><code>@Before</code></a> method used to run instead (Julien Herr) Fixed: An attempt that retries a failed test method is no longer held back by what the failed attempt recorded, and by nothing else. TestInvoker already exempted the retried test method from every recorded configuration failure, but not its setup, so a retry whose <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a> had failed ran the test method with no setup at all; and the exemption was a flag on the retry context, which MethodRunner hands to the data provider rows that follow, so those rows ran past their own configuration failures too. ConfigInvoker now numbers every failure it records, with the thread that recorded it; a retry takes the mark before it starts, and its setup, its test method and its teardown ignore the failures its own thread recorded up to that mark only -- which is what the attempt it retries recorded, since the retry runs on that attempt's thread right after it. A setup that fails during the retry still skips the setups and the test met hod after it; a failure another data provider row recorded on another thread before the retry started still counts, so the retry's setup and test method are skipped where the test method used to run without its setup; and the rows that follow the retry are back to reporting what they report when nothing is retried. Until the GITHUB-1622 fix in this release the missing setup was only reachable for a <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> without alwaysRun, since alwaysRun used to bypass the failure and hide it (Julien Herr) Fixed: GITHUB-3522: ignoreFailure on a <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> no longer lets a later class run a test after that class's own <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a> failed. The flags that pick which configuration list to scan for ignoreFailure are now per test class and instance, keyed by the class under test, not the declaring type of the configuration method. A real <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> failure in class Z used to make class A's <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a>(ignoreFailure = true) look like the failure that applied to A, so A.aTest ran with its setup skipped. The same leak happened across sibling subclasses that share a configuration method, across implementors of an interface default, and across <a href="https://github.com/Factory";><code>@Factory</code></a> instances of one class. org.testng. conffailure.issue3522.Issue3522Test covers A alone, Z then A in one <!-- raw HTML omitted -->, the sibling and interface cases, and two factory instances (Burak Kalaycı) Fixed: GITHUB-3513: Under parallel="tests", a suite time-out that cancels a <!-- raw HTML omitted --> worker now reports the methods that did not return, and the run exits non-zero. invokeAll interrupts the worker and returns; a method that ignores interruption keeps running, runTest never stores a SuiteResult, and ExitCodeListener then sees the <!-- raw HTML omitted --> tags that finished -- so a suite that still has a method in flight exits 0 and writes no testng-failed.xml. After the pool returns, SuiteRunner waits a short interval for workers that respect interruption to record themselves, then fails every unfinished invocation with ThreadTimeoutException and stores the <!-- raw HTML omitted -->. The same suite with parallel="none" already reported the method as failed. org.testng.timeout.issue3513.Issue3513Test covers a method that ignores interruption and one that stops when interrupted, each beside an empty <!-- raw HTML omitted --> that still passes (Bu rak Kalaycı) Fixed: org.testng.reporters.FileStringBuffer no longer throws a NullPointerException when the first thing appended to it is a string of 100 000 characters or more. A string that size bypasses the in-memory builder and is written to the temporary file directly, and the file is created as the builder is flushed -- but flushToFile answers early when the builder is empty, which it is when nothing has been appended yet, so that path went straight on to write to a file it had just decided not to create. Appending anything at all first, however short, was enough to hide it. The file is now created where it is needed rather than as a side effect of a flush, and a buffer whose file cannot be created says so with an IllegalStateException naming it instead of failing as a null one frame later, which is how FileStringBuffer.toString already reports the same impossibility. org.testng.reporters.FileStringBufferTest, which had asserted nothing since it was written in 2013 and was listed in no su ite, now covers both readers -- toString and toWriter -- below the flush threshold, across it, and on either side of that direct-to-file size. Two silent failures in the same class go with it: a flush that could not write used to replace the in-memory builder anyway, so the characters it still held vanished from the middle of the document with nothing raised, and toWriter logged what toString has always thrown for, so a spilled buffer whose temporary file had gone answered an empty document instead. Both now raise IllegalStateException, which is what temporaryFile already did for the same class of fault. A failed spill is recorded rather than raised on the spot, and raised when the buffer is asked for its content: appending happens deep inside building a document -- for the JUnit report, inside a listener TestNG calls bare -- so raising there would end the run at whatever tag happened to overflow. And JUnitXMLReporter catches it, because it is an IResultListener2 rather than an IRep orter: TestNG.generateReports wraps each reporter in its own try/catch, but TestRunner calls its listeners with nothing around them, so a report that could not be written would have cost the run and every other report with it. It loses its file and says so through its logger. The message on the way out no longer names the temporary file, since the same catch covers a buffer that never spilled and has none -- there the fault is the destination, which while index.html is being written is the likelier of the two. And a report that gives up part way leaves no file and says so through its logger, whatever it gave up with: Utils.writeUtf8File writes its prefix before the buffer, so index.html was left as a page header with no body and nothing to say why -- and the OutOfMemoryError this entry is about is an Error rather than a RuntimeException, so the case that mattered most was the one the removal missed. And the temporary file is written and read as UTF-8 rather than through the platform default charset: a character that charset could not encode was lost at spill time, before the explicitly UTF-8 page writer ever saw it, so a supplementary character in a report reached index.html as a question mark on a JDK whose default is not UTF-8 (Julien Herr) Fixed: GITHUB-1259, GITHUB-2334: index.html is no longer built by holding one panel of it in memory at a time. Every panel of the default HTML report writes into an XMLStringBuffer of its own -- backed, like the page it feeds, by a buffer that spills to a temporary file past 100 000 characters -- and used to hand that buffer to the page as a String, which read the whole file back and copied it once more for the regular expression that drops the characters XML 1.0 does not allow. So a buffer whose reason to exist is not being held in memory was held in memory twice over, once per panel, and the largest of them grows with the result count: GITHUB-2334 ran out of heap in NavigatorPanel.generateMethodList listing the passed methods of a run of 500 000, GITHUB-1259 in BaseMultiSuitePanel, both of them under a summary saying every test of the run had passed. The panels now stream their buffer into the page through the new XMLStringBuffer.addBuffer, which drops the same characters a slic e at a time, and TimesPanel writes its table a row at a time rather than building the whole of it as one String. On a run of 20 000 results whose report is 39 MB, generating it needed a 64 MB heap and now needs 28 MB; below that the limit is what the run itself retains, which is GITHUB-1979 and unchanged here. The page is identical, the order of what it lists included. org.testng.reporters.XMLStringBuffer.addBuffer(XMLStringBuffer) is new, and the panels of org.testng.reporters.jq write into a buffer BaseMultiSuitePanel owns, through a new package-private writeContent, rather than each building one and answering its String. BaseMultiSuitePanel.getContent(ISuite, XMLStringBuffer) stays where it is -- concrete now rather than abstract -- for callers outside the report, and is the one place left that materialises a panel. BaseMultiSuitePanel.getContent(ISuite, XMLStringBuffer) is final: generate no longer calls it, so an override outside the package decided nothing and would have been skipped without a word. TimesPanel no longer adds each result's duration into its running total on every call, which made a second call to getContent report the suite as having run twice as long (Julien Herr) Fixed: GITHUB-2187: testng-results.xml now writes a parameter <!-- raw HTML omitted --> as <!-- raw HTML omitted --><!-- raw HTML omitted --><!-- raw HTML omitted --> on one line, so XML consumers that read the element text get the parameter rather than the pretty-print whitespace around the CDATA (Burak Kalaycı) Fixed: GITHUB-1031: A data provider that supplies the wrong number or type of arguments now reports that mismatch. Count errors state the expected and actual counts. Type errors state that the types do not match. The matcher no longer claims the test method has no parameters (Burak Kalaycı) Fixed: GITHUB-3121: EmailableReporter2 now appends each test invocation's non-blank ITest custom name to the Java method name in the summary table and scenario headings. Configuration methods keep the Java method name (Burak Kalaycı) Fixed: GITHUB-2940: testng-failed.xml now includes passed members of a group that a skipped or failed test depends on. MethodHelper.getMethodsDependedUpon now adds those group members and walks each member's own method and group dependencies. The walk uses only direct graph predecessors. A rerun suite therefore keeps the full dependency closure the skipped test needs (Burak Kalaycı) Fixed: GITHUB-3234: A test class whose methods cannot be read because a referenced type is missing is no longer treated as if it had no TestNG annotations. TestNGClassFinder.isTestNGClass now throws TestNGException wrapping the NoClassDefFoundError when that class was named in the suite, so the class is reported rather than silently skipped. A class found through a package scan still returns false, as GITHUB-602, so an optional class with a missing type does not fail the scan. A missing type used only in a method body is still a test failure, as before (Burak Kalaycı) Fixed: GITHUB-3478: Report an empty DataProvider as a skipped test with EmptyDataProviderBehavior.SKIP by default, and provide EmptyDataProviderBehavior.IGNORE for backward compatibility, selectable on the command line with -emptydataproviderbehavior (Julien Herr) Fixed: GITHUB-2927: IMethodSelector.setTestMethods now receives the known test methods after they are bound to their ITestClass, and before selectors are consulted for test or configuration methods. A selector that snapshots the list sees the methods; includeMethod is never called first. (Burak Kalaycı) Fixed: GITHUB-3437: ClassHelper.getAvailableMethods now memoizes the hierarchy scan per class in a ClassValue, so data-provider and configuration lookups no longer rescan the same class on every call. Callers still receive a fresh set (Burak Kalaycı) Fixed: GITHUB-3239, GITHUB-2714: configuration-method inheritance is kept whenever the explicit dependency graph has no path between the two methods. A child <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a> no longer runs between two base methods, including when it only has an unrelated groups= value, and a base <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a> no longer runs before the child's. An inheritance edge is omitted when it would close a cycle with an existing dependsOnGroups or dependsOnMethods chain (Burak Kalaycı) Fixed: GITHUB-2333: skip module-info.class and META-INF/versions entries when discovering classes in a test jar (Burak Kalaycı) Fixed: GITHUB-2595: A programmatic IMethodSelector at priority 10 is no longer dropped as a duplicate of the built-in XmlMethodSelector. RunInfo stored selectors in a TreeSet ordered only by priority, so two descriptors with the same priority compared equal and one was discarded. Equal-priority selectors now keep insertion order (Burak Kalaycı) Fixed: GITHUB-3222: dependsOnMethods in a static nested test class now resolves sibling methods, including methods inherited onto that nested class. MethodHelper.findDependedUponMethods() used to drop every method whose class had an enclosing class, so the nested class never saw its own tests (Burak Kalaycı) Fixed: GITHUB-3388: XmlSuite.toXml() now reports an unnamed XmlDefine reached through a test-level <!-- raw HTML omitted --> with "<!-- raw HTML omitted --> has no name", matching the unnamed XmlPackage path, instead of a bare NullPointerException from Properties.setProperty (Burak Kalaycı) Documentation: GITHUB-1346: <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a> and <a href="https://github.com/AfterGroups";><code>@AfterGroups</code></a> now state in their javadoc where they sit in the lifecycle. The group configuration runs inside the lifecycle of the class the group is entered and left from, not around it: <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a> runs before <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a>, and <a href="https://github.com/AfterGroups";><code>@AfterGroups</code></a> before <a href="https://github.com/AfterClass";><code>@AfterClass</code></a>. GITHUB-1346 asked for the reverse -- <a href="https://github.com/BeforeTest";><code>@BeforeTest</code></a> -> <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a> -> <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a>, and <a href="https://github.com/AfterClass";><c ode>@AfterClass</code></a> -> <a href="https://github.com/AfterGroups";><code>@AfterGroups</code></a> -- on the reading that a group is declared on a <!-- raw HTML omitted --> and may span several classes, so it should be the outer scope. It cannot be one. A group is a selector over test methods, not a container of classes, and two shapes TestNG has always supported make the requested order undefined rather than merely different: a class whose methods alternate between two groups enters the second group in the middle of its own run, long after its single <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a>, and leaves the first one while its own methods of the second are still to come; and a test method belonging to two groups has the setup of both due before it, with neither group enclosing the other. An order that holds for the first group a class enters and is undefined for every later one is not a contract, and making it hold always would mean schedu ling every group's methods contiguously -- a different feature from a lifecycle reordering, and one that would collide with preserve-order, priority, dependsOnMethods and dependsOnGroups. Nothing changes in what TestNG does: the ordering described is the one it has produced all along, it is now written down (Julien Herr) Fixed: GITHUB-1263: An IMethodInterceptor can now tell what each test method it is handed declares -- a dependsOnMethods or a dependsOnGroups -- and so what it is free to reorder and what it is free to drop. ITestNGMethod.upstreamDependencies() and downstreamDependencies() were published from the graph the run is scheduled on, and that graph is built from what the interceptors return, so every method an interceptor was handed answered two empty sets. The declared relation is now resolved over the whole method set and published before the chain runs, and only when a user interceptor is registered. It is deliberately not the scheduling graph, which over methods about to be dropped would validate their dependsOnGroups, reject a cycle they resolve and materialise the lazy <a href="https://github.com/Factory";><code>@Factory</code></a> instances they carry. A dependency that cannot be resolved is therefore absent from what the interceptor is told rather than fatal; the scheduling gra ph remains the one that validates what actually runs. What an interceptor is handed is unchanged: still every test method of the <!-- raw HTML omitted -->, the ones taking part in a dependency included. Restoring the javadoc's promise that only free methods are passed would have taken from every filtering interceptor the ability to exclude a dependency-constrained test, so the wording is corrected instead, on IMethodInterceptor and on both accessors: the order preserve-order or group-by-instances imposes is TestNG's own and is not reported to an interceptor, the returned order cannot move a method ahead of what it has to follow, dropping a method another retained method depends upon ends the run with a TestNGException, and the answer is a view of a set TestNG rewrites rather than a value to hold. org.testng.internal.WrappedTestNGMethod now delegates the two accessors instead of inheriting an interface default that throws, and org.testng.internal.LiteWeightTestNGMethod answers the em pty set a snapshot holding no reference to the run can give (Julien Herr) Fixed: GITHUB-1333: A suite-level time-out no longer costs parallel="tests" its thread affinity. Under parallel="tests" a time-out declared on <!-- raw HTML omitted --> is no longer inherited as a time-out on each method of a <!-- raw HTML omitted -->; it bounds the <!-- raw HTML omitted -->, which is what SuiteRunner already did with it. The DTD has always given the attribute one meaning per mode -- it aborts "the method (if parallel=methods) or the test (if parallel=tests)" -- but BaseTestMethod.getTimeOut fell back to the XmlTest value in every mode, so under parallel="tests" the suite value was applied twice: once to the <!-- raw HTML omitted --> worker, by SuiteRunner, and again to every method. That second application is what caused the reported symptom: a method carrying a time-out is handed to a single-use executor of its own, so a <!-- raw HTML omitted --> put the <a href="https://github.com/BeforeClass";><code>@BeforeClass</code> </a> and each <a href="https://github.com/Test";><code>@Test</code></a> of a <!-- raw HTML omitted --> on three different threads, and removing the time-out was the only way to get the documented behaviour back. The inheritance is skipped exactly when the suite-level bound applies and carries the same value: the suite runs parallel="tests", and the <!-- raw HTML omitted --> declares no time-out of its own, or the same one. Anything else still reaches the method as before, because nothing else enforces it: a time-out declared on the <!-- raw HTML omitted --> alone, since SuiteRunner bounds its workers with the suite value only and TestTaskExecutor is not used in a mode that is not parallel at test level; and a parallel="tests" declared on the <!-- raw HTML omitted --> alone, since the parallel-tests path is selected from the suite's mode. Both keep their per-method time-out, and so keep running a timed method on a thread of its own. An explicit <a href="https:// github.com/Test"><code>@Test</code></a>(timeOut=...) is untouched and still bounds the method in every mode. Every other parallel mode still inherits the suite and test time-out per method exactly as before. Two shapes keep a pool of their own whatever the time-out says, and are unchanged here: a parallel data provider, and <a href="https://github.com/Test";><code>@Test</code></a>(invocationCount=N, threadPoolSize=N). They are listed under Possible backward incompatible changes below (Julien Herr) New: GITHUB-1164: A third-party library can now own a test method parameter, through the new org.testng.IParameterResolver. It is an ITestNGListener, so it is registered the way every other listener is -- <a href="https://github.com/Listeners";><code>@Listeners</code></a>, the <!-- raw HTML omitted --> tag, TestNG.addListener, or the ServiceLoader -- and it answers two questions: supportsParameter(Parameter, ITestNGMethod, ITestContext), asked while the arguments of an invocation are built, and resolveParameter(...), asked once per invocation. A parameter a resolver claims is taken out of data provider matching rather than padded: <a href="https://github.com/Test";><code>@Test</code></a>(dataProvider = "dp") void test(String a, <a href="https://github.com/Mine";><code>@Mine</code></a> Custom c, int b) runs against a row of {"value", 42} and is invoked with ["value", resolved, 42]. The same resolution applies to a method with no data provider at all, which is the half of <a href="https://redirect.github.com/cbeust/testng/issues/1164";>#1164</a> that used to fail with "Cannot inject <a href="https://github.com/Test";><code>@Test</code></a> annotated Method", and it composes with the varargs matching and with native injection: the first Method parameter, every ITestContext, ITestResult and XmlTest one, and an Object[] slot, is never offered to a resolver, so one cannot displace what TestNG already owns, while a second Method parameter -- which TestNG has never injected -- is offered like any other. A retry is an invocation of its own and is resolved afresh, whether or not its data provider row is cached; a data provider row that runs in parallel is resolved on the thread that invokes it; and a method that is only reported as skipped resolves nothing. A parameter carrying <a href="https://github.com/NoInjection";><code>@NoInjection</code></a> is offered, since that annotation already means TestNG has stopped ownin g it, and falls through to the data provider as before when no resolver wants it. Two resolvers claiming the same parameter is an error naming the method, the parameter and both resolvers, rather than an undocumented first-listener-wins; a resolver that throws, or that answers a value the parameter cannot take, fails that method with a diagnostic naming the resolver, the method and the parameter, and keeps the original exception as the cause. A parameter no resolver claims keeps the matching rules and the diagnostics it had. Resolvers are held per suite, beside the data provider listeners, and a run that registers none resolves nothing -- the resolution is the empty singleton and every filter and injection short circuits on it, so parameter matching is bit for bit what it was. This first implementation covers <a href="https://github.com/Test";><code>@Test</code></a> methods; constructors, <a href="https://github.com/Factory";><code>@Factory</code></a> and the configuration metho ds are not resolved. They are listed under Possible backward incompatible changes below (Julien Herr) Fixed: A retry that re-reads its data provider row now invokes the test method with the same kind of arguments the first attempt had. TestInvoker.retryFailed re-reads the row when the data provider declares cacheDataForTestRetries = false, and handed it to the invocation as it came, without running it back through the matcher -- so everything TestNG supplies itself was dropped and the method was called with the provider's arity instead of its own. A method taking a natively injected ITestContext beside one provider value failed its retry with "wrong number of arguments: 1 expected: 2", having never entered the body. The row now goes back through Parameters.injectParameters, which is what produced the first attempt's arguments, so native injections and the parameters an IParameterResolver owns are both restored at their declared positions. A retry whose row an interceptor excluded still falls back to the values it came in with, unchanged (Julien Herr) Fixed: A retry whose data provider fails to rebuild its row now reports that failure. With cacheDataForTestRetries = false, TestInvoker.retryFailed asks the provider again, and when that threw it left the retry loop with nothing recorded: the method's first attempt had already been marked as retried, so a method that failed and whose provider then broke was reported as one skip and no failure. The provider's exception is now the retry's own result, given the status a first attempt would have been given -- a skip, or a failure when the configuration or the provider asks for data provider failures to fail the test (Julien Herr) Fixed: A parallel data provider whose row does not fit the method now fails that row alone, and runs the others. The rows of a parallel provider used to be matched against the method one after the other on the scheduling thread, before any was submitted, so the first row that did not fit ended the whole method with a MethodMatcherException and no row ran. Each row is now matched on the worker that will invoke it, and a row that does not fit is reported as its own failure, carrying the row and the matcher's exception as it was thrown -- not a CompletionException around it, which is what letting the worker throw would have handed a listener. The rows that fit run and pass. A sequential provider is unchanged: it still stops at the first row that does not fit (Julien Herr) Deprecated: org.testng.internal.Parameters.injectParameters(Object[], Method, ITestContext), the form TestNG's own invoker used until now. It knows nothing of the IParameterResolvers a run registers, so a parameter one of them owns is counted against the data provider row and the method fails on arity with nothing saying resolution was skipped. injectParameters(Object[], ITestNGMethod, ITestContext, Collection) is what TestNG calls (Julien Herr) Fixed: GITHUB-2804: A dependsOnGroups declared by a <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a> method now orders the tests of the group that method runs before. <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a> and <a href="https://github.com/AfterGroups";><code>@AfterGroups</code></a> methods are not nodes of the scheduling graph -- they are pulled dynamically, right before the first test method of a group they target -- and MethodHelper.topologicalSort leaves the group dependencies of a group configuration method alone for exactly that reason, so the dependency reached no scheduler at all: <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a>(value = "A", dependsOnGroups = "Z") ran the whole of group A, configuration included, before group Z had started. The graph now carries that dependency on the test methods of the target group, which is where it can be scheduled, so every m ethod of Z runs before the configuration and the configuration before the first method of A, in parallel mode as well as sequentially. A group named by such a dependency but holding no method in the current <!-- raw HTML omitted --> stays a no-op rather than becoming an error, as it has always been. What a failure in Z does is unchanged: TestInvoker decides skips from the test method's own dependsOnGroups, so a failing Z orders A after it without skipping it (Julien Herr) Fixed: GITHUB-299: The chronological panel of the HTML report now closes its last <!-- raw HTML omitted -->. The block was opened on each class transition and closed only on the following one, and XMLStringBuffer.toXML() hands back the buffer without closing what is still on its tag stack, so index.html carried one unclosed <!-- raw HTML omitted --> for every suite it reported -- 61 opening tags against 60 closing ones for a two-class suite. Every closing tag after it then matched one element too shallow, which is what put the later suites' chronological panels inside the first suite's still-open block instead of beside it. ChronologicalPanel was the only panel doing this; the other six close everything they open (Julien Herr) Changed: org.testng.internal.IObject names its hash code accessor getObjectHashCodes(), and its static helper objectHashCodes(Object), where both used to say instance. The old names collided with the deprecated org.testng.IClass.getInstanceHashCodes(), so a single method body in ClassImpl, NoOpTestClass and TestClass served both a contract deprecated since 7.10.0 and a current one still reached through ITestNGMethod -- which made marking those bodies deprecated only half true. Pulling the two apart also showed that ClassImpl answered null before its objects were built, where IObject promises an array; the field starts empty now, which is what the only live caller already made of that null. org.testng.ITestNGMethod.getInstanceHashCodes() is a different contract and is untouched. They are listed under Possible backward incompatible changes below (Julien Herr) Fixed: GITHUB-2830: A parameter whose toString() throws no longer costs the run its reports. Rendering a value runs the user's code, and org.testng.internal.Utils.toString did not guard it, so a suite with one such parameter passed every test and then lost three reports at once: no testng-results.xml at all, no jq report, and emailable-report.html left at zero bytes. Nothing failed -- TestNG catches what an IReporter throws, prints it to stderr and moves on -- the files were simply never written. Utils.toString is now failsafe the way Utils.buildStackTrace already was, which covers every caller of it, each one being a report or a console line: a value that cannot render itself is written as com.example.Thing@1b6d3586, what Object.toString() answers for a class that does not override it. This is a behaviour change for the XML reports, where a run that produced an error now produces a value. An Error is caught as well as a RuntimeException, since the catch TestNG puts around a repor ter covers only Exception and a toString() that recurses on itself would otherwise end the run. TestHTMLReporter is unchanged: its own GITHUB-2830 failover calls toString() directly rather than through Utils, and spells the same identity in decimal (Julien Herr) Fixed: GITHUB-447: index.html and emailable-report.html now report the values an invocation ran with as they were when it started. Both are written once every invocation of the run is over, so a data provider that hands the same mutable row to every invocation, or a test that changes what it was given, left every scenario of that method carrying the value's final state: three invocations of one method were all named report(invocation-4) -- which also gave the three of them one HTML anchor -- and four overlapping data provider rows were all named report(mutated). The HTML report lists the configuration methods too, the ones that passed included, so a <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> handed the row its test method will run with was listed as prepare([mutated]) where VerboseReporter had printed [before-configuration], for the same invocation of the same run. Both reports now read the rendering TestNG takes as each invocation starts, which as ks nothing of the parameter's type; every value is written exactly as before, and is still rendered once per invocation however many built-in reports read it. The factory parameters emailable-report.html lists are unchanged, since no snapshot describes them (Julien Herr) Fixed: emailable-report.html is no longer lost whole to a parameter whose toString() answers null. Not the GITHUB-2830 case above, and not covered by it: the rendering does not fail, it answers nothing, so Utils.toString has nothing to guard and answers null for such an object and Utils.escapeHtml stopped accepting one, so generateReport threw a NullPointerException, TestNG printed "Reporter ... failed" and left the file empty -- one such parameter anywhere in a run was enough to lose the whole report. It is now written as the word null -- on a method parameter row and on a factory parameter row alike -- which is what index.html has always shown for such a value and what testng-results.xml has always contained (Julien Herr) New: GITHUB-2663: A configuration method can now carry a priority, the way a test method already could. <a href="https://github.com/BeforeSuite";><code>@BeforeSuite</code></a>, <a href="https://github.com/AfterSuite";><code>@AfterSuite</code></a>, <a href="https://github.com/BeforeTest";><code>@BeforeTest</code></a>, <a href="https://github.com/AfterTest";><code>@AfterTest</code></a>, <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a>, <a href="https://github.com/AfterClass";><code>@AfterClass</code></a>, <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a>, <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a>, <a href="https://github.com/BeforeGroups";><code>@BeforeGroups</code></a> and <a href="https://github.com/AfterGroups";><code>@AfterGroups</code></a> all accept priority(), and the lower value runs first -- so a project holding dozens of them, which until now had to name its methods for the o rder it wanted or chain them with dependsOnMethods, can state that order outright. The priority only ever arranges methods TestNG was otherwise free to run in any order. A dependsOnMethods or dependsOnGroups chain outranks it, and so does the inheritance guarantee: a subclass <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> does not overtake its superclass's by asking for a lower number, and the <a href="https://github.com/After";><code>@After</code></a> methods still walk back up the hierarchy child first, with the priority ordering only what sits at the same level. Note the priority itself is not mirrored for the <a href="https://github.com/After";><code>@After</code></a> methods the way the inheritance order is -- the lower value runs first there too. Priorities are compared inside the scope TestNG already collects the methods in, which is the test class, superclasses included, for the class-level, method-level and group-level ones, and the <!-- raw HTML omitted --> tag, all of its classes, for the test-level and suite-level ones; a priority therefore never couples two configuration methods TestNG would not have ordered against each other anyway, and in particular the <a href="https://github.com/BeforeSuite";><code>@BeforeSuite</code></a> methods contributed by different <!-- raw HTML omitted --> tags still run in <!-- raw HTML omitted --> order rather than being sorted across the suite. One shape is worth knowing before reaching for this: a configuration method taking part in a dependsOn chain is emitted ahead of the independent ones whatever the priorities say, which is the order TestNG already produced for such a set. Declaring no priority leaves every existing suite ordered exactly as it was (Julien Herr) Fixed: GITHUB-775: TestClass now looks its configuration methods up once for the test class instead of once per <a href="https://github.com/Factory";><code>@Factory</code></a> instance. The ten testMethodFinder.getBefore/After...Methods calls sat inside the loop that binds those methods to each instance, and every one of them rescans the whole class hierarchy, reads the annotations of every method it finds, builds a ConfigurationMethod per hit and then sorts the result -- none of which depends on the instance, since the argument is the test class every time. So a factory producing N instances paid for ten full scans N times over. The lookups are hoisted and their results bound to each instance in turn; a class that produced no instance is still not scanned at all. On a reduced form of the GITHUB-772 reproducer -- an eager constructor <a href="https://github.com/Factory";><code>@Factory</code></a> over a class carrying one method of every configuration type -- the collection ph ase is about six times faster -- roughly 2,4s down to 0,4s at 5000 instances and 0,48s down to 0,08s at 1000 -- and is still linear in the number of instances, the remainder being the per-instance ConfigurationMethod construction the binding performs. The methods that come out of it are unchanged, including the long-standing quirk that the array accessors for the suite, test, class and groups categories answer only the last instance's methods while the per-instance @BeforeClass/<a href="https://github.com/AfterClass";><code>@AfterClass</code></a> copies live in getInstanceBeforeClassMethods and getInstanceAfterClassMethods (Julien Herr) New: The same <!-- raw HTML omitted --> listed twice in one <!-- raw HTML omitted -->, and the same <!-- raw HTML omitted --> listed twice for one class, now run once per occurrence with that occurrence's own parameters. The DTD has always allowed both -- <!-- raw HTML omitted --> is class* and both tags take their own <!-- raw HTML omitted --> -- but discovery collapsed them: ClassInfoMap keyed the <!-- raw HTML omitted --> tags by java class, so the last one silently replaced the earlier ones, and XmlTestUtils resolved parameters by name, taking the last matching <!-- raw HTML omitted --> and the first matching <!-- raw HTML omitted -->. So <!-- raw HTML omitted --><!-- raw HTML omitted --><!-- raw HTML omitted --> followed by the same tag with value="two" ran C once with id=two, and two <!-- raw HTML omitted --> tags carrying id=one and id=two ran f once with id=one. Each method is now scheduled once per <!-- raw HTML omitted --> occurrence and once per <!-- raw HTML omitted --> that names it, and carries that tag with it, so it reads the parameters of the occurrence it was scheduled for. A class listed once is unaffected, down to the method identity: the occurrence index folded into ITestNGMethod equality is zero for every suite that repeats neither tag. <a href="https://github.com/BeforeClass";><code>@BeforeClass</code></a> and <a href="https://github.com/AfterClass";><code>@AfterClass</code></a> still run once per class per instance, which is what they are documented to do -- before the first and after all the test methods of the current class (Julien Herr) Fixed: A repeated <!-- raw HTML omitted --> whose <!-- raw HTML omitted --> select different methods no longer runs each of them for both occurrences. This follows from the above: an occurrence that lists <!-- raw HTML omitted --> schedules f and not g, where scheduling every method for every occurrence would have run both twice. A class listed once still creates its non-selected methods and lets XmlMethodSelector reject them, which is what makes them reportable as excluded (Julien Herr) Fixed: XmlClass.getAllParameters() now includes the <!-- raw HTML omitted --> parameters. It is documented as "the parameters defined in this test tag and the tags above it" but read only the <!-- raw HTML omitted --> tag's own parameters, stopping one level short, so it disagreed with XmlTest.getAllParameters() directly above it and with the resolution TestNG actually applies at invocation time. XmlInclude.getAllParameters(), which chains through it, gains the same level (Julien Herr) Fixed: GITHUB-3418: EmailableReporter2 no longer writes an empty invisible filler row under a result that already listed factory parameters. dumpParametersInfo for method parameters overwrote the flag the factory-parameter dump had just set, so a <a href="https://github.com/Factory";><code>@Factory</code></a> instance with no method parameters still got <!-- raw HTML omitted --><!-- raw HTML omitted --><!-- raw HTML omitted --> under those factory columns (Burak Kalaycı) Fixed: GITHUB-447: A configuration method that passed now reports in the XML reports the values it ran with, where it reported their final state. testng-results.xml lists the configurations that passed and is written once every invocation of the run is over, but the rendering taken as such a method started was dropped the moment it succeeded -- on the premise that only failed and skipped configurations are listed, which was true until the XML reports started reading those renderings. So a <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> handed the row its test method will run with reported what it left behind: VerboseReporter printed prepare([Ljava.lang.Object;)(value(s): [before-configuration]) and the file said [mutated], for the same invocation of the same run. The store now knows whether a reporter will read it after the invocations are over, and holds everything until the last one has, which also means the value is rendered once rather than captured , dropped, and rendered a second time by the fallback -- on <a href="https://github.com/BeforeMethod";><code>@BeforeMethod</code></a> and <a href="https://github.com/AfterMethod";><code>@AfterMethod</code></a>, the most frequent invocations of a run. A run whose only readers sit in the invocation lifecycle, which is what TextReporter and VerboseReporter do, still drops what they are finished with, so it retains nothing it has no use for. What is retained otherwise is bounded by what TestNG already holds: a configuration declaring no parameter stores nothing at all, and the results themselves are kept for the whole run either way, since that is where the report reads them (Julien Herr) Fixed: org.testng.internal.ClassHelper.forName no longer risks a ConcurrentModificationException when a class loader is registered while a suite is running. The list behind addClassLoader was a Vector, whose synchronised add says nothing about the iteration forName does over it from the runner's threads; it is a CopyOnWriteArrayList, which iterates a snapshot. addClassLoader is public static and reachable from a user thread, so the two really can overlap (Julien Herr)</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/testng-team/testng/commit/7ddeadb8e3efaa91a58a1706938fd41d7fea8e7c";><code>7ddeadb</code></a> Attempting Release 7.5.1</li> <li><a href="https://github.com/testng-team/testng/commit/18810fcb2745fcc9d1f7226e956c2aaabeec7f21";><code>18810fc</code></a> Cherrypick - 47afa2c8a29e2cf925238af1ad7c76fba282793f to 7.5 release</li> <li><a href="https://github.com/testng-team/testng/commit/d615583e2d63c809e9cf02357cb5338be24198ab";><code>d615583</code></a> assertEquals(Set,Set) now ignores ordering as it did before, fixes <a href="https://redirect.github.com/cbeust/testng/issues/2643";>#2643</a> (<a href="https://redirect.github.com/cbeust/testng/issues/2648";>#2648</a>)</li> <li><a href="https://github.com/testng-team/testng/commit/573607ba37bdb17f23ac641c5e00eeb742b37a20";><code>573607b</code></a> assertEquals(Set,Set) now ignores ordering as it did before <a href="https://redirect.github.com/cbeust/testng/issues/2643";>#2643</a></li> <li><a href="https://github.com/testng-team/testng/commit/49dbdc9fcc2447d9e2975d6740ac1bb435a861ce";><code>49dbdc9</code></a> Restore BeforeGroups|AfterGroups functionality back</li> <li><a href="https://github.com/testng-team/testng/commit/d9c6e4532e0119a16401e45c6a6e93271dfd68dd";><code>d9c6e45</code></a> Merge pull request <a href="https://redirect.github.com/cbeust/testng/issues/2659";>#2659</a> from RiJo/patch-1</li> <li><a href="https://github.com/testng-team/testng/commit/7308c97ea79d79100f15961799389ef54b2816bb";><code>7308c97</code></a> Bugfix in Assert#assertNotEquals(float, float, float, String)</li> <li><a href="https://github.com/testng-team/testng/commit/80e02ff41021210415a49c488089f05b1aa3dd12";><code>80e02ff</code></a> [GITHUB-2652] Assert methods requires casting since TestNg 7.0 for mixed boxe...</li> <li><a href="https://github.com/testng-team/testng/commit/c1553036aeae8d7cfe4bcb5cac26462b08f52906";><code>c155303</code></a> Fixing formatting issues which caused build failure</li> <li><a href="https://github.com/testng-team/testng/commit/874f36893c56c03f96ec046c13f3722cfeb838da";><code>874f368</code></a> Log skip reason when data provider returns no data</li> <li>Additional commits viewable in <a href="https://github.com/cbeust/testng/compare/7.5...7.5.1";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/maven-surefire/network/alerts). </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
