Hi Jon I run the whole jdk regression tests with jtreg-4.2-b03 [1], and then found lots of tests in different test groups failed with the error message like following: test test.java.time.format.TestNumberPrinter.test_pad_ALWAYS(): failure org.testng.internal.reflect.MethodMatcherException: Data provider mismatch Method: test_pad_ALWAYS([Parameter{index=0, type=int, declaredAnnotations=[]}, Parameter{index=1, type=int, declaredAnnotations=[]}, Parameter{index=2, type=long, declaredAnnotations=[]}, Parameter{index=3, type=java.lang.String, declaredAnnotations=[]}]) Arguments: [(java.lang.Integer)1,(java.lang.Integer)1,(java.lang.Integer)-10,null] at org.testng.internal.reflect.DataProviderMethodMatcher.getConformingArguments(DataProviderMethodMatcher.java:52) at org.testng.internal.Invoker.injectParameters(Invoker.java:1228) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1125) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112) at org.testng.TestRunner.privateRun(TestRunner.java:778) at org.testng.TestRunner.run(TestRunner.java:632) at org.testng.SuiteRunner.runTest(SuiteRunner.java:366) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319) at org.testng.SuiteRunner.run(SuiteRunner.java:268) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1225) at org.testng.TestNG.runSuitesLocally(TestNG.java:1150) at org.testng.TestNG.runSuites(TestNG.java:1075) at org.testng.TestNG.run(TestNG.java:1047) at com.sun.javatest.regtest.TestNGAction$TestNGRunner.main(TestNGAction.java:220) at com.sun.javatest.regtest.TestNGAction$TestNGRunner.main(TestNGAction.java:184) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:537) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:110) at java.base/java.lang.Thread.run(Thread.java:844)
This test methd signature is: public void test_pad_ALWAYS(int minPad, int maxPad, long value, String result) The provider return the following data Object[][] provider_pad() { return new Object[][] { {1, 1, -10, null}, {1, 1, -9, "9"}, {1, 1, -1, "1"}, ... However, I got message "Cannot find class java/lang/reflect/JTRegModuleHelper.class to init patch directory" when I run jtreg, although I didn't find any code related to testng dataprovider in jtreg source, I would double confirm with you if this is a definite issue or anything I made incorrectly? To Christoph Except above issue, I didn't find any other issue in jdk and langtools repo so far. [1] https://adopt-openjdk.ci.cloudbees.com/job/jtreg/lastSuccessfulBuild/artifact/jtreg-4.2-b03.tar.gz Thanks Frank > -----Original Message----- > From: Langer, Christoph [mailto:christoph.lan...@sap.com] > Sent: Wednesday, November 23, 2016 3:41 PM > To: Frank Yuan > Cc: code-tools-...@openjdk.java.net; core-libs-dev@openjdk.java.net; > jtreg-...@openjdk.java.net; 'Volker Simonis'; 'Daniel Fuchs' > Subject: RE: Issues running JAXP jtreg tests ("java.lang.RuntimePermission" > "accessDeclaredMembers") > > Thanks, Frank. > > we run scheduled jtreg tests for jdk every night so we should have > encountered issues if there were some. But langtools could be interesting, I > don't think those run automatically for OpenJDK in our environment. > > Best regards > Christoph > > > -----Original Message----- > > From: Frank Yuan [mailto:frank.y...@oracle.com] > > Sent: Mittwoch, 23. November 2016 06:26 > > To: Langer, Christoph <christoph.lan...@sap.com>; 'Volker Simonis' > > <volker.simo...@gmail.com>; 'Daniel Fuchs' <daniel.fu...@oracle.com> > > Cc: code-tools-...@openjdk.java.net; core-libs-dev@openjdk.java.net; jtreg- > > u...@openjdk.java.net > > Subject: RE: Issues running JAXP jtreg tests ("java.lang.RuntimePermission" > > "accessDeclaredMembers") > > > > Hi Christoph and Volker > > > > I have been launching jdk and langtools tests with the new jtreg, will > > update to > > you once I get the result. > > Hope jaxp test is special because most of tests should control the Security > > Manager setting inside the test methods. > > > > Thanks > > Frank > > > > > > > > > -----Original Message----- > > > From: core-libs-dev [mailto:core-libs-dev-boun...@openjdk.java.net] On > > Behalf Of Langer, Christoph > > > Sent: Wednesday, November 23, 2016 3:51 AM > > > Subject: RE: Issues running JAXP jtreg tests > > > ("java.lang.RuntimePermission" > > "accessDeclaredMembers") > > > > > > Thanks a lot Volker and Daniel for the big support to analyze and fix > > > this. > > > > > > It seems to me that the proposed fix > > > (http://cr.openjdk.java.net/~dfuchs/webrev_8170192/webrev.00/ ) looks like > > > the best that can be done at the moment. I agree that it would be nicer if > > > jtreg would leave the jtreg lib path as java property to be able to > > > elevate > > > all of its contents. But the current proposal with a set of TEST_JARS of > > > jtreg, javatest and testng is probably sufficient for jaxp testing. > > > > > > The best thing to find out about other issues with the new version of > > > testng > > > would certainly be if Oracle's internal version of jtreg be updated to use > > > the latest testng :-) > > > > > > Best regards > > > Christoph > > > > > > > > > > > > > -----Original Message----- > > > > From: Volker Simonis [mailto:volker.simo...@gmail.com] > > > > Sent: Dienstag, 22. November 2016 20:25 > > > > To: Daniel Fuchs <daniel.fu...@oracle.com> > > > > Cc: Langer, Christoph <christoph.lan...@sap.com>; code-tools- > > > > d...@openjdk.java.net; core-libs-dev@openjdk.java.net; jtreg- > > > > u...@openjdk.java.net > > > > Subject: Re: Issues running JAXP jtreg tests > > > > ("java.lang.RuntimePermission" > > > > "accessDeclaredMembers") > > > > > > > > Hi Daniel, > > > > > > > > thanks for your patch! > > > > > > > > I've meanwhile tried to better understand the root cause of the problem. > > > > > > > > I don't think that the invocation order of the data provider the > > > > listener have changed. If you look at the the two version 6.9.5 and > > > > 6.9.13 of testng, the org.testng.TestRunner.run() methods look exactly > > > > the same in both 6.9.5 [1] and 6.9.13 [2]: > > > > > > > > public void run() { > > > > beforeRun(); > > > > > > > > try { > > > > XmlTest test= getTest(); > > > > if(test.isJUnit()) { > > > > privateRunJUnit(test); > > > > } > > > > else { > > > > privateRun(test); > > > > } > > > > } > > > > finally { > > > > afterRun(); > > > > } > > > > > > > > I think the problem is in > > > > org.testng.internal.ClassHelper.getAvailableMethods() where we testng > > > > only collected the methods until (i.e. excluding) java.lang.Object in > > > > 6.9.5 [3] but including java.lang.Object in 6.9.13 [4]: > > > > > > > > 6.9.5 > > > > ===== > > > > public static Set<Method> getAvailableMethods(Class<?> clazz) { > > > > Set<Method> methods = Sets.newHashSet(); > > > > methods.addAll(Arrays.asList(clazz.getDeclaredMethods())); > > > > > > > > Class<?> parent = clazz.getSuperclass(); > > > > while (Object.class != parent) { > > > > methods.addAll(extractMethods(clazz, parent, methods)); > > > > parent = parent.getSuperclass(); > > > > } > > > > > > > > 6.9.13 > > > > ===== > > > > public static Set<Method> getAvailableMethods(Class<?> clazz) { > > > > Set<Method> methods = Sets.newHashSet(); > > > > methods.addAll(Arrays.asList(clazz.getDeclaredMethods())); > > > > > > > > Class<?> parent = clazz.getSuperclass(); > > > > while (null != parent) { > > > > methods.addAll(extractMethods(clazz, parent, methods)); > > > > parent = parent.getSuperclass(); > > > > } > > > > > > > > But java.lang.Object has a different class loader (null) compared to > > > > the test class (which is loaded by the application class loader), > > > > which leads to the AccessControlException with 6.9.13. > > > > > > > > As far as I can see, this was changed in testng 6.9.10 [5] to fix > > > > https://github.com/cbeust/testng/issues/876 > > > > > > > > This behavior may potentially also affect other tests which are > > > > running with a security manger so I'm not sure you fix will help for > > > > all of them. And I also wonder why this hasn't been detected by other > > > > who run testng with a security manager (but maybe nobody is doing that > > > > :) > > > > > > > > Regards, > > > > Volker > > > > > > > > [1] https://github.com/cbeust/testng/blob/testng- > > > > 6.9.5/src/main/java/org/testng/TestRunner.java > > > > [2] > > > > > > https://github.com/cbeust/testng/blob/6.9.13/src/main/java/org/testng/TestRu > > > > nner.java > > > > [3] https://github.com/cbeust/testng/blob/testng- > > > > 6.9.5/src/main/java/org/testng/internal/ClassHelper.java > > > > [4] > > > > > > https://github.com/cbeust/testng/blob/6.9.13/src/main/java/org/testng/interna > > > > l/ClassHelper.java > > > > [5] > > > > > > https://github.com/cbeust/testng/pull/886/commits/fefedec34706e40ff2bf780b > > > > ff7716fca29daaab > > > > > > > > > > > > On Tue, Nov 22, 2016 at 5:24 PM, Daniel Fuchs <daniel.fu...@oracle.com> > > > > wrote: > > > > > Hi Christoph, > > > > > > > > > > I have logged https://bugs.openjdk.java.net/browse/JDK-8170192 > > > > > > > > > > best regards, > > > > > > > > > > -- daniel > > > > > > > > > > > > > > > On 22/11/16 14:47, Daniel Fuchs wrote: > > > > >> > > > > >> On 22/11/16 14:43, Langer, Christoph wrote: > > > > >>> > > > > >>> But, as for fixing with the new testng, will you look into this? > > > > >>> Shall > > > > >>> I open a bug? > > > > > > > > > >