[JIRA] (JENKINS-15157) Failure in @DataProvider method is not reported and build stays SUCCESS
nalin_makar resolved JENKINS-15157 as Not A Defect Failure in @DataProvider method is not reported and build stays SUCCESS Please configure your maven or ant task to fail on testng failures. This is not a testng plugin issue. Change By: nalin_makar (02/Jan/13 5:33 AM) Status: Open Resolved Resolution: NotADefect This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-15157) Failure in @DataProvider method is not reported and build stays SUCCESS
Ruslan Makhmudau created JENKINS-15157 Failure in @DataProvider method is not reported and build stays SUCCESS Issue Type: Bug Assignee: kutzi Components: junit Created: 13/Sep/12 2:47 PM Description: Occasionally we have some test which lets Surefire fail hard which Hudson doesn't recognize. I.e. the tests are not executed, but the build result is still success. E.g. we had a test which failed with a NoSuchFieldException in the constructor of the test class: Running TestSuite log4j:WARN No appenders could be found for logger (...). log4j:WARN Please initialize the log4j system properly. org.apache.maven.surefire.booter.SurefireExecutionException: Cannot instantiate class XyzTest; nested exception is org.testng.TestNGException: Cannot instantiate class XyzTest org.testng.TestNGException: Cannot instantiate class XyzTest at org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:35) at org.testng.internal.ClassHelper.createInstance(ClassHelper.java:330) at org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:62) at org.testng.internal.ClassImpl.getInstances(ClassImpl.java:81) at org.testng.internal.TestNGClassFinder.init(TestNGClassFinder.java:114) at org.testng.TestRunner.initMethods(TestRunner.java:289) at org.testng.TestRunner.init(TestRunner.java:235) at org.testng.TestRunner.init(TestRunner.java:197) at org.testng.TestRunner.init(TestRunner.java:141) at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:488) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:250) at org.testng.SuiteRunner.run(SuiteRunner.java:204) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:877) at org.testng.TestNG.runSuitesLocally(TestNG.java:842) at org.testng.TestNG.run(TestNG.java:751) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141) at org.apache.maven.surefire.Surefire.run(Surefire.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:26) ... 23 more Caused by: java.lang.NoSuchFieldException: cipherService at java.lang.Class.getDeclaredField(Class.java:1882) at XyzTest.init(XyzTest.java:91) ... 28 more ERROR There are test failures. Environment: Hudson 1.351, Maven 2.2.1, Surefire 2.5, TestNG 5.9 Jenkins 1.420 Project: Jenkins Priority: Critical Reporter: Ruslan Makhmudau This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see:
[JIRA] (JENKINS-15157) Failure in @DataProvider method is not reported and build stays SUCCESS
Wise updated JENKINS-15157 Failure in @DataProvider method is not reported and build stays SUCCESS Change By: Wise (13/Sep/12 2:51 PM) Assignee: kutzi nalin_makar Environment: Hudson1.351,Maven2.2.1,Surefire2.5, TestNG5.9Jenkins1. 420 478 Priority: Critical Major Description: OccasionallywehavesometestwhichletsSurefirefailhardwhichHudsondoesntrecognize. I .e. got the testsarenotexecuted followingexceptionDuringthe@DataProviderMethod ,but Jenkkinsdoesntfail thebuild resultisstillsuccess.E.g.wehadatestwhichfailedwithaNoSuchFieldExceptionintheconstructorofthetestclass : RunningTestSuitelog4j:WARNNoappenderscouldbefoundforlogger(...).log4j:WARNPleaseinitializethelog4jsystemproperly.org.apache.maven.surefire.booter.SurefireExecutionException:CannotinstantiateclassXyzTest;nestedexceptionisorg.testng.TestNGException:CannotinstantiateclassXyzTestorg.testng.TestNGException:CannotinstantiateclassXyzTest atorg.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl. java :35) atorg . testng.internal.ClassHelper.createInstance(ClassHelper.java:330) atorg.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:62) atorg.testng.internal.ClassImpl.getInstances(ClassImpl.java:81) atorg.testng.internal.TestNGClassFinder.init(TestNGClassFinder.java:114) atorg.testng.TestRunner.initMethods(TestRunner.java:289) atorg.testng.TestRunner.init(TestRunner.java:235) atorg.testng.TestRunner.init(TestRunner.java:197) atorg.testng.TestRunner.init(TestRunner.java:141) atorg.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:488) atorg.testng.SuiteRunner.privateRun(SuiteRunner.java:250) atorg.testng.SuiteRunner.run(SuiteRunner.java:204) atorg.testng.TestNG.createAndRunSuiteRunners(TestNG.java:877) atorg.testng.TestNG.runSuitesLocally(TestNG.java:842) atorg.testng.TestNG.run(TestNG.java:751) atorg.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62) atorg.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141) atorg.apache.maven.surefire.Surefire.run(Surefire.java:180) atsun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod) atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) atjava. lang. reflect.Method.invoke(Method.java RuntimeException : 597) atorg.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter. java :350) atorg . apache io . maven.surefire.booter.SurefireBooter.main(SurefireBooter.java FileNotFoundException : 1021)Causedby:java TestData\www . lang.reflect.InvocationTargetException atsun.reflect.NativeConstructorAccessorImpl.newInstance0 xls ( NativeMethod Thesystemcannotfindthepathspecified ) atsun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) atsun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) atjava.lang.reflect.Constructor.newInstance(Constructor.java:513) atorg.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:26) ...23moreCausedby:java.lang.NoSuchFieldException:cipherService atjava.lang.Class.getDeclaredField(Class.java:1882) atXyzTest.init(XyzTest.java:91) ...28more[ERROR]Therearetestfailures. Component/s: testng Component/s: junit This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
