Earlier I updated MLContextProxy's getActiveMLContext to throw an MLContextException rather than return null if there wasn't an active MLContext (this is related to Spark Shell support). Unfortunately I didn't update SparkExecutionContext's initSparkContext to handle the new exception. I've made a change and I'll run the test suite and verify the tests work and then commit a hotfix if that fixes it.
Deron On Fri, Sep 9, 2016 at 4:01 PM, <[email protected]> wrote: > See <https://sparktc.ibmcloud.com/jenkins/job/SystemML- > DailyTest/495/changes> > > Changes: > > [Deron Eriksson] [SYSTEMML-902] Improve FrameObject toString output > > [Deron Eriksson] [SYSTEMML-900] Rename DataFrame ID column > > [Deron Eriksson] [SYSTEMML-901] Improve unavailable MLContext message > > ------------------------------------------ > [...truncated 152218 lines...] > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$2. > run(ParallelComputerBuilder.java:491) > >org.junit.runner.JUnitCore.run(JUnitCore.java:160) > >org.junit.runner.JUnitCore.run(JUnitCore.java:138) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun( > JUnitCoreWrapper.java:113) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > executeEager(JUnitCoreWrapper.java:85) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > execute(JUnitCoreWrapper.java:54) > >org.apache.maven.surefire.junitcore.JUnitCoreProvider. > invoke(JUnitCoreProvider.java:134) > >org.apache.maven.surefire.booter.ForkedBooter. > invokeProviderInSameClassLoader(ForkedBooter.java:200) > >org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess( > ForkedBooter.java:153) > >org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) > AppendMatrixTest.testMapAppendOutBlock2SparseSP > :115->commonAppendTest:259->AutomatedTestBase.runTest:1258 failed to run > script ./src/test/scripts/functions/append/AppendMatrixTest.dml > exception: org.apache.sysml.api.DMLException: No MLContext object is > currently active. Have you created one? Hint: in Scala, 'val ml = new > MLContext(sc)' > message: No MLContext object is currently active. Have you created one? > Hint: in Scala, 'val ml = new MLContext(sc)' > stack trace: > >org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:378) > >org.apache.sysml.api.DMLScript.main(DMLScript.java:199) > >org.apache.sysml.test.integration.AutomatedTestBase. > runTest(AutomatedTestBase.java:1238) > >org.apache.sysml.test.integration.functions.append.AppendMatrixTest. > commonAppendTest(AppendMatrixTest.java:259) > >org.apache.sysml.test.integration.functions.append.AppendMatrixTest. > testMapAppendOutBlock2SparseSP(AppendMatrixTest.java:115) > >sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > >sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > >java.lang.reflect.Method.invoke(Method.java:497) > >org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( > FrameworkMethod.java:47) > >org.junit.internal.runners.model.ReflectiveCallable.run( > ReflectiveCallable.java:12) > >org.junit.runners.model.FrameworkMethod.invokeExplosively( > FrameworkMethod.java:44) > >org.junit.internal.runners.statements.InvokeMethod. > evaluate(InvokeMethod.java:17) > >org.junit.internal.runners.statements.RunBefores. > evaluate(RunBefores.java:26) > >org.junit.internal.runners.statements.RunAfters.evaluate( > RunAfters.java:27) > >org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > >org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:70) > >org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:50) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.junit.runners.Suite.runChild(Suite.java:127) > >org.junit.runners.Suite.runChild(Suite.java:26) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.junit.runners.Suite.runChild(Suite.java:127) > >org.junit.runners.Suite.runChild(Suite.java:26) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$2. > run(ParallelComputerBuilder.java:491) > >org.junit.runner.JUnitCore.run(JUnitCore.java:160) > >org.junit.runner.JUnitCore.run(JUnitCore.java:138) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun( > JUnitCoreWrapper.java:113) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > executeEager(JUnitCoreWrapper.java:85) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > execute(JUnitCoreWrapper.java:54) > >org.apache.maven.surefire.junitcore.JUnitCoreProvider. > invoke(JUnitCoreProvider.java:134) > >org.apache.maven.surefire.booter.ForkedBooter. > invokeProviderInSameClassLoader(ForkedBooter.java:200) > >org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess( > ForkedBooter.java:153) > >org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) > > AppendMatrixTest.testAppendOutBlock1SparseSP:125->commonAppendTest:259->AutomatedTestBase.runTest:1258 > failed to run script ./src/test/scripts/functions/ > append/AppendMatrixTest.dml > exception: org.apache.sysml.api.DMLException: No MLContext object is > currently active. Have you created one? Hint: in Scala, 'val ml = new > MLContext(sc)' > message: No MLContext object is currently active. Have you created one? > Hint: in Scala, 'val ml = new MLContext(sc)' > stack trace: > >org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:378) > >org.apache.sysml.api.DMLScript.main(DMLScript.java:199) > >org.apache.sysml.test.integration.AutomatedTestBase. > runTest(AutomatedTestBase.java:1238) > >org.apache.sysml.test.integration.functions.append.AppendMatrixTest. > commonAppendTest(AppendMatrixTest.java:259) > >org.apache.sysml.test.integration.functions.append.AppendMatrixTest. > testAppendOutBlock1SparseSP(AppendMatrixTest.java:125) > >sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > >sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > >java.lang.reflect.Method.invoke(Method.java:497) > >org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( > FrameworkMethod.java:47) > >org.junit.internal.runners.model.ReflectiveCallable.run( > ReflectiveCallable.java:12) > >org.junit.runners.model.FrameworkMethod.invokeExplosively( > FrameworkMethod.java:44) > >org.junit.internal.runners.statements.InvokeMethod. > evaluate(InvokeMethod.java:17) > >org.junit.internal.runners.statements.RunBefores. > evaluate(RunBefores.java:26) > >org.junit.internal.runners.statements.RunAfters.evaluate( > RunAfters.java:27) > >org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > >org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:70) > >org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:50) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.junit.runners.Suite.runChild(Suite.java:127) > >org.junit.runners.Suite.runChild(Suite.java:26) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.junit.runners.Suite.runChild(Suite.java:127) > >org.junit.runners.Suite.runChild(Suite.java:26) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$2. > run(ParallelComputerBuilder.java:491) > >org.junit.runner.JUnitCore.run(JUnitCore.java:160) > >org.junit.runner.JUnitCore.run(JUnitCore.java:138) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun( > JUnitCoreWrapper.java:113) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > executeEager(JUnitCoreWrapper.java:85) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > execute(JUnitCoreWrapper.java:54) > >org.apache.maven.surefire.junitcore.JUnitCoreProvider. > invoke(JUnitCoreProvider.java:134) > >org.apache.maven.surefire.booter.ForkedBooter. > invokeProviderInSameClassLoader(ForkedBooter.java:200) > >org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess( > ForkedBooter.java:153) > >org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) > > AppendMatrixTest.testAppendInBlock1DenseSP:89->commonAppendTest:259->AutomatedTestBase.runTest:1258 > failed to run script ./src/test/scripts/functions/ > append/AppendMatrixTest.dml > exception: org.apache.sysml.api.DMLException: No MLContext object is > currently active. Have you created one? Hint: in Scala, 'val ml = new > MLContext(sc)' > message: No MLContext object is currently active. Have you created one? > Hint: in Scala, 'val ml = new MLContext(sc)' > stack trace: > >org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:378) > >org.apache.sysml.api.DMLScript.main(DMLScript.java:199) > >org.apache.sysml.test.integration.AutomatedTestBase. > runTest(AutomatedTestBase.java:1238) > >org.apache.sysml.test.integration.functions.append.AppendMatrixTest. > commonAppendTest(AppendMatrixTest.java:259) > >org.apache.sysml.test.integration.functions.append.AppendMatrixTest. > testAppendInBlock1DenseSP(AppendMatrixTest.java:89) > >sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > >sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > >java.lang.reflect.Method.invoke(Method.java:497) > >org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( > FrameworkMethod.java:47) > >org.junit.internal.runners.model.ReflectiveCallable.run( > ReflectiveCallable.java:12) > >org.junit.runners.model.FrameworkMethod.invokeExplosively( > FrameworkMethod.java:44) > >org.junit.internal.runners.statements.InvokeMethod. > evaluate(InvokeMethod.java:17) > >org.junit.internal.runners.statements.RunBefores. > evaluate(RunBefores.java:26) > >org.junit.internal.runners.statements.RunAfters.evaluate( > RunAfters.java:27) > >org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > >org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:70) > >org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:50) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.junit.runners.Suite.runChild(Suite.java:127) > >org.junit.runners.Suite.runChild(Suite.java:26) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.junit.runners.Suite.runChild(Suite.java:127) > >org.junit.runners.Suite.runChild(Suite.java:26) > >org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >org.apache.maven.surefire.junitcore.pc.Scheduler$1.run( > Scheduler.java:318) > >org.apache.maven.surefire.junitcore.pc.InvokerStrategy. > schedule(InvokerStrategy.java:41) > >org.apache.maven.surefire.junitcore.pc.Scheduler. > schedule(Scheduler.java:274) > >org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$2. > run(ParallelComputerBuilder.java:491) > >org.junit.runner.JUnitCore.run(JUnitCore.java:160) > >org.junit.runner.JUnitCore.run(JUnitCore.java:138) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun( > JUnitCoreWrapper.java:113) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > executeEager(JUnitCoreWrapper.java:85) > >org.apache.maven.surefire.junitcore.JUnitCoreWrapper. > execute(JUnitCoreWrapper.java:54) > >org.apache.maven.surefire.junitcore.JUnitCoreProvider. > invoke(JUnitCoreProvider.java:134) > >org.apache.maven.surefire.booter.ForkedBooter. > invokeProviderInSameClassLoader(ForkedBooter.java:200) > >org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess( > ForkedBooter.java:153) > >org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) > > Tests in error: > FrameTest.testCSVInTextOut » MLContext No MLContext object is currently > active... > FrameTest.testDataFrameInTextOut » MLContext No MLContext object is > currently ... > FrameTest.testTextInCSVOut » MLContext No MLContext object is currently > active... > FrameTest.testTextInTextOut » MLContext No MLContext object is currently > activ... > FrameTest.testDataFrameInDataFrameOut » MLContext No MLContext object > is curre... > FrameTest.testDataFrameInCSVOut » MLContext No MLContext object is > currently a... > FrameTest.testCSVInCSVOut » MLContext No MLContext object is currently > active.... > GNMFTest.testGNMFWithRDMLAndJava[0] » MLContext No MLContext object is > current... > GNMFTest.testGNMFWithRDMLAndJava[1] » MLContext No MLContext object is > current... > GNMFTest.testGNMFWithRDMLAndJava[2] » MLContext No MLContext object is > current... > GNMFTest.testGNMFWithRDMLAndJava[3] » MLContext No MLContext object is > current... > GNMFTest.testGNMFWithRDMLAndJava[4] » MLContext No MLContext object is > current... > GNMFTest.testGNMFWithRDMLAndJava[5] » MLContext No MLContext object is > current... > > FrameConverterTest.testFrameMixedMatrixBinSpark:167->runFrameConverterTest:264 > Runtime > FrameConverterTest.testFrameMixedCsvBinSpark:127->runFrameConverterTest:264 > Runtime > FrameConverterTest.testFrameStringsTxtCellBinSpar > k:142->runFrameConverterTest:264 Runtime > FrameConverterTest.testFrameStringsBinTxtCellSpar > k:152->runFrameConverterTest:264 Runtime > > FrameConverterTest.testFrameMixedBinDFrameSpark:197->runFrameConverterTest:264 > Runtime > > FrameConverterTest.testFrameMixedMultiColBlkBinMatrixSpark:187->runFrameConverterTest:264 > Runtime > FrameConverterTest.testFrameStringsBinMatrixSpark > :172->runFrameConverterTest:264 Runtime > > FrameConverterTest.testFrameStringsBinCsvSpark:132->runFrameConverterTest:264 > Runtime > > FrameConverterTest.testFrameMixedTxtCellBinSpark:147->runFrameConverterTest:264 > Runtime > > FrameConverterTest.testFrameMixedBinMatrixSpark:177->runFrameConverterTest:264 > Runtime > > FrameConverterTest.testFrameMixedMultiColBlkMatrixBinSpark:182->runFrameConverterTest:264 > Runtime > > FrameConverterTest.testFrameMixedBinTxtCellSpark:157->runFrameConverterTest:264 > Runtime > > FrameConverterTest.testFrameMixedDFrameBinSpark:192->runFrameConverterTest:264 > Runtime > FrameConverterTest.testFrameStringsMatrixBinSpark > :162->runFrameConverterTest:264 Runtime > > FrameConverterTest.testFrameStringsCsvBinSpark:122->runFrameConverterTest:264 > Runtime > FrameConverterTest.testFrameMixedBinCsvSpark:137->runFrameConverterTest:264 > Runtime > > Tests run: 5838, Failures: 1112, Errors: 29, Skipped: 0 > > [INFO] > [INFO] --- maven-failsafe-plugin:2.17:verify (default) @ systemml --- > [INFO] Failsafe report directory: <https://sparktc.ibmcloud.com/ > jenkins/job/SystemML-DailyTest/ws/target/failsafe-reports> > [INFO] ------------------------------------------------------------ > ------------ > [INFO] BUILD FAILURE > [INFO] ------------------------------------------------------------ > ------------ > [INFO] Total time: 02:05 h > [INFO] Finished at: 2016-09-09T18:01:55-05:00 > [INFO] Final Memory: 63M/2173M > [INFO] ------------------------------------------------------------ > ------------ > [ERROR] Failed to execute goal org.apache.maven.plugins: > maven-failsafe-plugin:2.17:verify (default) on project systemml: There > are test failures. > [ERROR] > [ERROR] Please refer to <https://sparktc.ibmcloud.com/ > jenkins/job/SystemML-DailyTest/ws/target/failsafe-reports> for the > individual test results. > [ERROR] -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the > -e switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, > please read the following articles: > [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ > MojoFailureException > Build step 'Execute shell' marked build as failure > Run condition [Always] enabling perform for step [[]] >
