Jenkins build is back to normal : SystemML-DailyTest #817

See

Re: Proposal to add 'accuracy test suite' before 1.0 release

There is also the possibility of writing the correctness tests completely in DML itself, thus allowing an ML researcher / data scientist to easily create the tests. For example, the SystemML-NN library has a full test suite written entirely in DML in the `nn/test/` directory (i.e. no Java tests)

Re: Proposal to add 'accuracy test suite' before 1.0 release

+1 for creating tests for the main algorithm scripts. This would be a great addition to the project. Note that the creation of tests (junit) typically requires some Java skills (and knowledge of ml algorithms) whereas a new algorithm script typically requires R/Python skills. Therefore, testing of

Re: Proposal to add 'accuracy test suite' before 1.0 release

+1 for testing our actual (vs simplified test version) scripts against some metric of choice. This will allow us to (1) ensure that each script does not have a showstopper bug (engine bug), and (2) that this script is still producing a reasonable mathematical result (math bug). -Mike -- Mike

Re: Proposal to add 'accuracy test suite' before 1.0 release

For now, I have updated our python mllearn tests to compare the prediction of our algorithm to that of scikit-learn: https://github.com/apache/incubator-systemml/blob/master/src/main/python/tests/test_mllearn_numpy.py#L81 The test now uses scikit-learn predictions as the baseline and computes the

Re: Operators in HOP DAG

Thanks for explaining, Matthias! Nantia 2017-02-17 22:01 GMT+02:00 Matthias Boehm : > ad 1: t(-*): ternary minus mult (for patterns like X-s*Y) > > ad 2: ua(+RC): unary aggregate with aggregation function + (at runtime > level you will see k+ for Kahan plus) and direction RC, i.e., full > aggreg

Re: Build failed in Jenkins: SystemML-DailyTest #816

yes, this is one of the flaky tests with occasional errors - unfortunately, even with the exact seeds of a failed run, this behavior is not reproducible locally. Regards, Matthias On Fri, Feb 17, 2017 at 10:16 AM, wrote: > > Failed tests: > > > > FrameMatrixReblockTest.testFrameWriteMultiple

Re: Operators in HOP DAG

ad 1: t(-*): ternary minus mult (for patterns like X-s*Y) ad 2: ua(+RC): unary aggregate with aggregation function + (at runtime level you will see k+ for Kahan plus) and direction RC, i.e., full aggregate over rows and columns. ad 3: lix: matrix or frame left indexing (for patterns like X[a:

Re: Proposal to add 'accuracy test suite' before 1.0 release

Yes, this has been discussed a couple of times now, most recently in SYSTEMML-546. It takes quite some effort though to create a sophisticated algorithm-level test suite as done for GLM. So by all means, please, go ahead and add these tests. However, I would not impose any constraints on the c

Proposal to add 'accuracy test suite' before 1.0 release

Hi all, We currently test the correctness of individual runtime operators using our integration tests but not the "released" algorithms. To be fair, we do test a subset of "simplified" algorithms on synthetic datasets and compare the accuracy with R. Also, we are testing subset of released algor

Re: Build failed in Jenkins: SystemML-DailyTest #816

> Failed tests: > > FrameMatrixReblockTest.testFrameWriteMultipleSparseBinarySpark:170->runFrameReblockTest:230 > 31 values are not in equal What's going on with this test and the associated logic that it is testing? Isn't this the same test that has been intermittently failing for a while n

Operators in HOP DAG

Hello, I generated a HOP plan using -explain, but I can't find the meaning of the following operators: 1. t(-*) 2. ua(+RC) 3. lix 4. u(cast_as_scalar) Thank you in advance, Nantia

Build failed in Jenkins: SystemML-DailyTest #816

See Changes: [npansar] [MINOR] Code refactoring MatrixIndexingSPInstruction to enable parallel [Deron Eriksson] [SYSTEMML-1280] Restore and deprecate SQLContext methods [Deron Eriksson] [SYSTEMML-1279] Decrease numCols t