Jenkins build is back to normal : SystemML-DailyTest #967

2017-05-03 Thread jenkins
See

Re: Standard code styles for DML and Java?

2017-05-03 Thread Deron Eriksson
Hi Matthias, I like your suggestion of space indentation for DML scripts and tab indentation for Java. I definitely support this and think this would be a great way to go. I also really like the idea of standardizing other aspects of our Java. For inline formatting in Java, we might want to use

Re: Sparse Matrix Storage Consumption Issue

2017-05-03 Thread Matthias Boehm
to summarize, this was an issue of selecting serialized representations for large ultra-sparse matrices. Thanks again for sharing your feedback with us. 1) In-memory representation: In CSR every non-zero will require 12 bytes - this is 240MB in your case. The overall memory consumption, howeve

Sparse Matrix Storage Consumption Issue

2017-05-03 Thread Mingyang Wang
Hi all, I was playing with a super sparse matrix FK, 2e7 by 1e6, with only one non-zero value on each row, that is 2e7 non-zero values in total. With driver memory of 1GB and executor memory of 100GB, I found the HOP "Spark chkpoint", which is used to pin the FK matrix in memory, is really expens

Re: Combining Multiple Matrix

2017-05-03 Thread arijit chakraborty
Thank you Fred! We are actually doing what you suggested. We were concerned about the large "null" matrix we are generating at the beginning of the process. But if large zero matrix is not a performance issue, then I guess we are on the right track. Thanks a lot! Arijit __

Build failed in Jenkins: SystemML-DailyTest #966

2017-05-03 Thread jenkins
See Changes: [npansar] [MINOR] Updated documentation and improved log messages [npansar] [MINOR] Show native library paths only when log4j is set debug or lower -- [...truncated 29