[jira] [Comment Edited] (SYSTEMML-776) Update SystemML to Support Spark 2.0.0

2016-08-01 Thread Glenn Weidner (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403266#comment-15403266
 ] 

Glenn Weidner edited comment on SYSTEMML-776 at 8/2/16 2:48 AM:


Submitted initial [PR 
#202|https://github.com/apache/incubator-systemml/pull/202].


was (Author: gweidner):
Submitted initial [PR|https://github.com/apache/incubator-systemml/pull/202].

> Update SystemML to Support Spark 2.0.0
> --
>
> Key: SYSTEMML-776
> URL: https://issues.apache.org/jira/browse/SYSTEMML-776
> Project: SystemML
>  Issue Type: Improvement
>Affects Versions: SystemML 0.9, SystemML 0.10, SystemML 0.11
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>Priority: Critical
>
> In the upcoming Spark 2.0.0 release, the {{DataFrame}} class has been changed 
> to a Scala type pointing to {{DataSet[Row]}}, with no {{DataFrame}} available 
> from Java.  Therefore, our current build is not compatible with this upcoming 
> release.  This can be tested by updating the pom to using {{2.0.0-preview}} 
> as the Spark version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-776) Update SystemML to Support Spark 2.0.0

2016-08-01 Thread Glenn Weidner (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403266#comment-15403266
 ] 

Glenn Weidner commented on SYSTEMML-776:


Submitted initial [PR|https://github.com/apache/incubator-systemml/pull/202].

> Update SystemML to Support Spark 2.0.0
> --
>
> Key: SYSTEMML-776
> URL: https://issues.apache.org/jira/browse/SYSTEMML-776
> Project: SystemML
>  Issue Type: Improvement
>Affects Versions: SystemML 0.9, SystemML 0.10, SystemML 0.11
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>Priority: Critical
>
> In the upcoming Spark 2.0.0 release, the {{DataFrame}} class has been changed 
> to a Scala type pointing to {{DataSet[Row]}}, with no {{DataFrame}} available 
> from Java.  Therefore, our current build is not compatible with this upcoming 
> release.  This can be tested by updating the pom to using {{2.0.0-preview}} 
> as the Spark version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-842) Remove useless javadoc comments and fix javadoc warnings

2016-08-01 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-842:
---

Assignee: Deron Eriksson

> Remove useless javadoc comments and fix javadoc warnings
> 
>
> Key: SYSTEMML-842
> URL: https://issues.apache.org/jira/browse/SYSTEMML-842
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>
> The project contains an enormous number of useless empty javadoc comments. In 
> addition, it has many other javadoc warnings.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-843) leftIndex and cache release extremely slow

2016-08-01 Thread Imran Younus (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Imran Younus updated SYSTEMML-843:
--
Description: 
I'm running the tSNE script in standalone mode with a subset of MNIST data 
(2500 points). I ran this with and without  `-exec singlenode`. Here are the 
stats:

(BTW, the same function implemented in python takes less than 10 sec!)

-> with singlenode flag

{code}
./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt
16/08/01 16:46:54 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 109.667 sec.
Total compilation time: 0.407 sec.
Total execution time:   109.260 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 223692/0/0/1.
Cache writes (WB, FS, HDFS):80351/0/2.
Cache times (ACQr/m, RLS, EXP): 0.289/0.015/85.192/0.043 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time:0.007 sec.
Functions recompiled:   1.
Functions recompile time:   0.039 sec.
Total JIT compile time: 4.924 sec.
Total JVM GC count: 312.
Total JVM GC time:  1.12 sec.
Heavy hitter instructions (name, time, count):
-- 1)   tsne109.202 sec 1
-- 2)   x2p 109.189 sec 1
-- 3)   leftIndex   106.728 sec 32136
-- 4)   tsmm0.564 sec   1
-- 5)   exp 0.376 sec   8034
-- 6)   rangeReIndex0.201 sec   40170
-- 7)   /   0.183 sec   24103
-- 8)   *   0.161 sec   16069
-- 9)   +   0.144 sec   22840
-- 10)  uak+0.106 sec   8036

16/08/01 16:46:54 INFO api.DMLScript: END DML run 08/01/2016 16:46:54
{code}

-> without singlenode flag
{code}
> ./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
> X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt


16/08/01 16:52:59 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 127.290 sec.
Total compilation time: 0.396 sec.
Total execution time:   126.894 sec.
Number of compiled MR Jobs: 1.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 223693/0/0/1.
Cache writes (WB, FS, HDFS):80352/0/2.
Cache times (ACQr/m, RLS, EXP): 0.421/0.016/100.974/0.041 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time:0.009 sec.
Functions recompiled:   1.
Functions recompile time:   0.038 sec.
Total JIT compile time: 4.835 sec.
Total JVM GC count: 312.
Total JVM GC time:  1.226 sec.
Heavy hitter instructions (name, time, count):
-- 1)   tsne126.426 sec 1
-- 2)   x2p 126.412 sec 1
-- 3)   leftIndex   123.982 sec 32136
-- 4)   exp 0.427 sec   8034
-- 5)   MR-Job_CSV_REBLOCK  0.412 sec   1
-- 6)   tsmm0.308 sec   1
-- 7)   rangeReIndex0.242 sec   40170
-- 8)   /   0.208 sec   24103
-- 9)   +   0.172 sec   22840
-- 10)  *   0.151 sec   16069

16/08/01 16:52:59 INFO api.DMLScript: END DML run 08/01/2016 16:52:59
{code}


  was:
I'm running the tSNE script in standalone mode with a subset of MNIST data 
(2500 points). I ran this with and without  `-exec singlenode`. Here are the 
stats:

-> with singlenode flag

{code}
./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt
16/08/01 16:46:54 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 109.667 sec.
Total compilation time: 0.407 sec.
Total execution time:   109.260 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 223692/0/0/1.
Cache writes (WB, FS, HDFS):80351/0/2.
Cache times (ACQr/m, RLS, EXP): 0.289/0.015/85.192/0.043 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time:0.007 sec.
Functions recompiled:   1.
Functions recompile time:   0.039 sec.
Total JIT compile time: 4.924 sec.
Total JVM GC count: 312.
Total JVM GC time:  1.12 sec.
Heavy hitter instructions (name, time, count):
-- 1)   tsne109.202 sec 1
-- 2)   x2p 109.189 sec 1
-- 3)   leftIndex   106.728 sec 32136
-- 4)   tsmm0.564 sec   1
-- 5)   exp 0.376 sec   8034
-- 6)   rangeReIndex0.201 sec   40170
-- 7)   /   0.183 sec   24103
-- 8)   *   0.161 sec   16069
-- 9)   +   0.144 sec   22840
-- 10)  uak+0.106 sec   8036

16/08/01 16:46:54 INFO api.DMLScript: END DML run 08/01/2016 16:46:54
{code}

-> without singlenode flag
{code}
> ./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
> X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt


16/08/01 16:52:59 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 

[jira] [Commented] (SYSTEMML-843) leftIndex and cache release extremely slow

2016-08-01 Thread Imran Younus (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403076#comment-15403076
 ] 

Imran Younus commented on SYSTEMML-843:
---

[~mboehm7] [~niketanpansare]


> leftIndex and cache release extremely slow
> --
>
> Key: SYSTEMML-843
> URL: https://issues.apache.org/jira/browse/SYSTEMML-843
> Project: SystemML
>  Issue Type: Bug
>Reporter: Imran Younus
> Attachments: tSNT.tar.gz
>
>
> I'm running the tSNE script in standalone mode with a subset of MNIST data 
> (2500 points). I ran this with and without  `-exec singlenode`. Here are the 
> stats:
> -> with singlenode flag
> {code}
> ./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
> X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt
> 16/08/01 16:46:54 INFO api.DMLScript: SystemML Statistics:
> Total elapsed time:   109.667 sec.
> Total compilation time:   0.407 sec.
> Total execution time: 109.260 sec.
> Number of compiled MR Jobs:   0.
> Number of executed MR Jobs:   0.
> Cache hits (Mem, WB, FS, HDFS):   223692/0/0/1.
> Cache writes (WB, FS, HDFS):  80351/0/2.
> Cache times (ACQr/m, RLS, EXP):   0.289/0.015/85.192/0.043 sec.
> HOP DAGs recompiled (PRED, SB):   0/0.
> HOP DAGs recompile time:  0.007 sec.
> Functions recompiled: 1.
> Functions recompile time: 0.039 sec.
> Total JIT compile time:   4.924 sec.
> Total JVM GC count:   312.
> Total JVM GC time:1.12 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) tsne109.202 sec 1
> -- 2) x2p 109.189 sec 1
> -- 3) leftIndex   106.728 sec 32136
> -- 4) tsmm0.564 sec   1
> -- 5) exp 0.376 sec   8034
> -- 6) rangeReIndex0.201 sec   40170
> -- 7) /   0.183 sec   24103
> -- 8) *   0.161 sec   16069
> -- 9) +   0.144 sec   22840
> -- 10)uak+0.106 sec   8036
> 16/08/01 16:46:54 INFO api.DMLScript: END DML run 08/01/2016 16:46:54
> {code}
> -> without singlenode flag
> {code}
> > ./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
> > X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt 
> > C=C_out.txt
> 16/08/01 16:52:59 INFO api.DMLScript: SystemML Statistics:
> Total elapsed time:   127.290 sec.
> Total compilation time:   0.396 sec.
> Total execution time: 126.894 sec.
> Number of compiled MR Jobs:   1.
> Number of executed MR Jobs:   0.
> Cache hits (Mem, WB, FS, HDFS):   223693/0/0/1.
> Cache writes (WB, FS, HDFS):  80352/0/2.
> Cache times (ACQr/m, RLS, EXP):   0.421/0.016/100.974/0.041 sec.
> HOP DAGs recompiled (PRED, SB):   0/0.
> HOP DAGs recompile time:  0.009 sec.
> Functions recompiled: 1.
> Functions recompile time: 0.038 sec.
> Total JIT compile time:   4.835 sec.
> Total JVM GC count:   312.
> Total JVM GC time:1.226 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) tsne126.426 sec 1
> -- 2) x2p 126.412 sec 1
> -- 3) leftIndex   123.982 sec 32136
> -- 4) exp 0.427 sec   8034
> -- 5) MR-Job_CSV_REBLOCK  0.412 sec   1
> -- 6) tsmm0.308 sec   1
> -- 7) rangeReIndex0.242 sec   40170
> -- 8) /   0.208 sec   24103
> -- 9) +   0.172 sec   22840
> -- 10)*   0.151 sec   16069
> 16/08/01 16:52:59 INFO api.DMLScript: END DML run 08/01/2016 16:52:59
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-843) leftIndex and cache release extremely slow

2016-08-01 Thread Imran Younus (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Imran Younus updated SYSTEMML-843:
--
Attachment: tSNT.tar.gz

data and code for tSNE test


> leftIndex and cache release extremely slow
> --
>
> Key: SYSTEMML-843
> URL: https://issues.apache.org/jira/browse/SYSTEMML-843
> Project: SystemML
>  Issue Type: Bug
>Reporter: Imran Younus
> Attachments: tSNT.tar.gz
>
>
> I'm running the tSNE script in standalone mode with a subset of MNIST data 
> (2500 points). I ran this with and without  `-exec singlenode`. Here are the 
> stats:
> -> with singlenode flag
> {code}
> ./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
> X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt
> 16/08/01 16:46:54 INFO api.DMLScript: SystemML Statistics:
> Total elapsed time:   109.667 sec.
> Total compilation time:   0.407 sec.
> Total execution time: 109.260 sec.
> Number of compiled MR Jobs:   0.
> Number of executed MR Jobs:   0.
> Cache hits (Mem, WB, FS, HDFS):   223692/0/0/1.
> Cache writes (WB, FS, HDFS):  80351/0/2.
> Cache times (ACQr/m, RLS, EXP):   0.289/0.015/85.192/0.043 sec.
> HOP DAGs recompiled (PRED, SB):   0/0.
> HOP DAGs recompile time:  0.007 sec.
> Functions recompiled: 1.
> Functions recompile time: 0.039 sec.
> Total JIT compile time:   4.924 sec.
> Total JVM GC count:   312.
> Total JVM GC time:1.12 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) tsne109.202 sec 1
> -- 2) x2p 109.189 sec 1
> -- 3) leftIndex   106.728 sec 32136
> -- 4) tsmm0.564 sec   1
> -- 5) exp 0.376 sec   8034
> -- 6) rangeReIndex0.201 sec   40170
> -- 7) /   0.183 sec   24103
> -- 8) *   0.161 sec   16069
> -- 9) +   0.144 sec   22840
> -- 10)uak+0.106 sec   8036
> 16/08/01 16:46:54 INFO api.DMLScript: END DML run 08/01/2016 16:46:54
> {code}
> -> without singlenode flag
> {code}
> > ./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
> > X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt 
> > C=C_out.txt
> 16/08/01 16:52:59 INFO api.DMLScript: SystemML Statistics:
> Total elapsed time:   127.290 sec.
> Total compilation time:   0.396 sec.
> Total execution time: 126.894 sec.
> Number of compiled MR Jobs:   1.
> Number of executed MR Jobs:   0.
> Cache hits (Mem, WB, FS, HDFS):   223693/0/0/1.
> Cache writes (WB, FS, HDFS):  80352/0/2.
> Cache times (ACQr/m, RLS, EXP):   0.421/0.016/100.974/0.041 sec.
> HOP DAGs recompiled (PRED, SB):   0/0.
> HOP DAGs recompile time:  0.009 sec.
> Functions recompiled: 1.
> Functions recompile time: 0.038 sec.
> Total JIT compile time:   4.835 sec.
> Total JVM GC count:   312.
> Total JVM GC time:1.226 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) tsne126.426 sec 1
> -- 2) x2p 126.412 sec 1
> -- 3) leftIndex   123.982 sec 32136
> -- 4) exp 0.427 sec   8034
> -- 5) MR-Job_CSV_REBLOCK  0.412 sec   1
> -- 6) tsmm0.308 sec   1
> -- 7) rangeReIndex0.242 sec   40170
> -- 8) /   0.208 sec   24103
> -- 9) +   0.172 sec   22840
> -- 10)*   0.151 sec   16069
> 16/08/01 16:52:59 INFO api.DMLScript: END DML run 08/01/2016 16:52:59
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-843) leftIndex and cache release extremely slow

2016-08-01 Thread Imran Younus (JIRA)
Imran Younus created SYSTEMML-843:
-

 Summary: leftIndex and cache release extremely slow
 Key: SYSTEMML-843
 URL: https://issues.apache.org/jira/browse/SYSTEMML-843
 Project: SystemML
  Issue Type: Bug
Reporter: Imran Younus


I'm running the tSNE script in standalone mode with a subset of MNIST data 
(2500 points). I ran this with and without  `-exec singlenode`. Here are the 
stats:

-> with singlenode flag

{code}
./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt
16/08/01 16:46:54 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 109.667 sec.
Total compilation time: 0.407 sec.
Total execution time:   109.260 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 223692/0/0/1.
Cache writes (WB, FS, HDFS):80351/0/2.
Cache times (ACQr/m, RLS, EXP): 0.289/0.015/85.192/0.043 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time:0.007 sec.
Functions recompiled:   1.
Functions recompile time:   0.039 sec.
Total JIT compile time: 4.924 sec.
Total JVM GC count: 312.
Total JVM GC time:  1.12 sec.
Heavy hitter instructions (name, time, count):
-- 1)   tsne109.202 sec 1
-- 2)   x2p 109.189 sec 1
-- 3)   leftIndex   106.728 sec 32136
-- 4)   tsmm0.564 sec   1
-- 5)   exp 0.376 sec   8034
-- 6)   rangeReIndex0.201 sec   40170
-- 7)   /   0.183 sec   24103
-- 8)   *   0.161 sec   16069
-- 9)   +   0.144 sec   22840
-- 10)  uak+0.106 sec   8036

16/08/01 16:46:54 INFO api.DMLScript: END DML run 08/01/2016 16:46:54
{code}

-> without singlenode flag
{code}
> ./bin/systemml scripts/staging/tSNE.dml -stats -nvargs 
> X=/home/iyounus/workspace/tsne_python/mnist2500_X.txt Y=Y_out.txt C=C_out.txt


16/08/01 16:52:59 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 127.290 sec.
Total compilation time: 0.396 sec.
Total execution time:   126.894 sec.
Number of compiled MR Jobs: 1.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 223693/0/0/1.
Cache writes (WB, FS, HDFS):80352/0/2.
Cache times (ACQr/m, RLS, EXP): 0.421/0.016/100.974/0.041 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time:0.009 sec.
Functions recompiled:   1.
Functions recompile time:   0.038 sec.
Total JIT compile time: 4.835 sec.
Total JVM GC count: 312.
Total JVM GC time:  1.226 sec.
Heavy hitter instructions (name, time, count):
-- 1)   tsne126.426 sec 1
-- 2)   x2p 126.412 sec 1
-- 3)   leftIndex   123.982 sec 32136
-- 4)   exp 0.427 sec   8034
-- 5)   MR-Job_CSV_REBLOCK  0.412 sec   1
-- 6)   tsmm0.308 sec   1
-- 7)   rangeReIndex0.242 sec   40170
-- 8)   /   0.208 sec   24103
-- 9)   +   0.172 sec   22840
-- 10)  *   0.151 sec   16069

16/08/01 16:52:59 INFO api.DMLScript: END DML run 08/01/2016 16:52:59
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-842) Remove useless javadoc comments and fix javadoc warnings

2016-08-01 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-842:
---

 Summary: Remove useless javadoc comments and fix javadoc warnings
 Key: SYSTEMML-842
 URL: https://issues.apache.org/jira/browse/SYSTEMML-842
 Project: SystemML
  Issue Type: Task
  Components: Documentation
Reporter: Deron Eriksson


The project contains an enormous number of useless empty javadoc comments. In 
addition, it has many other javadoc warnings.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-841) Create Java documentation for new MLContext

2016-08-01 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-841:
---

 Summary: Create Java documentation for new MLContext
 Key: SYSTEMML-841
 URL: https://issues.apache.org/jira/browse/SYSTEMML-841
 Project: SystemML
  Issue Type: Task
  Components: APIs, Documentation
Reporter: Deron Eriksson


Update MLContext programming guide Java section to use new MLContext



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-610) Add Databricks Notebook example to MLContext documentation

2016-08-01 Thread Madison J Myers (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402871#comment-15402871
 ] 

Madison J Myers commented on SYSTEMML-610:
--

LGTM

> Add Databricks Notebook example to MLContext documentation
> --
>
> Key: SYSTEMML-610
> URL: https://issues.apache.org/jira/browse/SYSTEMML-610
> Project: SystemML
>  Issue Type: Task
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>
> Add an example of configuring a Databricks Notebook to use SystemML to the 
> Spark MLContext Programming Guide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-824) Improve the performance of binary cell-wise operations

2016-08-01 Thread Mike Dusenberry (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402400#comment-15402400
 ] 

Mike Dusenberry commented on SYSTEMML-824:
--

+1 for increased performance

> Improve the performance of binary cell-wise operations
> --
>
> Key: SYSTEMML-824
> URL: https://issues.apache.org/jira/browse/SYSTEMML-824
> Project: SystemML
>  Issue Type: Task
>Reporter: Niketan Pansare
>
> The cellwise (matrix-matrix as well as matrix-scalar) operations take 
> significant amount of time while training Lenet. Here are few ways to improve 
> the performance of cell-wise operations:
> 1. Inject in-place updates [1] (saving on zero-ing out the matrix).
> 2. Fused cell-wise operations (as an example, recently added axpy operations: 
> https://github.com/apache/incubator-systemml/commit/b584aecf6b3a1eb96ff83b78cc3ad7c7c6d15baa).
>  
> 3. Parallelize cellwise operations (initial investigation need to be 
> conducted before proceeding in this direction especially in sparse case: 
> https://github.com/apache/incubator-systemml/blob/master/src/main/java/org/apache/sysml/runtime/matrix/data/LibMatrixBincell.java#L274).
> [~nakul02] [~mwdus...@us.ibm.com] [~prithvi_r_s] [~mboehm7] [~reinwald]
> Reference:
> [1] http://www.diku.dk/hjemmesider/ansatte/torbenm/ICD/Register.pdf



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-840) Create Zeppelin docs for new MLContext

2016-08-01 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-840:
---

 Summary: Create Zeppelin docs for new MLContext
 Key: SYSTEMML-840
 URL: https://issues.apache.org/jira/browse/SYSTEMML-840
 Project: SystemML
  Issue Type: Task
  Components: APIs, Documentation
Reporter: Deron Eriksson


Update MLContext programming guide Zeppelin section to use new MLContext



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-838) Create Spark Shell docs for new MLContext

2016-08-01 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-838:

Description: Update MLContext programming guide Spark Shell section to use 
new MLContext

> Create Spark Shell docs for new MLContext
> -
>
> Key: SYSTEMML-838
> URL: https://issues.apache.org/jira/browse/SYSTEMML-838
> Project: SystemML
>  Issue Type: Task
>  Components: APIs, Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>
> Update MLContext programming guide Spark Shell section to use new MLContext



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-839) Create Jupyter docs for new MLContext

2016-08-01 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-839:
---

 Summary: Create Jupyter docs for new MLContext
 Key: SYSTEMML-839
 URL: https://issues.apache.org/jira/browse/SYSTEMML-839
 Project: SystemML
  Issue Type: Task
  Components: APIs, Documentation
Reporter: Deron Eriksson


Update MLContext programming guide Jupyter section to use new MLContext



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-838) Create Spark Shell docs for new MLContext

2016-08-01 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-838:
---

 Summary: Create Spark Shell docs for new MLContext
 Key: SYSTEMML-838
 URL: https://issues.apache.org/jira/browse/SYSTEMML-838
 Project: SystemML
  Issue Type: Bug
  Components: APIs, Documentation
Reporter: Deron Eriksson
Assignee: Deron Eriksson






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-838) Create Spark Shell docs for new MLContext

2016-08-01 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-838:

Issue Type: Task  (was: Bug)

> Create Spark Shell docs for new MLContext
> -
>
> Key: SYSTEMML-838
> URL: https://issues.apache.org/jira/browse/SYSTEMML-838
> Project: SystemML
>  Issue Type: Task
>  Components: APIs, Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)