[jira] [Closed] (SYSTEMML-544) Create Convenience Abstractions for MLContext

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-544.
---

> Create Convenience Abstractions for MLContext
> -
>
> Key: SYSTEMML-544
> URL: https://issues.apache.org/jira/browse/SYSTEMML-544
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs
>Reporter: Mike Dusenberry
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Currently, our {{MLContext}} API has good capabilities, but requires 
> boilerplate code that the end-user has to use.  This JIRA aims to hide some 
> of the boilerplate code under higher-level APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-544) Create Convenience Abstractions for MLContext

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-544.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

MLContext boilerplate code has now mostly been eliminated through introduction 
of the new MLContext API (SYSTEMML-593). Therefore I will close this JIRA since 
all issues have been addressed.

> Create Convenience Abstractions for MLContext
> -
>
> Key: SYSTEMML-544
> URL: https://issues.apache.org/jira/browse/SYSTEMML-544
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs
>Reporter: Mike Dusenberry
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Currently, our {{MLContext}} API has good capabilities, but requires 
> boilerplate code that the end-user has to use.  This JIRA aims to hide some 
> of the boilerplate code under higher-level APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-713) Create release process document

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-713.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

The release-process.md document addresses the main steps involved in validating 
a release. See http://apache.github.io/incubator-systemml/release-process.html 
. See [PR170|https://github.com/apache/incubator-systemml/pull/170].

This document should be updated to include how to build a release candidate and 
how to deploy a release candidate to Apache servers. This can be addressed in a 
separate JIRA.

> Create release process document
> ---
>
> Key: SYSTEMML-713
> URL: https://issues.apache.org/jira/browse/SYSTEMML-713
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Describe the SystemML release process in a document so that the release 
> validation consists of well-defined, documented, reproducible steps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-713) Create release process document

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-713.
---

> Create release process document
> ---
>
> Key: SYSTEMML-713
> URL: https://issues.apache.org/jira/browse/SYSTEMML-713
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Describe the SystemML release process in a document so that the release 
> validation consists of well-defined, documented, reproducible steps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-848) Describe release candidate build and deployment

2016-08-05 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-848:
---

 Summary: Describe release candidate build and deployment
 Key: SYSTEMML-848
 URL: https://issues.apache.org/jira/browse/SYSTEMML-848
 Project: SystemML
  Issue Type: Task
  Components: Build, Documentation
Reporter: Deron Eriksson
Priority: Minor


Describe the steps involved in building a release candidate and deploying the 
release candidate to Apache servers so that multiple project members can 
understand and help with the process.

Add these steps to "Release Candidate Build and Deployment" section of Release 
Process document (release-process.md). See
http://apache.github.io/incubator-systemml/release-process.html




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-848) Describe release candidate build and deployment

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-848:

Assignee: Luciano Resende

> Describe release candidate build and deployment
> ---
>
> Key: SYSTEMML-848
> URL: https://issues.apache.org/jira/browse/SYSTEMML-848
> Project: SystemML
>  Issue Type: Task
>  Components: Build, Documentation
>Reporter: Deron Eriksson
>Assignee: Luciano Resende
>Priority: Minor
>
> Describe the steps involved in building a release candidate and deploying the 
> release candidate to Apache servers so that multiple project members can 
> understand and help with the process.
> Add these steps to "Release Candidate Build and Deployment" section of 
> Release Process document (release-process.md). See
> http://apache.github.io/incubator-systemml/release-process.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-649) JMLC/MLContext support for scalar output variables

2016-08-05 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409817#comment-15409817
 ] 

Deron Eriksson commented on SYSTEMML-649:
-

Scalars supported by new MLContext API (SYSTEMML-593). Fixed by 
[PR199|https://github.com/apache/incubator-systemml/pull/199].

> JMLC/MLContext support for scalar output variables
> --
>
> Key: SYSTEMML-649
> URL: https://issues.apache.org/jira/browse/SYSTEMML-649
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
>
> Right now neither JMLC nor MLContext supports scalar output variables. This 
> task aims to extend both APIs with the required primitives.
> The workaround is to cast any output scalar on script-level with as.matrix to 
> a 1-1 matrix and handle it in the calling application. However, especially 
> with MLContext this puts an unnecessary burden on the user as he needs to 
> deal with RDDs for a simple scalar too. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-649) JMLC/MLContext support for scalar output variables

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-649.
---

> JMLC/MLContext support for scalar output variables
> --
>
> Key: SYSTEMML-649
> URL: https://issues.apache.org/jira/browse/SYSTEMML-649
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Right now neither JMLC nor MLContext supports scalar output variables. This 
> task aims to extend both APIs with the required primitives.
> The workaround is to cast any output scalar on script-level with as.matrix to 
> a 1-1 matrix and handle it in the calling application. However, especially 
> with MLContext this puts an unnecessary burden on the user as he needs to 
> deal with RDDs for a simple scalar too. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-649) JMLC/MLContext support for scalar output variables

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-649.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

> JMLC/MLContext support for scalar output variables
> --
>
> Key: SYSTEMML-649
> URL: https://issues.apache.org/jira/browse/SYSTEMML-649
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Right now neither JMLC nor MLContext supports scalar output variables. This 
> task aims to extend both APIs with the required primitives.
> The workaround is to cast any output scalar on script-level with as.matrix to 
> a 1-1 matrix and handle it in the calling application. However, especially 
> with MLContext this puts an unnecessary burden on the user as he needs to 
> deal with RDDs for a simple scalar too. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-56) Expand and organize documentation for R-like language syntax

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-56.
--

> Expand and organize documentation for R-like language syntax
> 
>
> Key: SYSTEMML-56
> URL: https://issues.apache.org/jira/browse/SYSTEMML-56
> Project: SystemML
>  Issue Type: Task
>Reporter: Frederick Reiss
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>
> This task covers making a complete pass through the reference manual for the 
> R-like (DML) language syntax. After this task is complete, the reference 
> manual should have complete coverage of all major language features and 
> should have usable indexing and a complete table of contents.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-56) Expand and organize documentation for R-like language syntax

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-56.

   Resolution: Fixed
Fix Version/s: SystemML 0.10

The DML Language Reference has been ported to markdown and has undergone 
significant improvements since August 2015. Therefore I will close this issue, 
which I probably should have closed many months ago.

> Expand and organize documentation for R-like language syntax
> 
>
> Key: SYSTEMML-56
> URL: https://issues.apache.org/jira/browse/SYSTEMML-56
> Project: SystemML
>  Issue Type: Task
>Reporter: Frederick Reiss
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>
> This task covers making a complete pass through the reference manual for the 
> R-like (DML) language syntax. After this task is complete, the reference 
> manual should have complete coverage of all major language features and 
> should have usable indexing and a complete table of contents.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-52) User Documentation Enhancement

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-52.

   Resolution: Fixed
Fix Version/s: SystemML 0.10

> User Documentation Enhancement
> --
>
> Key: SYSTEMML-52
> URL: https://issues.apache.org/jira/browse/SYSTEMML-52
> Project: SystemML
>  Issue Type: Epic
>Reporter: Frederick Reiss
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> Developer ramping up



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-52) User Documentation Enhancement

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-52.
--

> User Documentation Enhancement
> --
>
> Key: SYSTEMML-52
> URL: https://issues.apache.org/jira/browse/SYSTEMML-52
> Project: SystemML
>  Issue Type: Epic
>Reporter: Frederick Reiss
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> Developer ramping up



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-52) User Documentation Enhancement

2016-08-05 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409871#comment-15409871
 ] 

Deron Eriksson commented on SYSTEMML-52:


The documentation has significantly been expanded since this issue was created. 
We now have a README, Quick Start Guide, Spark MLContext Programming Guide, 
Hadoop Batch Mode Guide, JMLC Guide, DML Language Reference, Beginner's Guide 
to DML and PyDML, Algorithms Reference, Debugger Guide, IDE Guide, Contributing 
to SystemML Guide, Troubleshooting Guide, and Release Process Guide.

So, I will consider this issue resolved and closed.


> User Documentation Enhancement
> --
>
> Key: SYSTEMML-52
> URL: https://issues.apache.org/jira/browse/SYSTEMML-52
> Project: SystemML
>  Issue Type: Epic
>Reporter: Frederick Reiss
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> Developer ramping up



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-707) diag not generate square matrix if given Nx1 matrix of zeroes

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-707.
---

Verified that diag now generates a square matrix if given Nx1 matrix of zeroes

> diag not generate square matrix if given Nx1 matrix of zeroes
> -
>
> Key: SYSTEMML-707
> URL: https://issues.apache.org/jira/browse/SYSTEMML-707
> Project: SystemML
>  Issue Type: Bug
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Matthias Boehm
> Fix For: SystemML 0.11
>
>
> Thank you Matthew Plourde for finding this!
> If an Nx1 matrix of 0's is given to the diag() function, an Nx1 matrix of 0's 
> is returned. However, if an Nx1 matrix consists of any values that aren't 
> 0's, an NxN diagonal matrix is returned. This is inconsistent and the Nx1 
> matrix of 0's to diag() should probably return an NxN matrix.
> Example 1:
> {code}
> zeroes=matrix(0, 5, 1);
> print(toString(zeroes));
> print(toString(diag(zeroes)));
> {code}
> gives:
> {code}
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> {code}
> Example 2:
> {code}
> ones=matrix(1, 5, 1);
> print(toString(ones));
> print(toString(diag(ones)));
> {code}
> gives:
> {code}
> 1.000
> 1.000
> 1.000
> 1.000
> 1.000
> 1.000 0.000 0.000 0.000 0.000
> 0.000 1.000 0.000 0.000 0.000
> 0.000 0.000 1.000 0.000 0.000
> 0.000 0.000 0.000 1.000 0.000
> 0.000 0.000 0.000 0.000 1.000
> {code}
> Example 3:
> {code}
> nums=matrix("0 1 2 3 4", 5, 1);
> print(toString(nums));
> print(toString(diag(nums)));
> {code}
> gives:
> {code}
> 0.000
> 1.000
> 2.000
> 3.000
> 4.000
> 0.000 0.000 0.000 0.000 0.000
> 0.000 1.000 0.000 0.000 0.000
> 0.000 0.000 2.000 0.000 0.000
> 0.000 0.000 0.000 3.000 0.000
> 0.000 0.000 0.000 0.000 4.000
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-708) Release checklist for 0.10.0-incubating-rc1

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-708.
---

> Release checklist for 0.10.0-incubating-rc1
> ---
>
> Key: SYSTEMML-708
> URL: https://issues.apache.org/jira/browse/SYSTEMML-708
> Project: SystemML
>  Issue Type: Task
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> || Task || Status || Notes ||
> | All Artifacts and Checksums Present | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Windows   | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - OS X  | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Windows |{panel:bgColor=#bfffba}Pass{panel} | 
> SYSTEMML-712 opened for intermittent test failure |
> | Test Suite Passes - OS X| {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Linux   | {panel:bgColor=#bfffba}Pass{panel} | |
> | All Binaries Execute| {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Check LICENSE and NOTICE Files  | {panel:bgColor=#bfffba}Pass{panel} | 
> non-blocker, SYSTEMML-711 filed |
> | Src Artifact Builds and Tests Pass  | {panel:bgColor=#bfffba}Pass{panel} | 
> 5037 of 5038 passed on OS X (RightIndexingMatrixTest failed) |
> | Single-Node Standalone - Windows| {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - OS X   | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - Linux  | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Spark   | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X
> | Single-Node Hadoop  | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X
> | Notebooks - Jupyter | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Notebooks - Zeppelin| {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Performance Suite - Spark   | {panel:bgColor=#bfffba}Pass{panel} | 
> Run on Spark 1.6.1 for data sizes {80MB, 800MB, 8GB, 80GB} |
> | Performance Suite - Hadoop  | | |



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-708) Release checklist for 0.10.0-incubating-rc1

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-708.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.10

> Release checklist for 0.10.0-incubating-rc1
> ---
>
> Key: SYSTEMML-708
> URL: https://issues.apache.org/jira/browse/SYSTEMML-708
> Project: SystemML
>  Issue Type: Task
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> || Task || Status || Notes ||
> | All Artifacts and Checksums Present | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Windows   | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - OS X  | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Windows |{panel:bgColor=#bfffba}Pass{panel} | 
> SYSTEMML-712 opened for intermittent test failure |
> | Test Suite Passes - OS X| {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Linux   | {panel:bgColor=#bfffba}Pass{panel} | |
> | All Binaries Execute| {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Check LICENSE and NOTICE Files  | {panel:bgColor=#bfffba}Pass{panel} | 
> non-blocker, SYSTEMML-711 filed |
> | Src Artifact Builds and Tests Pass  | {panel:bgColor=#bfffba}Pass{panel} | 
> 5037 of 5038 passed on OS X (RightIndexingMatrixTest failed) |
> | Single-Node Standalone - Windows| {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - OS X   | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - Linux  | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Spark   | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X
> | Single-Node Hadoop  | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X
> | Notebooks - Jupyter | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Notebooks - Zeppelin| {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Performance Suite - Spark   | {panel:bgColor=#bfffba}Pass{panel} | 
> Run on Spark 1.6.1 for data sizes {80MB, 800MB, 8GB, 80GB} |
> | Performance Suite - Hadoop  | | |



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-708) Release checklist for 0.10.0-incubating-rc1

2016-08-05 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409894#comment-15409894
 ] 

Deron Eriksson commented on SYSTEMML-708:
-

Note that RC1 was rejected due to unexpected binaries in the src release. This 
was fixed in RC2.

> Release checklist for 0.10.0-incubating-rc1
> ---
>
> Key: SYSTEMML-708
> URL: https://issues.apache.org/jira/browse/SYSTEMML-708
> Project: SystemML
>  Issue Type: Task
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> || Task || Status || Notes ||
> | All Artifacts and Checksums Present | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Windows   | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - OS X  | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Windows |{panel:bgColor=#bfffba}Pass{panel} | 
> SYSTEMML-712 opened for intermittent test failure |
> | Test Suite Passes - OS X| {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Linux   | {panel:bgColor=#bfffba}Pass{panel} | |
> | All Binaries Execute| {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Check LICENSE and NOTICE Files  | {panel:bgColor=#bfffba}Pass{panel} | 
> non-blocker, SYSTEMML-711 filed |
> | Src Artifact Builds and Tests Pass  | {panel:bgColor=#bfffba}Pass{panel} | 
> 5037 of 5038 passed on OS X (RightIndexingMatrixTest failed) |
> | Single-Node Standalone - Windows| {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - OS X   | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - Linux  | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Spark   | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X
> | Single-Node Hadoop  | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X
> | Notebooks - Jupyter | {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Notebooks - Zeppelin| {panel:bgColor=#bfffba}Pass{panel} | 
> Verified on OS X |
> | Performance Suite - Spark   | {panel:bgColor=#bfffba}Pass{panel} | 
> Run on Spark 1.6.1 for data sizes {80MB, 800MB, 8GB, 80GB} |
> | Performance Suite - Hadoop  | | |



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-478) Upgrade maven-failsafe-plugin if fixed

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-478.
---

> Upgrade maven-failsafe-plugin if fixed
> --
>
> Key: SYSTEMML-478
> URL: https://issues.apache.org/jira/browse/SYSTEMML-478
> Project: SystemML
>  Issue Type: Improvement
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> The maven-failsafe-plugin entry in pom.xml is set to version 2.17 due to:
> "Failsafe 2.18 has a bug in computing # cores, so use 2.17"
> The maven central repo has the following later versions of 
> maven-failsafe-plugin available:
>   2.19.1
>   2.19
>   2.18.1
> We should determine if a later version of the plugin can be used and the 
> "HACK ALERT" message can be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-478) Upgrade maven-failsafe-plugin if fixed

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-478.
-
   Resolution: Won't Fix
 Assignee: Deron Eriksson
Fix Version/s: SystemML 0.11

Since issue seems to currently exist with all versions later than 2.17, we 
shouldn't upgrade this plugin for now.

> Upgrade maven-failsafe-plugin if fixed
> --
>
> Key: SYSTEMML-478
> URL: https://issues.apache.org/jira/browse/SYSTEMML-478
> Project: SystemML
>  Issue Type: Improvement
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> The maven-failsafe-plugin entry in pom.xml is set to version 2.17 due to:
> "Failsafe 2.18 has a bug in computing # cores, so use 2.17"
> The maven central repo has the following later versions of 
> maven-failsafe-plugin available:
>   2.19.1
>   2.19
>   2.18.1
> We should determine if a later version of the plugin can be used and the 
> "HACK ALERT" message can be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-458) Create Spark batch mode guide

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-458:

Assignee: Mike Dusenberry

> Create Spark batch mode guide
> -
>
> Key: SYSTEMML-458
> URL: https://issues.apache.org/jira/browse/SYSTEMML-458
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
>
> Spark batch mode is one of the primary execution modes of SystemML. Document 
> is needed similar to Hadoop Batch Mode document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-457) Create Standalone mode guide

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-457:

Assignee: Mike Dusenberry

> Create Standalone mode guide
> 
>
> Key: SYSTEMML-457
> URL: https://issues.apache.org/jira/browse/SYSTEMML-457
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
>
> Create a document that focuses on Standalone mode execution. Documentation 
> has recently been reorganized to emphasize the various execution modes.
> Quick Start Guide can be used as a starting point for this. Quick Start Guide 
> has a bit of a broader focus than only Standalone mode.
> Topics to include:
> Download release
> Standalone on Linux, Mac, Windows
> Hello World
> Algorithm examples
> Execution on single machine, Hadoop, and Spark.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-836) Create ScriptFactory convenience methods for resources on classpath

2016-08-05 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-836:
---

Assignee: Deron Eriksson

> Create ScriptFactory convenience methods for resources on classpath
> ---
>
> Key: SYSTEMML-836
> URL: https://issues.apache.org/jira/browse/SYSTEMML-836
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
>
> Currently a DML Script object can be created by ScriptFactory by getting an 
> input stream to a resource on the classpath, such as this example:
> {code}
> val inputStream = 
> getClass.getResourceAsStream("/scripts/algorithms/Univar-Stats.dml")
> val script = ScriptFactory.dmlFromInputStream(inputStream)
> {code}
> This can be further simplified by creating a ScriptFactory method like 
> dmlFromResourcePath (or dmlFromClasspath) so a user could instead do:
> {code}
> val script = 
> ScriptFactory.dmlFromResourcePath("/scripts/algorithms/Univar-Stats.dml")
> {code}
> An addition method should be created for PyDML, such as pydmlFromResourcePath 
> (or pydmlFromClasspath).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-458) Create Spark batch mode guide

2016-08-05 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410234#comment-15410234
 ] 

Deron Eriksson commented on SYSTEMML-458:
-

Thank you [~mwdus...@us.ibm.com]

> Create Spark batch mode guide
> -
>
> Key: SYSTEMML-458
> URL: https://issues.apache.org/jira/browse/SYSTEMML-458
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.11
>
>
> Spark batch mode is one of the primary execution modes of SystemML. Document 
> is needed similar to Hadoop Batch Mode document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-457) Create Standalone mode guide

2016-08-05 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410235#comment-15410235
 ] 

Deron Eriksson commented on SYSTEMML-457:
-

Thank you [~mwdus...@us.ibm.com]

> Create Standalone mode guide
> 
>
> Key: SYSTEMML-457
> URL: https://issues.apache.org/jira/browse/SYSTEMML-457
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.11
>
>
> Create a document that focuses on Standalone mode execution. Documentation 
> has recently been reorganized to emphasize the various execution modes.
> Quick Start Guide can be used as a starting point for this. Quick Start Guide 
> has a bit of a broader focus than only Standalone mode.
> Topics to include:
> Download release
> Standalone on Linux, Mac, Windows
> Hello World
> Algorithm examples
> Execution on single machine, Hadoop, and Spark.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-851) MLContext source from URLs

2016-08-08 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-851:
---

 Summary: MLContext source from URLs
 Key: SYSTEMML-851
 URL: https://issues.apache.org/jira/browse/SYSTEMML-851
 Project: SystemML
  Issue Type: Improvement
  Components: APIs
Reporter: Deron Eriksson


The new MLContext API allows scripts to be created from URLs. Additionally, it 
should allow scripts or groups of scripts to be imported/sourced from URLs.

For example, there should be a mechanism to pull in the SystemML-NN package of 
DML scripts from a URL. Preferably, a user can do this with a single line of 
code (specify a URL to a script and that script and all sourced scripts get 
pulled in).





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-852) MLContext API javadocs online

2016-08-08 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-852:
---

 Summary: MLContext API javadocs online
 Key: SYSTEMML-852
 URL: https://issues.apache.org/jira/browse/SYSTEMML-852
 Project: SystemML
  Issue Type: Task
  Components: APIs, Documentation
Reporter: Deron Eriksson
Assignee: Deron Eriksson


The new MLContext API has a significant amount of Javadoc documentation 
associated with it. This documentation needs to be available online for users.

A determination needs to be made where this documentation should be placed. 
Possibilities include the main site and the documentation site on GitHub.

In addition, these Javadocs should be included in the documentation for each 
version of SystemML that is released.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-853) Create Python wrapper API for new MLContext API

2016-08-09 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-853:
---

 Summary: Create Python wrapper API for new MLContext API
 Key: SYSTEMML-853
 URL: https://issues.apache.org/jira/browse/SYSTEMML-853
 Project: SystemML
  Issue Type: Task
  Components: APIs
Reporter: Deron Eriksson
Assignee: Deron Eriksson


The new MLContext API needs a Python wrapper API to access MLContext from 
Python. This will allow Jupyter notebooks to use the new API.

The old Python API is located at 
src/main/java/org/apache/sysml/api/python/SystemML.py

The new MLContext API is located in the org.apache.sysml.api.mlcontext package.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-853) Create Python wrapper API for new MLContext API

2016-08-09 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-853:

Description: 
The new MLContext API needs a Python wrapper API to access MLContext from 
Python. This will allow Jupyter notebooks to use the new API.

The old Python API is located at 
src/main/java/org/apache/sysml/api/python/SystemML.py

The new MLContext API is located in the org.apache.sysml.api.mlcontext package.

Examples of using the new MLContext API from Spark Shell can be found at 
http://apache.github.io/incubator-systemml/spark-mlcontext-programming-guide.html


  was:
The new MLContext API needs a Python wrapper API to access MLContext from 
Python. This will allow Jupyter notebooks to use the new API.

The old Python API is located at 
src/main/java/org/apache/sysml/api/python/SystemML.py

The new MLContext API is located in the org.apache.sysml.api.mlcontext package.



> Create Python wrapper API for new MLContext API
> ---
>
> Key: SYSTEMML-853
> URL: https://issues.apache.org/jira/browse/SYSTEMML-853
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>
> The new MLContext API needs a Python wrapper API to access MLContext from 
> Python. This will allow Jupyter notebooks to use the new API.
> The old Python API is located at 
> src/main/java/org/apache/sysml/api/python/SystemML.py
> The new MLContext API is located in the org.apache.sysml.api.mlcontext 
> package.
> Examples of using the new MLContext API from Spark Shell can be found at 
> http://apache.github.io/incubator-systemml/spark-mlcontext-programming-guide.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-854) Change Python MLContext to support new MLContext

2016-08-09 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-854.
-
   Resolution: Duplicate
Fix Version/s: SystemML 0.11

Duplicate of SYSTEMML-853

> Change Python MLContext to support new MLContext
> 
>
> Key: SYSTEMML-854
> URL: https://issues.apache.org/jira/browse/SYSTEMML-854
> Project: SystemML
>  Issue Type: Task
>Reporter: Niketan Pansare
> Fix For: SystemML 0.11
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-854) Change Python MLContext to support new MLContext

2016-08-09 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-854.
---

> Change Python MLContext to support new MLContext
> 
>
> Key: SYSTEMML-854
> URL: https://issues.apache.org/jira/browse/SYSTEMML-854
> Project: SystemML
>  Issue Type: Task
>Reporter: Niketan Pansare
> Fix For: SystemML 0.11
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-856) Add Gemfile to docs for Bundler

2016-08-09 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-856:
---

 Summary: Add Gemfile to docs for Bundler
 Key: SYSTEMML-856
 URL: https://issues.apache.org/jira/browse/SYSTEMML-856
 Project: SystemML
  Issue Type: Task
  Components: Documentation
Reporter: Deron Eriksson
Assignee: Deron Eriksson
Priority: Minor


GitHub describes setting up a documentation site locally using Bundler. See 
https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/

SystemML docs also describe generating documentation site using Bundler. See 
http://apache.github.io/incubator-systemml/contributing-to-systemml.html

Bundler would like a Gemfile in the docs folder.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-856) Add Gemfile to docs for Bundler

2016-08-09 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-856:

Assignee: Mike Dusenberry  (was: Deron Eriksson)

> Add Gemfile to docs for Bundler
> ---
>
> Key: SYSTEMML-856
> URL: https://issues.apache.org/jira/browse/SYSTEMML-856
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
>Priority: Minor
>
> GitHub describes setting up a documentation site locally using Bundler. See 
> https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/
> SystemML docs also describe generating documentation site using Bundler. See 
> http://apache.github.io/incubator-systemml/contributing-to-systemml.html
> Bundler would like a Gemfile in the docs folder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-856) Add Gemfile to docs for Bundler

2016-08-09 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-856.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

Fixed by commit 
https://github.com/apache/incubator-systemml/commit/24d57ad2a170d2f9ee34bca7ee896c728e555f1c

> Add Gemfile to docs for Bundler
> ---
>
> Key: SYSTEMML-856
> URL: https://issues.apache.org/jira/browse/SYSTEMML-856
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> GitHub describes setting up a documentation site locally using Bundler. See 
> https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/
> SystemML docs also describe generating documentation site using Bundler. See 
> http://apache.github.io/incubator-systemml/contributing-to-systemml.html
> Bundler would like a Gemfile in the docs folder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-856) Add Gemfile to docs for Bundler

2016-08-09 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-856.
---

> Add Gemfile to docs for Bundler
> ---
>
> Key: SYSTEMML-856
> URL: https://issues.apache.org/jira/browse/SYSTEMML-856
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> GitHub describes setting up a documentation site locally using Bundler. See 
> https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/
> SystemML docs also describe generating documentation site using Bundler. See 
> http://apache.github.io/incubator-systemml/contributing-to-systemml.html
> Bundler would like a Gemfile in the docs folder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-853) Create Python wrapper API for new MLContext API

2016-08-09 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-853:

Assignee: Manoj Kumar  (was: Deron Eriksson)

> Create Python wrapper API for new MLContext API
> ---
>
> Key: SYSTEMML-853
> URL: https://issues.apache.org/jira/browse/SYSTEMML-853
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Manoj Kumar
>
> The new MLContext API needs a Python wrapper API to access MLContext from 
> Python. This will allow Jupyter notebooks to use the new API.
> The old Python API is located at 
> src/main/java/org/apache/sysml/api/python/SystemML.py
> The new MLContext API is located in the org.apache.sysml.api.mlcontext 
> package.
> Examples of using the new MLContext API from Spark Shell can be found at 
> http://apache.github.io/incubator-systemml/spark-mlcontext-programming-guide.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-857) Describe release process for documentation

2016-08-10 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-857:
---

 Summary: Describe release process for documentation
 Key: SYSTEMML-857
 URL: https://issues.apache.org/jira/browse/SYSTEMML-857
 Project: SystemML
  Issue Type: Task
  Components: Documentation
Reporter: Deron Eriksson
Assignee: Deron Eriksson


The project documentation (content of docs folder, javadocs, etc) needs to be 
generated for each release. In addition to generation with correct version 
numbers, the documentation needs to be deployed to the main website. This 
documentation handling needs to be included in the official project release 
process.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-858) Add version number to global documentation header

2016-08-10 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-858:
---

 Summary: Add version number to global documentation header
 Key: SYSTEMML-858
 URL: https://issues.apache.org/jira/browse/SYSTEMML-858
 Project: SystemML
  Issue Type: Task
  Components: Documentation
Reporter: Deron Eriksson
Assignee: Deron Eriksson


Add the version number (SYSTEMML_VERSION) from _config.yml to the global.html 
documentation template header section so that the documentation version number 
can visually be seen in the header by our users. This will help prevent 
mismatches between SystemML project versions and SystemML project documentation 
versions.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-853) Create Python wrapper API for new MLContext API

2016-08-11 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417596#comment-15417596
 ] 

Deron Eriksson commented on SYSTEMML-853:
-

Thank you [~MechCoder]!

> Create Python wrapper API for new MLContext API
> ---
>
> Key: SYSTEMML-853
> URL: https://issues.apache.org/jira/browse/SYSTEMML-853
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Manoj Kumar
>
> The new MLContext API needs a Python wrapper API to access MLContext from 
> Python. This will allow Jupyter notebooks to use the new API.
> The old Python API is located at 
> src/main/java/org/apache/sysml/api/python/SystemML.py
> The new MLContext API is located in the org.apache.sysml.api.mlcontext 
> package.
> Examples of using the new MLContext API from Spark Shell can be found at 
> http://apache.github.io/incubator-systemml/spark-mlcontext-programming-guide.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-857) Describe release process for documentation

2016-08-11 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-857.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

Fixed by [PR208|https://github.com/apache/incubator-systemml/pull/208].

> Describe release process for documentation
> --
>
> Key: SYSTEMML-857
> URL: https://issues.apache.org/jira/browse/SYSTEMML-857
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> The project documentation (content of docs folder, javadocs, etc) needs to be 
> generated for each release. In addition to generation with correct version 
> numbers, the documentation needs to be deployed to the main website. This 
> documentation handling needs to be included in the official project release 
> process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-858) Add version number to global documentation header

2016-08-11 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-858.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

Fixed by [PR208|https://github.com/apache/incubator-systemml/pull/208].

> Add version number to global documentation header
> -
>
> Key: SYSTEMML-858
> URL: https://issues.apache.org/jira/browse/SYSTEMML-858
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Add the version number (SYSTEMML_VERSION) from _config.yml to the global.html 
> documentation template header section so that the documentation version 
> number can visually be seen in the header by our users. This will help 
> prevent mismatches between SystemML project versions and SystemML project 
> documentation versions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-858) Add version number to global documentation header

2016-08-11 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-858.
---

> Add version number to global documentation header
> -
>
> Key: SYSTEMML-858
> URL: https://issues.apache.org/jira/browse/SYSTEMML-858
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Add the version number (SYSTEMML_VERSION) from _config.yml to the global.html 
> documentation template header section so that the documentation version 
> number can visually be seen in the header by our users. This will help 
> prevent mismatches between SystemML project versions and SystemML project 
> documentation versions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-857) Describe release process for documentation

2016-08-11 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-857.
---

> Describe release process for documentation
> --
>
> Key: SYSTEMML-857
> URL: https://issues.apache.org/jira/browse/SYSTEMML-857
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> The project documentation (content of docs folder, javadocs, etc) needs to be 
> generated for each release. In addition to generation with correct version 
> numbers, the documentation needs to be deployed to the main website. This 
> documentation handling needs to be included in the official project release 
> process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-836) Create ScriptFactory convenience methods for resources on classpath

2016-08-11 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-836.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

Fixed by [PR207|https://github.com/apache/incubator-systemml/pull/207].

> Create ScriptFactory convenience methods for resources on classpath
> ---
>
> Key: SYSTEMML-836
> URL: https://issues.apache.org/jira/browse/SYSTEMML-836
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> Currently a DML Script object can be created by ScriptFactory by getting an 
> input stream to a resource on the classpath, such as this example:
> {code}
> val inputStream = 
> getClass.getResourceAsStream("/scripts/algorithms/Univar-Stats.dml")
> val script = ScriptFactory.dmlFromInputStream(inputStream)
> {code}
> This can be further simplified by creating a ScriptFactory method like 
> dmlFromResourcePath (or dmlFromClasspath) so a user could instead do:
> {code}
> val script = 
> ScriptFactory.dmlFromResourcePath("/scripts/algorithms/Univar-Stats.dml")
> {code}
> An addition method should be created for PyDML, such as pydmlFromResourcePath 
> (or pydmlFromClasspath).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-836) Create ScriptFactory convenience methods for resources on classpath

2016-08-11 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-836.
---

> Create ScriptFactory convenience methods for resources on classpath
> ---
>
> Key: SYSTEMML-836
> URL: https://issues.apache.org/jira/browse/SYSTEMML-836
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> Currently a DML Script object can be created by ScriptFactory by getting an 
> input stream to a resource on the classpath, such as this example:
> {code}
> val inputStream = 
> getClass.getResourceAsStream("/scripts/algorithms/Univar-Stats.dml")
> val script = ScriptFactory.dmlFromInputStream(inputStream)
> {code}
> This can be further simplified by creating a ScriptFactory method like 
> dmlFromResourcePath (or dmlFromClasspath) so a user could instead do:
> {code}
> val script = 
> ScriptFactory.dmlFromResourcePath("/scripts/algorithms/Univar-Stats.dml")
> {code}
> An addition method should be created for PyDML, such as pydmlFromResourcePath 
> (or pydmlFromClasspath).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-823) Update QuickStart guide univariate statistics description

2016-08-11 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-823.
---

> Update QuickStart guide univariate statistics description
> -
>
> Key: SYSTEMML-823
> URL: https://issues.apache.org/jira/browse/SYSTEMML-823
> Project: SystemML
>  Issue Type: Bug
>  Components: Documentation
>Reporter: Glenn Weidner
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> A table was removed and corresponding text needs to be updated appropriately. 
>  In addition, a line needs to be removed from bottom of page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-823) Update QuickStart guide univariate statistics description

2016-08-12 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-823.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

Fixed by [PR209|https://github.com/apache/incubator-systemml/pull/209].

> Update QuickStart guide univariate statistics description
> -
>
> Key: SYSTEMML-823
> URL: https://issues.apache.org/jira/browse/SYSTEMML-823
> Project: SystemML
>  Issue Type: Bug
>  Components: Documentation
>Reporter: Glenn Weidner
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> A table was removed and corresponding text needs to be updated appropriately. 
>  In addition, a line needs to be removed from bottom of page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-474) Read matrix from URL

2016-08-12 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-474:
---

Assignee: Deron Eriksson

> Read matrix from URL
> 
>
> Key: SYSTEMML-474
> URL: https://issues.apache.org/jira/browse/SYSTEMML-474
> Project: SystemML
>  Issue Type: Wish
>  Components: Runtime
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>
> The ability to read a matrix from a URL can increase usability of SystemML by 
> eliminating the need to have already existing local or cluster data when 
> invoking SystemML.
> For instance, rather than doing something like:
> {code}
> wget http://example.com/ml/matrix.csv
> ./runStandaloneSystemML.sh example.dml -nvargs X=matrix.csv
> {code}
> Someone could do:
> {code}
> ./runStandaloneSystemML.sh example.dml -nvargs 
> X=http://example.com/ml/matrix.csv
> {code}
> One way that this can be useful is to have a small set of example data for 
> each algorithm sitting on a server. Documentation could show a single command 
> that can be invoked for each algorithm that references the server-based 
> example data. This would mean that a user can essentially copy/paste a single 
> command from the documentation, and it would run using the server example 
> data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-474) Read matrix from URL

2016-08-12 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419188#comment-15419188
 ] 

Deron Eriksson commented on SYSTEMML-474:
-

I am adding this capability to the new MLContext API.

> Read matrix from URL
> 
>
> Key: SYSTEMML-474
> URL: https://issues.apache.org/jira/browse/SYSTEMML-474
> Project: SystemML
>  Issue Type: Wish
>  Components: Runtime
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>
> The ability to read a matrix from a URL can increase usability of SystemML by 
> eliminating the need to have already existing local or cluster data when 
> invoking SystemML.
> For instance, rather than doing something like:
> {code}
> wget http://example.com/ml/matrix.csv
> ./runStandaloneSystemML.sh example.dml -nvargs X=matrix.csv
> {code}
> Someone could do:
> {code}
> ./runStandaloneSystemML.sh example.dml -nvargs 
> X=http://example.com/ml/matrix.csv
> {code}
> One way that this can be useful is to have a small set of example data for 
> each algorithm sitting on a server. Documentation could show a single command 
> that can be invoked for each algorithm that references the server-based 
> example data. This would mean that a user can essentially copy/paste a single 
> command from the documentation, and it would run using the server example 
> data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-837) Add additional Explain options to new MLContext API

2016-08-15 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-837:
---

Assignee: Deron Eriksson

> Add additional Explain options to new MLContext API
> ---
>
> Key: SYSTEMML-837
> URL: https://issues.apache.org/jira/browse/SYSTEMML-837
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-860) SparkR/HydraR integration with SystemML

2016-08-15 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-860:

Assignee: Alok Singh

> SparkR/HydraR integration with SystemML
> ---
>
> Key: SYSTEMML-860
> URL: https://issues.apache.org/jira/browse/SYSTEMML-860
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs, Runtime
>Affects Versions: SystemML 0.11
>Reporter: Alok Singh
>Assignee: Alok Singh
>
> With our effort to create the BigR like R pacakge on the top of the SystemML 
> and Spark. We have extended the SparkR package and call it HydraR.
> However, for that package to work, we have to enhance the MLContext, and 
> RDDConverterUtilsExt, so that DataFrame and other args can be passed to 
> SystemML via the SparkR connector.
> This effort is for SystemML related java code.
> Alok



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-860) SparkR/HydraR integration with SystemML

2016-08-15 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-860.
---

> SparkR/HydraR integration with SystemML
> ---
>
> Key: SYSTEMML-860
> URL: https://issues.apache.org/jira/browse/SYSTEMML-860
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs, Runtime
>Affects Versions: SystemML 0.11
>Reporter: Alok Singh
>Assignee: Alok Singh
> Fix For: SystemML 0.11
>
>
> With our effort to create the BigR like R pacakge on the top of the SystemML 
> and Spark. We have extended the SparkR package and call it HydraR.
> However, for that package to work, we have to enhance the MLContext, and 
> RDDConverterUtilsExt, so that DataFrame and other args can be passed to 
> SystemML via the SparkR connector.
> This effort is for SystemML related java code.
> Alok



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-860) SparkR/HydraR integration with SystemML

2016-08-15 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-860.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

Fixed by [PR212|https://github.com/apache/incubator-systemml/pull/212].

> SparkR/HydraR integration with SystemML
> ---
>
> Key: SYSTEMML-860
> URL: https://issues.apache.org/jira/browse/SYSTEMML-860
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs, Runtime
>Affects Versions: SystemML 0.11
>Reporter: Alok Singh
>Assignee: Alok Singh
> Fix For: SystemML 0.11
>
>
> With our effort to create the BigR like R pacakge on the top of the SystemML 
> and Spark. We have extended the SparkR package and call it HydraR.
> However, for that package to work, we have to enhance the MLContext, and 
> RDDConverterUtilsExt, so that DataFrame and other args can be passed to 
> SystemML via the SparkR connector.
> This effort is for SystemML related java code.
> Alok



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-744) Update LogisticRegression wrapper class example for newer Spark version

2016-08-15 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422086#comment-15422086
 ] 

Deron Eriksson commented on SYSTEMML-744:
-

LogisticRegression.java was recently removed by 
[PR205|https://github.com/apache/incubator-systemml/pull/205].
I just created [PR213|https://github.com/apache/incubator-systemml/pull/213] to 
address the getTimestamp issue with MLBlock.java w/ Spark 1.6.
I believe we will shortly be creating a 0.11.0 release, so hopefully these 
compilation issues should be resolved in the next release.

> Update LogisticRegression wrapper class example for newer Spark version
> ---
>
> Key: SYSTEMML-744
> URL: https://issues.apache.org/jira/browse/SYSTEMML-744
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs
>Reporter: Glenn Weidner
>Priority: Minor
>
> When building SystemML against newer Spark version (e.g., 1.6.1), two files 
> failed to compile:
> src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java
> src/main/java/org/apache/sysml/api/MLBlock.java
> Default version uses Spark 1.4.1 to maintain backwards compatibility and 
> prevent utilizing interfaces only available in newer Spark versions.  It 
> would be helpful to update (or provide additional) wrapper in cases where 
> building against later Spark version is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-744) Update LogisticRegression wrapper class example for newer Spark version

2016-08-15 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-744:
---

Assignee: Deron Eriksson

> Update LogisticRegression wrapper class example for newer Spark version
> ---
>
> Key: SYSTEMML-744
> URL: https://issues.apache.org/jira/browse/SYSTEMML-744
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs
>Reporter: Glenn Weidner
>Assignee: Deron Eriksson
>Priority: Minor
>
> When building SystemML against newer Spark version (e.g., 1.6.1), two files 
> failed to compile:
> src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java
> src/main/java/org/apache/sysml/api/MLBlock.java
> Default version uses Spark 1.4.1 to maintain backwards compatibility and 
> prevent utilizing interfaces only available in newer Spark versions.  It 
> would be helpful to update (or provide additional) wrapper in cases where 
> building against later Spark version is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-744) Update LogisticRegression wrapper class example for newer Spark version

2016-08-16 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423093#comment-15423093
 ] 

Deron Eriksson commented on SYSTEMML-744:
-

Hello [~vanstee]. On the master branch, LogisticRegression.java is now gone and 
MLBlock.java now has getTimestamp(). So if you are building SystemML, you might 
want to use the master branch. If you need to build using 0.10.0, you should be 
able to delete LogisticRegression.java/LogisticRegressionModel.java and add 
getTimestamp to MLBlock.java. This should be resolved when 0.11.0 is released.

> Update LogisticRegression wrapper class example for newer Spark version
> ---
>
> Key: SYSTEMML-744
> URL: https://issues.apache.org/jira/browse/SYSTEMML-744
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs
>Reporter: Glenn Weidner
>Assignee: Deron Eriksson
>Priority: Minor
>
> When building SystemML against newer Spark version (e.g., 1.6.1), two files 
> failed to compile:
> src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java
> src/main/java/org/apache/sysml/api/MLBlock.java
> Default version uses Spark 1.4.1 to maintain backwards compatibility and 
> prevent utilizing interfaces only available in newer Spark versions.  It 
> would be helpful to update (or provide additional) wrapper in cases where 
> building against later Spark version is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-744) Update LogisticRegression wrapper class example for newer Spark version

2016-08-16 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-744.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

LogisticRegression.java is now deleted and MLBlock.java has getTimestamp, so 
this issue should be resolved now.

> Update LogisticRegression wrapper class example for newer Spark version
> ---
>
> Key: SYSTEMML-744
> URL: https://issues.apache.org/jira/browse/SYSTEMML-744
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs
>Reporter: Glenn Weidner
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>
> When building SystemML against newer Spark version (e.g., 1.6.1), two files 
> failed to compile:
> src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java
> src/main/java/org/apache/sysml/api/MLBlock.java
> Default version uses Spark 1.4.1 to maintain backwards compatibility and 
> prevent utilizing interfaces only available in newer Spark versions.  It 
> would be helpful to update (or provide additional) wrapper in cases where 
> building against later Spark version is required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-565) Documentation frame support

2016-08-16 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-565:

Assignee: Arvind Surve  (was: Deron Eriksson)

> Documentation frame support
> ---
>
> Key: SYSTEMML-565
> URL: https://issues.apache.org/jira/browse/SYSTEMML-565
> Project: SystemML
>  Issue Type: Documentation
>Reporter: Matthias Boehm
>Assignee: Arvind Surve
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-565) Documentation frame support

2016-08-16 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423314#comment-15423314
 ] 

Deron Eriksson commented on SYSTEMML-565:
-

For now, let's add this information to the DML Language Reference in its own 
section. The DML Language Reference needs to be restructured/refactored, so 
adding Frames in its own section would be helpful for this documentation 
refactoring.

> Documentation frame support
> ---
>
> Key: SYSTEMML-565
> URL: https://issues.apache.org/jira/browse/SYSTEMML-565
> Project: SystemML
>  Issue Type: Documentation
>Reporter: Matthias Boehm
>Assignee: Arvind Surve
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-861) Restructure DML Language Reference

2016-08-16 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-861:
---

 Summary: Restructure DML Language Reference
 Key: SYSTEMML-861
 URL: https://issues.apache.org/jira/browse/SYSTEMML-861
 Project: SystemML
  Issue Type: Task
  Components: Documentation
Reporter: Deron Eriksson
Assignee: Deron Eriksson


The DML Language Reference contains lots of great information. However, it 
needs to be refactored to make it more web-friendly. Additionally, the PYDML 
Language Reference could potentially either be folded into the DML Language 
Reference, or at least based on the same format.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-565) Documentation frame support

2016-08-16 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423331#comment-15423331
 ] 

Deron Eriksson commented on SYSTEMML-565:
-

I created SYSTEMML-861 to deal with refactoring the DML Language Reference.

> Documentation frame support
> ---
>
> Key: SYSTEMML-565
> URL: https://issues.apache.org/jira/browse/SYSTEMML-565
> Project: SystemML
>  Issue Type: Documentation
>Reporter: Matthias Boehm
>Assignee: Arvind Surve
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (SYSTEMML-862) Fail gracefully if Spark version not supported

2016-08-16 Thread Deron Eriksson (JIRA)
Deron Eriksson created SYSTEMML-862:
---

 Summary: Fail gracefully if Spark version not supported
 Key: SYSTEMML-862
 URL: https://issues.apache.org/jira/browse/SYSTEMML-862
 Project: SystemML
  Issue Type: Task
  Components: Runtime
Reporter: Deron Eriksson
Assignee: Glenn Weidner


If SystemML is run with a Spark version that is not supported, it should fail 
gracefully.

See isSparkVersionSupported() and compareVersion() in MLContextUtil.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-474) Read matrix from URL

2016-08-16 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-474.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

This capability has been added to the new MLContext API by 
[PR210|https://github.com/apache/incubator-systemml/pull/210].

> Read matrix from URL
> 
>
> Key: SYSTEMML-474
> URL: https://issues.apache.org/jira/browse/SYSTEMML-474
> Project: SystemML
>  Issue Type: Wish
>  Components: Runtime
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> The ability to read a matrix from a URL can increase usability of SystemML by 
> eliminating the need to have already existing local or cluster data when 
> invoking SystemML.
> For instance, rather than doing something like:
> {code}
> wget http://example.com/ml/matrix.csv
> ./runStandaloneSystemML.sh example.dml -nvargs X=matrix.csv
> {code}
> Someone could do:
> {code}
> ./runStandaloneSystemML.sh example.dml -nvargs 
> X=http://example.com/ml/matrix.csv
> {code}
> One way that this can be useful is to have a small set of example data for 
> each algorithm sitting on a server. Documentation could show a single command 
> that can be invoked for each algorithm that references the server-based 
> example data. This would mean that a user can essentially copy/paste a single 
> command from the documentation, and it would run using the server example 
> data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-474) Read matrix from URL

2016-08-16 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-474.
---

> Read matrix from URL
> 
>
> Key: SYSTEMML-474
> URL: https://issues.apache.org/jira/browse/SYSTEMML-474
> Project: SystemML
>  Issue Type: Wish
>  Components: Runtime
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> The ability to read a matrix from a URL can increase usability of SystemML by 
> eliminating the need to have already existing local or cluster data when 
> invoking SystemML.
> For instance, rather than doing something like:
> {code}
> wget http://example.com/ml/matrix.csv
> ./runStandaloneSystemML.sh example.dml -nvargs X=matrix.csv
> {code}
> Someone could do:
> {code}
> ./runStandaloneSystemML.sh example.dml -nvargs 
> X=http://example.com/ml/matrix.csv
> {code}
> One way that this can be useful is to have a small set of example data for 
> each algorithm sitting on a server. Documentation could show a single command 
> that can be invoked for each algorithm that references the server-based 
> example data. This would mean that a user can essentially copy/paste a single 
> command from the documentation, and it would run using the server example 
> data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-46) Build/test automation

2016-08-17 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425635#comment-15425635
 ] 

Deron Eriksson commented on SYSTEMML-46:


Since automatic building and testing (on-demand, pull requests, and 
daily/nightly) has been running for many months at 
https://sparktc.ibmcloud.com/jenkins/, I will resolve and close this issue. If 
performance testing needs to be automated, we can create a separate JIRA issue 
for this.

> Build/test automation
> -
>
> Key: SYSTEMML-46
> URL: https://issues.apache.org/jira/browse/SYSTEMML-46
> Project: SystemML
>  Issue Type: Epic
>Reporter: Frederick Reiss
>Assignee: Alan Chin
>
> HW is located,build server is available,need to get jenkins job setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-46) Build/test automation

2016-08-17 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-46.
--

> Build/test automation
> -
>
> Key: SYSTEMML-46
> URL: https://issues.apache.org/jira/browse/SYSTEMML-46
> Project: SystemML
>  Issue Type: Epic
>Reporter: Frederick Reiss
>Assignee: Alan Chin
> Fix For: SystemML 0.10
>
>
> HW is located,build server is available,need to get jenkins job setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-46) Build/test automation

2016-08-17 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-46.

   Resolution: Fixed
Fix Version/s: SystemML 0.10

> Build/test automation
> -
>
> Key: SYSTEMML-46
> URL: https://issues.apache.org/jira/browse/SYSTEMML-46
> Project: SystemML
>  Issue Type: Epic
>Reporter: Frederick Reiss
>Assignee: Alan Chin
> Fix For: SystemML 0.10
>
>
> HW is located,build server is available,need to get jenkins job setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-46) Build/test automation

2016-08-17 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425639#comment-15425639
 ] 

Deron Eriksson commented on SYSTEMML-46:


I see performance suite JIRAs exist already (SYSTEMML-50 and SYSTEMML-51)

> Build/test automation
> -
>
> Key: SYSTEMML-46
> URL: https://issues.apache.org/jira/browse/SYSTEMML-46
> Project: SystemML
>  Issue Type: Epic
>Reporter: Frederick Reiss
>Assignee: Alan Chin
> Fix For: SystemML 0.10
>
>
> HW is located,build server is available,need to get jenkins job setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-55) Document SystemML developer tools (debugger,explain,etc.)

2016-08-17 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425642#comment-15425642
 ] 

Deron Eriksson commented on SYSTEMML-55:


[~freiss] Is it OK if we go ahead and close this issue? We have a debugger 
guide and an engine developer guide. In addition options such as 'explain' and 
'statistics' are quite accessible in the new MLContext API.

> Document SystemML developer tools (debugger,explain,etc.)
> -
>
> Key: SYSTEMML-55
> URL: https://issues.apache.org/jira/browse/SYSTEMML-55
> Project: SystemML
>  Issue Type: Task
>Reporter: Frederick Reiss
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-57) Expand and organize documentation for Python-like language syntax

2016-08-17 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-57?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-57:
--

Assignee: Deron Eriksson  (was: Frederick Reiss)

> Expand and organize documentation for Python-like language syntax
> -
>
> Key: SYSTEMML-57
> URL: https://issues.apache.org/jira/browse/SYSTEMML-57
> Project: SystemML
>  Issue Type: Task
>Reporter: Frederick Reiss
>Assignee: Deron Eriksson
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>
> Turn the existing documentation for PyDML into a proper language reference 
> manual, with a detailed table of contents and a full description of every 
> language feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-57) Expand and organize documentation for Python-like language syntax

2016-08-17 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-57?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425647#comment-15425647
 ] 

Deron Eriksson commented on SYSTEMML-57:


I will assign this issue to me. Very shortly I will be reworking the DML 
Language Reference [SYSTEMML-861], and this would be a good time to tackle the 
PyDML so that everything is symmetrical and consistent across the languages.

> Expand and organize documentation for Python-like language syntax
> -
>
> Key: SYSTEMML-57
> URL: https://issues.apache.org/jira/browse/SYSTEMML-57
> Project: SystemML
>  Issue Type: Task
>Reporter: Frederick Reiss
>Assignee: Deron Eriksson
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>
> Turn the existing documentation for PyDML into a proper language reference 
> manual, with a detailed table of contents and a full description of every 
> language feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-72) Notebook on DML level

2016-08-17 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425656#comment-15425656
 ] 

Deron Eriksson commented on SYSTEMML-72:


Given that we now have significant examples of SystemML in Jupyter and 
Zeppelin, perhaps we can resolve and close this PR?

[~mwdus...@us.ibm.com] Do you agree?

> Notebook on DML level
> -
>
> Key: SYSTEMML-72
> URL: https://issues.apache.org/jira/browse/SYSTEMML-72
> Project: SystemML
>  Issue Type: Story
>Reporter: Steve Beier
>Assignee: Frederick Reiss
>   Original Estimate: 480h
>  Remaining Estimate: 480h
>
> Create a Notebook (Yupiter or Zeppelin) to edit compile and run SystemML DML 
> language snippets in a REPL fashion to allow data scientists to easily 
> evaluate SystemML.
> Objectives:
> (1) - usability: editing DML (tab completion, syntax highlighting, meaningful 
> error messages, ...)
> (2) - performance analysis: running DML in a Spark notebook (compiler)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-72) Notebook on DML level

2016-08-17 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425658#comment-15425658
 ] 

Deron Eriksson commented on SYSTEMML-72:


[~mwdus...@us.ibm.com] Also, could you look at SYSTEMML-125, SYSTEMML-126, 
SYSTEMML-127, and SYSTEMML-129 and determine if these can be resolved and 
closed?

> Notebook on DML level
> -
>
> Key: SYSTEMML-72
> URL: https://issues.apache.org/jira/browse/SYSTEMML-72
> Project: SystemML
>  Issue Type: Story
>Reporter: Steve Beier
>Assignee: Frederick Reiss
>   Original Estimate: 480h
>  Remaining Estimate: 480h
>
> Create a Notebook (Yupiter or Zeppelin) to edit compile and run SystemML DML 
> language snippets in a REPL fashion to allow data scientists to easily 
> evaluate SystemML.
> Objectives:
> (1) - usability: editing DML (tab completion, syntax highlighting, meaningful 
> error messages, ...)
> (2) - performance analysis: running DML in a Spark notebook (compiler)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-145) Crc file created for each output file when run in standalone mode

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-145:
---

Assignee: Deron Eriksson

> Crc file created for each output file when run in standalone mode
> -
>
> Key: SYSTEMML-145
> URL: https://issues.apache.org/jira/browse/SYSTEMML-145
> Project: SystemML
>  Issue Type: Improvement
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
>
> When a DML script is executed in standalone mode (e.g., 
> ./runStandaloneSystemML.sh algorithms/datagen/genLinearRegressionData.dml 
> -nvargs numSamples=1000 numFeatures=50 maxFeatureValue=5 maxWeight=5 
> addNoise=FALSE b=0 sparsity=1000 output=linRegData.csv format=csv), a crc 
> file is created for each file.
> For example, with the above script, the following files were written to the 
> file system:
> linRegData.csv
> .linRegData.csv.crc
> linRegData.csv.mtd
> .linRegData.csv.mtd.crc
> This litters the file system with a lot of basically unnecessary files. If 
> several scripts are executed, the number of crc grows rapidly.
> There should be a mechanism for one of the following:
> 1) don't generate crc files in the first place
> 2) delete generated crc files
> 3) have an option to generate crc files, with option by default set to false



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-72) Notebook on DML level

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426960#comment-15426960
 ] 

Deron Eriksson commented on SYSTEMML-72:


It looks like SYSTEMML-126 also relates to a DML kernel.

> Notebook on DML level
> -
>
> Key: SYSTEMML-72
> URL: https://issues.apache.org/jira/browse/SYSTEMML-72
> Project: SystemML
>  Issue Type: Story
>Reporter: Steve Beier
>Assignee: Frederick Reiss
>   Original Estimate: 480h
>  Remaining Estimate: 480h
>
> Create a Notebook (Yupiter or Zeppelin) to edit compile and run SystemML DML 
> language snippets in a REPL fashion to allow data scientists to easily 
> evaluate SystemML.
> Objectives:
> (1) - usability: editing DML (tab completion, syntax highlighting, meaningful 
> error messages, ...)
> (2) - performance analysis: running DML in a Spark notebook (compiler)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-125) Notebook DML - Gather Requirements

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-125:

Issue Type: Sub-task  (was: Task)
Parent: SYSTEMML-72

> Notebook DML - Gather Requirements
> --
>
> Key: SYSTEMML-125
> URL: https://issues.apache.org/jira/browse/SYSTEMML-125
> Project: SystemML
>  Issue Type: Sub-task
>Reporter: Christian Kadner
>Assignee: Frederick Reiss
>   Original Estimate: 24h
>  Remaining Estimate: 8h
>
> Questions:
> - what should the Notebook achieve
> - which technology is best suited (usability, easy/quick to implement, ...)
> - what features should be supported (tab completion, syntax highlighting, 
> algorithm reference pop-ups, ...)
> - how long would which features take to implement
> - ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-126) Notebook DML - Implement Kernel

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-126:

Issue Type: Sub-task  (was: Task)
Parent: SYSTEMML-72

> Notebook DML - Implement Kernel
> ---
>
> Key: SYSTEMML-126
> URL: https://issues.apache.org/jira/browse/SYSTEMML-126
> Project: SystemML
>  Issue Type: Sub-task
>Reporter: Christian Kadner
>Assignee: Frederick Reiss
>
> Implement a new language kernel for SystemML DML as the "backend" for the 
> notebook



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-72) Notebook on DML level

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427011#comment-15427011
 ] 

Deron Eriksson commented on SYSTEMML-72:


125, 126, 127, and 129 must be subtasks of 72 that did not get converted to 
subtasks when project JIRA was imported into Apache. I will re-add them as 
subtasks.

> Notebook on DML level
> -
>
> Key: SYSTEMML-72
> URL: https://issues.apache.org/jira/browse/SYSTEMML-72
> Project: SystemML
>  Issue Type: Story
>Reporter: Steve Beier
>Assignee: Frederick Reiss
>   Original Estimate: 480h
>  Remaining Estimate: 480h
>
> Create a Notebook (Yupiter or Zeppelin) to edit compile and run SystemML DML 
> language snippets in a REPL fashion to allow data scientists to easily 
> evaluate SystemML.
> Objectives:
> (1) - usability: editing DML (tab completion, syntax highlighting, meaningful 
> error messages, ...)
> (2) - performance analysis: running DML in a Spark notebook (compiler)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-127) Notebook DML - Create Examples

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-127:

Issue Type: Sub-task  (was: Task)
Parent: SYSTEMML-72

> Notebook DML - Create Examples
> --
>
> Key: SYSTEMML-127
> URL: https://issues.apache.org/jira/browse/SYSTEMML-127
> Project: SystemML
>  Issue Type: Sub-task
>Reporter: Christian Kadner
>Assignee: Frederick Reiss
>
> Create compact examples for sample notebooks along with documentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-129) Notebook DML - Evaluate Design, Reiterate, ...

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-129:

Issue Type: Sub-task  (was: Task)
Parent: SYSTEMML-72

> Notebook DML - Evaluate Design, Reiterate, ...
> --
>
> Key: SYSTEMML-129
> URL: https://issues.apache.org/jira/browse/SYSTEMML-129
> Project: SystemML
>  Issue Type: Sub-task
>Reporter: Christian Kadner
>Assignee: Frederick Reiss
>
> Evaluate Notebook prototype with research, data scientists, developers, exec? 
> and improve it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-210) SparkR integration (data conversion, wrappers)

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427038#comment-15427038
 ] 

Deron Eriksson commented on SYSTEMML-210:
-

[~freiss] [~reinwald]
Should this be assigned to [~aloknsingh]?

> SparkR integration (data conversion, wrappers)
> --
>
> Key: SYSTEMML-210
> URL: https://issues.apache.org/jira/browse/SYSTEMML-210
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Frederick Reiss
>   Original Estimate: 160h
>  Remaining Estimate: 160h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-220) New Second-Order Builtin Function 'apply'

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427040#comment-15427040
 ] 

Deron Eriksson commented on SYSTEMML-220:
-

[~mwdus...@us.ibm.com] Did the subtasks get created for this JIRA?

> New Second-Order Builtin Function 'apply'
> -
>
> Key: SYSTEMML-220
> URL: https://issues.apache.org/jira/browse/SYSTEMML-220
> Project: SystemML
>  Issue Type: Task
>  Components: Compiler, Parser, Runtime
>Reporter: Matthias Boehm
>   Original Estimate: 160h
>  Remaining Estimate: 160h
>
> In several scripts, there is a need to apply rather complex functions to each 
> cell of a matrix. The most natural way of expressing that, especially if this 
> function involves loops and branches, are DML-bodied functions over scalars 
> and surrounding loops over all cells in the matrix. Below is an artificial 
> example:
> {code}
> foo = function( Double in ) return ( Double out ){
> x = in^2;
> if( x > in*2 )
> x = x - in/3;
> out = x + 7;
> }
> R = matrix(0, rows=nrow(A), cols=ncol(A));
> for( i in 1:nrow(A) )
> for( j in 1:ncol(A) )
> R[i,j] = foo(A[i,j]) 
> {code}
> Especially, on large data, this would however cause severe performance 
> problems. Accordingly, people usually "vectorize" these operations by hand 
> which is unfortunately not too easy for very complex functions.
> {code}
> R = A^2 - ppred(A^2, A*2, ">")*A/3 + 7;
> {code}
> For this reason, we would like to integrate a second-order builtin function 
> {{apply}} that would allow users to use their custom functions with 
> reasonable performance. 
> {code}
> R = apply(A, foo);
> {code}
> We would initially constraint this builtin function to DML-bodied functions 
> with (1) single scalar in / single scalar out, (2) no support for nested 
> function invocations, (3) no creation of arbitrarily large intermediates (we 
> assume small memory footprint per cell). These constraints would allow us to 
> provide a very efficient new unary apply operation (multi-threaded in CP, 
> narrow transformation in distributed backends). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-233) Scala ML context hardening

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-233:
---

Assignee: Deron Eriksson  (was: Frederick Reiss)

> Scala ML context hardening
> --
>
> Key: SYSTEMML-233
> URL: https://issues.apache.org/jira/browse/SYSTEMML-233
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-233) Scala ML context hardening

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427044#comment-15427044
 ] 

Deron Eriksson commented on SYSTEMML-233:
-

The MLContext API was completely redesigned by SYSTEMML-593. Although written 
in Java, the new API was designed to have strong support for Spark Shell 
(Scala). Therefore I believe this JIRA can be resolved and closed.

> Scala ML context hardening
> --
>
> Key: SYSTEMML-233
> URL: https://issues.apache.org/jira/browse/SYSTEMML-233
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-233) Scala ML context hardening

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427047#comment-15427047
 ] 

Deron Eriksson commented on SYSTEMML-233:
-

BTW, additional functionality will need to be added to the new API. These 
issues can be identified and added as new JIRAs.

> Scala ML context hardening
> --
>
> Key: SYSTEMML-233
> URL: https://issues.apache.org/jira/browse/SYSTEMML-233
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-233) Scala ML context hardening

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-233.
---

> Scala ML context hardening
> --
>
> Key: SYSTEMML-233
> URL: https://issues.apache.org/jira/browse/SYSTEMML-233
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-233) Scala ML context hardening

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-233.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

> Scala ML context hardening
> --
>
> Key: SYSTEMML-233
> URL: https://issues.apache.org/jira/browse/SYSTEMML-233
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-237) Notebook (dml execute, stats/explain)

2016-08-18 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-237:

Assignee: Mike Dusenberry  (was: Frederick Reiss)

> Notebook (dml execute, stats/explain)
> -
>
> Key: SYSTEMML-237
> URL: https://issues.apache.org/jira/browse/SYSTEMML-237
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Mike Dusenberry
>   Original Estimate: 80h
>  Remaining Estimate: 80h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-237) Notebook (dml execute, stats/explain)

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427053#comment-15427053
 ] 

Deron Eriksson commented on SYSTEMML-237:
-

The new Java/Scala MLContext API features methods for displaying 'statistics' 
and 'explain' information.

The Python API is being redesigned to use the new Java/Scala MLContext API, so 
it should include the ability to display 'statistics' and 'explain' information 
by calling the new MLContext API.

> Notebook (dml execute, stats/explain)
> -
>
> Key: SYSTEMML-237
> URL: https://issues.apache.org/jira/browse/SYSTEMML-237
> Project: SystemML
>  Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Mike Dusenberry
>   Original Estimate: 80h
>  Remaining Estimate: 80h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-295) Unexpected order when print !boolean

2016-08-18 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427074#comment-15427074
 ] 

Deron Eriksson commented on SYSTEMML-295:
-

The prints in SystemML are not necessarily synchronous with the ordering of the 
prints in the DML/PyDML script. It would be nice to be able to have print in 
the order they appear in the script.

> Unexpected order when print !boolean
> 
>
> Key: SYSTEMML-295
> URL: https://issues.apache.org/jira/browse/SYSTEMML-295
> Project: SystemML
>  Issue Type: Improvement
>  Components: APIs, Runtime
>Reporter: Deron Eriksson
>Priority: Minor
>
> If I print the 'not' of a boolean value using print(''+!booleanValue), the 
> resulting output to the console happens later than would be expected. For 
> example, consider the following DML:
> {code}
> b = TRUE
> print(b)
> print(''+b)
> print('hello')
> print(b)
> print(''+!b)
> print('goodbye')
> {code}
> This outputs the following:
> {code}
> TRUE
> TRUE
> hello
> TRUE
> goodbye
> FALSE
> {code}
> I would expect the following:
> {code}
> TRUE
> TRUE
> hello
> TRUE
> FALSE
> goodbye
> {code}
> This also happens for PyDML.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (SYSTEMML-287) LDA: (Umbrella Jira) Develop the new algorithm LDA (Latent Dirichlet Allocation)

2016-08-19 Thread Deron Eriksson (JIRA)

[ 
https://issues.apache.org/jira/browse/SYSTEMML-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428699#comment-15428699
 ] 

Deron Eriksson commented on SYSTEMML-287:
-

Hi [~aloknsingh]. Is this work at a stage where a pull request could be done to 
add it to scripts/staging?

> LDA: (Umbrella Jira) Develop the new algorithm LDA (Latent Dirichlet 
> Allocation)
> 
>
> Key: SYSTEMML-287
> URL: https://issues.apache.org/jira/browse/SYSTEMML-287
> Project: SystemML
>  Issue Type: New Feature
>Reporter: Alok Singh
>Assignee: Alok Singh
>Priority: Critical
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-837) Add additional Explain options to new MLContext API

2016-08-19 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-837.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

Fixed by [PR214|https://github.com/apache/incubator-systemml/pull/214/files].

> Add additional Explain options to new MLContext API
> ---
>
> Key: SYSTEMML-837
> URL: https://issues.apache.org/jira/browse/SYSTEMML-837
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-837) Add additional Explain options to new MLContext API

2016-08-19 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-837.
---

> Add additional Explain options to new MLContext API
> ---
>
> Key: SYSTEMML-837
> URL: https://issues.apache.org/jira/browse/SYSTEMML-837
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
> Fix For: SystemML 0.11
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (SYSTEMML-467) Simplify script invocation section of Beginner's Guide to DML

2016-08-22 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson reassigned SYSTEMML-467:
---

Assignee: Deron Eriksson

> Simplify script invocation section of Beginner's Guide to DML
> -
>
> Key: SYSTEMML-467
> URL: https://issues.apache.org/jira/browse/SYSTEMML-467
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>
> New user needs to be able run a DML script in as short a time as possible. 
> Shortest way is probably to download a standalone release (when it is 
> available) and run a "hello world" script. Current version of the doc 
> mentions building the project. The build command is not shown on this page 
> and building the project can be a potential turnoff. Better option is to 
> download a standalone release and use that.
> Additionally, move Eclipse Debug Configuration out of the Beginner's Guide 
> since it would make more sense in a separate document that focuses more on 
> contributing code to the SystemML project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-520) Move MLContext API info from lang ref to MLContext prog guide

2016-08-22 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-520:

Assignee: Mike Dusenberry

> Move MLContext API info from lang ref to MLContext prog guide
> -
>
> Key: SYSTEMML-520
> URL: https://issues.apache.org/jira/browse/SYSTEMML-520
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.11
>
>
> The last section of the DML Language Reference currently pertains to the 
> MLContext API. Since there is now an "Spark MLContext Programming Guide", the 
> information should either be moved to the new location or if it is redundant 
> it should be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-520) Move MLContext API info from lang ref to MLContext prog guide

2016-08-22 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-520.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.11

This was fixed by [PR203|https://github.com/apache/incubator-systemml/pull/203].

> Move MLContext API info from lang ref to MLContext prog guide
> -
>
> Key: SYSTEMML-520
> URL: https://issues.apache.org/jira/browse/SYSTEMML-520
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.11
>
>
> The last section of the DML Language Reference currently pertains to the 
> MLContext API. Since there is now an "Spark MLContext Programming Guide", the 
> information should either be moved to the new location or if it is redundant 
> it should be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-520) Move MLContext API info from lang ref to MLContext prog guide

2016-08-22 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-520.
---

> Move MLContext API info from lang ref to MLContext prog guide
> -
>
> Key: SYSTEMML-520
> URL: https://issues.apache.org/jira/browse/SYSTEMML-520
> Project: SystemML
>  Issue Type: Task
>  Components: Documentation
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.11
>
>
> The last section of the DML Language Reference currently pertains to the 
> MLContext API. Since there is now an "Spark MLContext Programming Guide", the 
> information should either be moved to the new location or if it is redundant 
> it should be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (SYSTEMML-578) Refactor org.apache.sysml.api.ml package

2016-08-22 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson updated SYSTEMML-578:

Assignee: Tommy Yu

> Refactor org.apache.sysml.api.ml package
> 
>
> Key: SYSTEMML-578
> URL: https://issues.apache.org/jira/browse/SYSTEMML-578
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Affects Versions: SystemML 0.9
>Reporter: Tommy Yu
>Assignee: Tommy Yu
>Priority: Minor
> Fix For: SystemML 0.10
>
>
> It might be better to move the Java version to a org.apache.sysml.api.ml.java 
> folder (or possibly remove it), and move this file to org.apache.sysml.api.ml 
> in order to encourage other Scala contributions to the API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (SYSTEMML-578) Refactor org.apache.sysml.api.ml package

2016-08-22 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson resolved SYSTEMML-578.
-
   Resolution: Fixed
Fix Version/s: SystemML 0.10

> Refactor org.apache.sysml.api.ml package
> 
>
> Key: SYSTEMML-578
> URL: https://issues.apache.org/jira/browse/SYSTEMML-578
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Affects Versions: SystemML 0.9
>Reporter: Tommy Yu
>Assignee: Tommy Yu
>Priority: Minor
> Fix For: SystemML 0.10
>
>
> It might be better to move the Java version to a org.apache.sysml.api.ml.java 
> folder (or possibly remove it), and move this file to org.apache.sysml.api.ml 
> in order to encourage other Scala contributions to the API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (SYSTEMML-578) Refactor org.apache.sysml.api.ml package

2016-08-22 Thread Deron Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SYSTEMML-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deron Eriksson closed SYSTEMML-578.
---

> Refactor org.apache.sysml.api.ml package
> 
>
> Key: SYSTEMML-578
> URL: https://issues.apache.org/jira/browse/SYSTEMML-578
> Project: SystemML
>  Issue Type: Task
>  Components: APIs
>Affects Versions: SystemML 0.9
>Reporter: Tommy Yu
>Assignee: Tommy Yu
>Priority: Minor
> Fix For: SystemML 0.10
>
>
> It might be better to move the Java version to a org.apache.sysml.api.ml.java 
> folder (or possibly remove it), and move this file to org.apache.sysml.api.ml 
> in order to encourage other Scala contributions to the API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


<    1   2   3   4   5   6   7   8   9   10   >