[jira] [Updated] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-845:
-
Component/s: Compiler
Algorithms
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
> Components: Algorithms, Compiler
>Affects Versions: SystemML 0.11
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-845:
-
Affects Version/s: SystemML 0.11
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Affects Versions: SystemML 0.11
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-845:
-
Assignee: (was: Mike Dusenberry)
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Affects Versions: SystemML 0.11
>Reporter: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry reassigned SYSTEMML-845:
Assignee: Mike Dusenberry
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Affects Versions: SystemML 0.11
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry reassigned SYSTEMML-845:
Assignee: Mike Dusenberry
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-710) Add `SystemML.py` To All Distribution Releases
[
https://issues.apache.org/jira/browse/SYSTEMML-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410336#comment-15410336
]
Mike Dusenberry commented on SYSTEMML-710:
--
[~deron], [~gweidner] This is definitely relevant for the upcoming releases.
> Add `SystemML.py` To All Distribution Releases
> --
>
> Key: SYSTEMML-710
> URL: https://issues.apache.org/jira/browse/SYSTEMML-710
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>Priority: Minor
> Labels: starter
>
> Currently, our {{SystemML.py}} Python API file is not included in the release
> assemblies. We should add it to the base directory of each of the release
> packages so that our {{MLContext}} PySpark API can be used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-582) Determine If Multiple Builds Are Needed For Different Scala Versions.
[ https://issues.apache.org/jira/browse/SYSTEMML-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410334#comment-15410334 ] Mike Dusenberry commented on SYSTEMML-582: -- [~deron], [~gweidner] This will be relevant for the upcoming releases. > Determine If Multiple Builds Are Needed For Different Scala Versions. > - > > Key: SYSTEMML-582 > URL: https://issues.apache.org/jira/browse/SYSTEMML-582 > Project: SystemML > Issue Type: New Feature >Reporter: Mike Dusenberry > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-490) Runtime Platform Should Automatically Be Set To Hybrid_Spark When Executed On Spark
[
https://issues.apache.org/jira/browse/SYSTEMML-490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-490.
> Runtime Platform Should Automatically Be Set To Hybrid_Spark When Executed On
> Spark
> ---
>
> Key: SYSTEMML-490
> URL: https://issues.apache.org/jira/browse/SYSTEMML-490
> Project: SystemML
> Issue Type: Bug
>Reporter: Mike Dusenberry
>Assignee: Matthias Boehm
> Fix For: SystemML 0.11
>
>
> Currently, the default runtime platform is set to "hybrid" mode, which is an
> automatically optimized hybrid between single-node and Hadoop MR. When
> running on Spark, we should automatically detect and change the mode to the
> correct setting of "hybrid_spark". Of course, our {{sparkDML.sh}} script
> appends this runtime mode explicitly, but a user shouldn't have to do this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408490#comment-15408490
]
Mike Dusenberry edited comment on SYSTEMML-845 at 8/6/16 12:25 AM:
---
Attaching logs ({{log08.03.16-1470268602.txt}}) for running both scripts twice
in standalone mode with the {{-exec singlenode}} flag with 20GB of memory,
using data inputs in the SystemML binary format -- see {{run.sh}} and
{{perf.sh}} for information.
Results:
- Run #1:
|| Script | Time (s) | Accuracy ||
| mnist_lenet-train.dml | 2987.400704441 | 99.32% |
| lenet-train.dml | 2816.369435579 | 99.28% |
- Run #2:
|| Script | Time (s) | Accuracy ||
| mnist_lenet-train.dml | 2847.790531812 | 99.16% |
| lenet-train.dml | 2950.520494210 | 99.18% |
So, same accuracy, and same runtime in singlenode mode!
was (Author: mwdus...@us.ibm.com):
Attaching logs ({{log08.03.16-1470268602.txt}}) for running both scripts twice
in standalone mode with the {{-exec singlenode}} flag with 20GB of memory,
using data inputs in the SystemML binary format -- see {{run.sh}} and
{{perf.sh}} for information.
Results:
- Run #1:
|| Script | Time (s) | Accuracy ||
| mnist_lenet-train.dml | 2987.400704441 | 99.32% |
| lenet-train.dml | 2816.369435579 | 99.28% |
- Run #2:
|| Script | Time (s) | Accuracy ||
| mnist_lenet-train.dml | 2847.790531812 | 99.16% |
| lenet-train.dml | 2950.520494210 | 99.18% |
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410312#comment-15410312
]
Mike Dusenberry edited comment on SYSTEMML-845 at 8/6/16 12:17 AM:
---
[~niketanpansare] Adding logs with {{-explain recompile_hops}}, stopped early
to limit log size.
Files:
* {{mnist_lenet-train-spark-explain-recompile-hops.log}} - The SystemML-NN
version using DML functions.
* {{lenet-train-spark-explain-recompile-hops.log}} - The version without any
DML functions.
was (Author: mwdus...@us.ibm.com):
[~niketanpansare] Adding logs with {{-explain recompile_hops}}, stopped early
to limit log size.
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-845:
-
Attachment: mnist_lenet-train-spark-explain-recompile-hops.log
lenet-train-spark-explain-recompile-hops.log
[~niketanpansare] Adding logs with {{-explain recompile_hops}}, stopped early
to limit log size.
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
> Attachments: convert.dml,
> lenet-train-spark-explain-recompile-hops.log, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt,
> mnist_lenet-train-spark-explain-recompile-hops.log,
> mnist_lenet-train-spark-explain.log, perf.sh, run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-457) Create Standalone mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410235#comment-15410235 ] Deron Eriksson commented on SYSTEMML-457: - Thank you [~mwdus...@us.ibm.com] > Create Standalone mode guide > > > Key: SYSTEMML-457 > URL: https://issues.apache.org/jira/browse/SYSTEMML-457 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > Create a document that focuses on Standalone mode execution. Documentation > has recently been reorganized to emphasize the various execution modes. > Quick Start Guide can be used as a starting point for this. Quick Start Guide > has a bit of a broader focus than only Standalone mode. > Topics to include: > Download release > Standalone on Linux, Mac, Windows > Hello World > Algorithm examples > Execution on single machine, Hadoop, and Spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-458) Create Spark batch mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410234#comment-15410234 ] Deron Eriksson commented on SYSTEMML-458: - Thank you [~mwdus...@us.ibm.com] > Create Spark batch mode guide > - > > Key: SYSTEMML-458 > URL: https://issues.apache.org/jira/browse/SYSTEMML-458 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > Spark batch mode is one of the primary execution modes of SystemML. Document > is needed similar to Hadoop Batch Mode document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-849) Clean Up and Reorganize Documentation Targeted At Data Scientists
[ https://issues.apache.org/jira/browse/SYSTEMML-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410211#comment-15410211 ] Mike Dusenberry commented on SYSTEMML-849: -- Merged in [commit 77363c0 | https://github.com/apache/incubator-systemml/commit/77363c0c67b131cbf937b023a353765af3c6e3bb]. > Clean Up and Reorganize Documentation Targeted At Data Scientists > - > > Key: SYSTEMML-849 > URL: https://issues.apache.org/jira/browse/SYSTEMML-849 > Project: SystemML > Issue Type: Documentation > Components: Documentation >Affects Versions: SystemML 0.11 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > This JIRA issue aims to clean up and reorganize a lot of the existing > documentation. The goal here is to work towards cleaning up our external > message and targeting specific types of users in order to increase ease of > adoption. > My vision is that we target data scientists using Spark first and foremost. > Without this focus in our documentation, the project is seemingly too > confusing, and will deter this key user demographic from adoption. Once these > users are onboard, engine developers will follow. > This PR is a first effort towards this goal, and provides a nicer, cleaned-up > version of the docs. We should collectively work to improve these, with clear > separation between data scientists, systems engineers/researchers, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-849) Clean Up and Reorganize Documentation Targeted At Data Scientists
[ https://issues.apache.org/jira/browse/SYSTEMML-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry closed SYSTEMML-849. > Clean Up and Reorganize Documentation Targeted At Data Scientists > - > > Key: SYSTEMML-849 > URL: https://issues.apache.org/jira/browse/SYSTEMML-849 > Project: SystemML > Issue Type: Documentation > Components: Documentation >Affects Versions: SystemML 0.11 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > This JIRA issue aims to clean up and reorganize a lot of the existing > documentation. The goal here is to work towards cleaning up our external > message and targeting specific types of users in order to increase ease of > adoption. > My vision is that we target data scientists using Spark first and foremost. > Without this focus in our documentation, the project is seemingly too > confusing, and will deter this key user demographic from adoption. Once these > users are onboard, engine developers will follow. > This PR is a first effort towards this goal, and provides a nicer, cleaned-up > version of the docs. We should collectively work to improve these, with clear > separation between data scientists, systems engineers/researchers, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-458) Create Spark batch mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry resolved SYSTEMML-458. -- Resolution: Fixed Fix Version/s: SystemML 0.11 > Create Spark batch mode guide > - > > Key: SYSTEMML-458 > URL: https://issues.apache.org/jira/browse/SYSTEMML-458 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > Spark batch mode is one of the primary execution modes of SystemML. Document > is needed similar to Hadoop Batch Mode document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-458) Create Spark batch mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry closed SYSTEMML-458. > Create Spark batch mode guide > - > > Key: SYSTEMML-458 > URL: https://issues.apache.org/jira/browse/SYSTEMML-458 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > Spark batch mode is one of the primary execution modes of SystemML. Document > is needed similar to Hadoop Batch Mode document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-458) Create Spark batch mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410213#comment-15410213 ] Mike Dusenberry commented on SYSTEMML-458: -- Merged in [commit 77363c0 | https://github.com/apache/incubator-systemml/commit/77363c0c67b131cbf937b023a353765af3c6e3bb]. > Create Spark batch mode guide > - > > Key: SYSTEMML-458 > URL: https://issues.apache.org/jira/browse/SYSTEMML-458 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > > Spark batch mode is one of the primary execution modes of SystemML. Document > is needed similar to Hadoop Batch Mode document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-457) Create Standalone mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry resolved SYSTEMML-457. -- Resolution: Fixed Fix Version/s: SystemML 0.11 > Create Standalone mode guide > > > Key: SYSTEMML-457 > URL: https://issues.apache.org/jira/browse/SYSTEMML-457 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > Create a document that focuses on Standalone mode execution. Documentation > has recently been reorganized to emphasize the various execution modes. > Quick Start Guide can be used as a starting point for this. Quick Start Guide > has a bit of a broader focus than only Standalone mode. > Topics to include: > Download release > Standalone on Linux, Mac, Windows > Hello World > Algorithm examples > Execution on single machine, Hadoop, and Spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-457) Create Standalone mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410212#comment-15410212 ] Mike Dusenberry commented on SYSTEMML-457: -- Merged in [commit 77363c0 | https://github.com/apache/incubator-systemml/commit/77363c0c67b131cbf937b023a353765af3c6e3bb]. > Create Standalone mode guide > > > Key: SYSTEMML-457 > URL: https://issues.apache.org/jira/browse/SYSTEMML-457 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > Create a document that focuses on Standalone mode execution. Documentation > has recently been reorganized to emphasize the various execution modes. > Quick Start Guide can be used as a starting point for this. Quick Start Guide > has a bit of a broader focus than only Standalone mode. > Topics to include: > Download release > Standalone on Linux, Mac, Windows > Hello World > Algorithm examples > Execution on single machine, Hadoop, and Spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-457) Create Standalone mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry closed SYSTEMML-457. > Create Standalone mode guide > > > Key: SYSTEMML-457 > URL: https://issues.apache.org/jira/browse/SYSTEMML-457 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > Create a document that focuses on Standalone mode execution. Documentation > has recently been reorganized to emphasize the various execution modes. > Quick Start Guide can be used as a starting point for this. Quick Start Guide > has a bit of a broader focus than only Standalone mode. > Topics to include: > Download release > Standalone on Linux, Mac, Windows > Hello World > Algorithm examples > Execution on single machine, Hadoop, and Spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-849) Clean Up and Reorganize Documentation Targeted At Data Scientists
[ https://issues.apache.org/jira/browse/SYSTEMML-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry resolved SYSTEMML-849. -- Resolution: Fixed Fix Version/s: SystemML 0.11 > Clean Up and Reorganize Documentation Targeted At Data Scientists > - > > Key: SYSTEMML-849 > URL: https://issues.apache.org/jira/browse/SYSTEMML-849 > Project: SystemML > Issue Type: Documentation > Components: Documentation >Affects Versions: SystemML 0.11 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > This JIRA issue aims to clean up and reorganize a lot of the existing > documentation. The goal here is to work towards cleaning up our external > message and targeting specific types of users in order to increase ease of > adoption. > My vision is that we target data scientists using Spark first and foremost. > Without this focus in our documentation, the project is seemingly too > confusing, and will deter this key user demographic from adoption. Once these > users are onboard, engine developers will follow. > This PR is a first effort towards this goal, and provides a nicer, cleaned-up > version of the docs. We should collectively work to improve these, with clear > separation between data scientists, systems engineers/researchers, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-849) Clean Up and Reorganize Documentation Targeted At Data Scientists
[ https://issues.apache.org/jira/browse/SYSTEMML-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410207#comment-15410207 ] Mike Dusenberry commented on SYSTEMML-849: -- [PR 203 | https://github.com/apache/incubator-systemml/pull/203] submitted for discussion. > Clean Up and Reorganize Documentation Targeted At Data Scientists > - > > Key: SYSTEMML-849 > URL: https://issues.apache.org/jira/browse/SYSTEMML-849 > Project: SystemML > Issue Type: Documentation > Components: Documentation >Affects Versions: SystemML 0.11 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry > Fix For: SystemML 0.11 > > > This JIRA issue aims to clean up and reorganize a lot of the existing > documentation. The goal here is to work towards cleaning up our external > message and targeting specific types of users in order to increase ease of > adoption. > My vision is that we target data scientists using Spark first and foremost. > Without this focus in our documentation, the project is seemingly too > confusing, and will deter this key user demographic from adoption. Once these > users are onboard, engine developers will follow. > This PR is a first effort towards this goal, and provides a nicer, cleaned-up > version of the docs. We should collectively work to improve these, with clear > separation between data scientists, systems engineers/researchers, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-849) Clean Up and Reorganize Documentation Targeted At Data Scientists
[ https://issues.apache.org/jira/browse/SYSTEMML-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-849: - Summary: Clean Up and Reorganize Documentation Targeted At Data Scientists (was: Clean Up and Reorganize Documentation Targeted Against Data Scientists) > Clean Up and Reorganize Documentation Targeted At Data Scientists > - > > Key: SYSTEMML-849 > URL: https://issues.apache.org/jira/browse/SYSTEMML-849 > Project: SystemML > Issue Type: Documentation > Components: Documentation >Affects Versions: SystemML 0.11 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry > > This JIRA issue aims to clean up and reorganize a lot of the existing > documentation. The goal here is to work towards cleaning up our external > message and targeting specific types of users in order to increase ease of > adoption. > My vision is that we target data scientists using Spark first and foremost. > Without this focus in our documentation, the project is seemingly too > confusing, and will deter this key user demographic from adoption. Once these > users are onboard, engine developers will follow. > This PR is a first effort towards this goal, and provides a nicer, cleaned-up > version of the docs. We should collectively work to improve these, with clear > separation between data scientists, systems engineers/researchers, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SYSTEMML-849) Clean Up and Reorganize Documentation Targeted Against Data Scientists
Mike Dusenberry created SYSTEMML-849: Summary: Clean Up and Reorganize Documentation Targeted Against Data Scientists Key: SYSTEMML-849 URL: https://issues.apache.org/jira/browse/SYSTEMML-849 Project: SystemML Issue Type: Documentation Components: Documentation Affects Versions: SystemML 0.11 Reporter: Mike Dusenberry Assignee: Mike Dusenberry This JIRA issue aims to clean up and reorganize a lot of the existing documentation. The goal here is to work towards cleaning up our external message and targeting specific types of users in order to increase ease of adoption. My vision is that we target data scientists using Spark first and foremost. Without this focus in our documentation, the project is seemingly too confusing, and will deter this key user demographic from adoption. Once these users are onboard, engine developers will follow. This PR is a first effort towards this goal, and provides a nicer, cleaned-up version of the docs. We should collectively work to improve these, with clear separation between data scientists, systems engineers/researchers, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410156#comment-15410156
]
Mike Dusenberry commented on SYSTEMML-845:
--
[~niketanpansare] Yes, your assumption is correct -- same DML, just the
{{mnist_lenet-train.dml}} uses DML-bodied functions. Same performance in
forced singlenode, but performance regressions for the DML-bodied version in
hybrid MR or hybrid Spark. I'll go run the limited, 2-epoch versions with
{{-explain recompile_hops}}. For time reasons, I'll just terminate the
processes once they've spit out the explain info.
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
> Attachments: convert.dml, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt, mnist_lenet-train-spark-explain.log, perf.sh,
> run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-836) Create ScriptFactory convenience methods for resources on classpath
[
https://issues.apache.org/jira/browse/SYSTEMML-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson reassigned SYSTEMML-836:
---
Assignee: Deron Eriksson
> Create ScriptFactory convenience methods for resources on classpath
> ---
>
> Key: SYSTEMML-836
> URL: https://issues.apache.org/jira/browse/SYSTEMML-836
> Project: SystemML
> Issue Type: Task
> Components: APIs
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
>Priority: Minor
>
> Currently a DML Script object can be created by ScriptFactory by getting an
> input stream to a resource on the classpath, such as this example:
> {code}
> val inputStream =
> getClass.getResourceAsStream("/scripts/algorithms/Univar-Stats.dml")
> val script = ScriptFactory.dmlFromInputStream(inputStream)
> {code}
> This can be further simplified by creating a ScriptFactory method like
> dmlFromResourcePath (or dmlFromClasspath) so a user could instead do:
> {code}
> val script =
> ScriptFactory.dmlFromResourcePath("/scripts/algorithms/Univar-Stats.dml")
> {code}
> An addition method should be created for PyDML, such as pydmlFromResourcePath
> (or pydmlFromClasspath).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-845) Compare Performance of LeNet Scripts With & Without Using SystemML-NN
[
https://issues.apache.org/jira/browse/SYSTEMML-845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410063#comment-15410063
]
Niketan Pansare commented on SYSTEMML-845:
--
[~mwdus...@us.ibm.com] The performance of
https://issues.apache.org/jira/secure/attachment/12822211/lenet-train-spark-explain.log
is as expected. I am planning to deliver a commit with performance improvement
to maxpool_bwd by next week.
Regarding
https://issues.apache.org/jira/secure/attachment/12822212/mnist_lenet-train-spark-explain.log,
am I correct in assuming that both scripts have identical DML except that
mnist_lenet-train has UDF. Can you please run the scripts again with `-explain
recompile_hops` ?
> Compare Performance of LeNet Scripts With & Without Using SystemML-NN
> -
>
> Key: SYSTEMML-845
> URL: https://issues.apache.org/jira/browse/SYSTEMML-845
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
> Attachments: convert.dml, lenet-train-spark-explain.log,
> log08.03.16-1470268602.txt, mnist_lenet-train-spark-explain.log, perf.sh,
> run.sh
>
>
> This JIRA issue tracks the comparison of the performance of the LeNet scripts
> with & without using SystemML-NN. The goal is that they should have equal
> performance in terms of both accuracy and time. Any difference will be
> indicate areas of engine improvement.
> Scripts:
> * [mnist_lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/SystemML-NN/examples/mnist_lenet-train.dml]
> - LeNet script that *does* use the SystemML-NN library.
> * [lenet-train.dml |
> https://github.com/apache/incubator-systemml/blob/master/scripts/staging/lenet-train.dml]
> - LeNet script that *does not* use the SystemML-NN library.
> To fully reproduce, I basically created a directory, placed the two attached
> bash scripts in it, grabbed a copy of the NN library and placed it into the
> directory, ran the examples/get_mnist_data.sh script from the library to get
> the data (placed into examples/data), then used the attached convert.dml to
> create binary copies of the data for both scripts, then ran run.sh. Also, I
> copied examples/data to the base directory as well. Adjust the {{EXEC}} and
> related variables in {{perf.sh}} to switch between standalone, Spark, memory
> sizes, explain, stats, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-457) Create Standalone mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson updated SYSTEMML-457: Assignee: Mike Dusenberry > Create Standalone mode guide > > > Key: SYSTEMML-457 > URL: https://issues.apache.org/jira/browse/SYSTEMML-457 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > > Create a document that focuses on Standalone mode execution. Documentation > has recently been reorganized to emphasize the various execution modes. > Quick Start Guide can be used as a starting point for this. Quick Start Guide > has a bit of a broader focus than only Standalone mode. > Topics to include: > Download release > Standalone on Linux, Mac, Windows > Hello World > Algorithm examples > Execution on single machine, Hadoop, and Spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-458) Create Spark batch mode guide
[ https://issues.apache.org/jira/browse/SYSTEMML-458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson updated SYSTEMML-458: Assignee: Mike Dusenberry > Create Spark batch mode guide > - > > Key: SYSTEMML-458 > URL: https://issues.apache.org/jira/browse/SYSTEMML-458 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Mike Dusenberry > > Spark batch mode is one of the primary execution modes of SystemML. Document > is needed similar to Hadoop Batch Mode document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-478) Upgrade maven-failsafe-plugin if fixed
[ https://issues.apache.org/jira/browse/SYSTEMML-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-478. - Resolution: Won't Fix Assignee: Deron Eriksson Fix Version/s: SystemML 0.11 Since issue seems to currently exist with all versions later than 2.17, we shouldn't upgrade this plugin for now. > Upgrade maven-failsafe-plugin if fixed > -- > > Key: SYSTEMML-478 > URL: https://issues.apache.org/jira/browse/SYSTEMML-478 > Project: SystemML > Issue Type: Improvement >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > Fix For: SystemML 0.11 > > > The maven-failsafe-plugin entry in pom.xml is set to version 2.17 due to: > "Failsafe 2.18 has a bug in computing # cores, so use 2.17" > The maven central repo has the following later versions of > maven-failsafe-plugin available: > 2.19.1 > 2.19 > 2.18.1 > We should determine if a later version of the plugin can be used and the > "HACK ALERT" message can be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-478) Upgrade maven-failsafe-plugin if fixed
[ https://issues.apache.org/jira/browse/SYSTEMML-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson closed SYSTEMML-478. --- > Upgrade maven-failsafe-plugin if fixed > -- > > Key: SYSTEMML-478 > URL: https://issues.apache.org/jira/browse/SYSTEMML-478 > Project: SystemML > Issue Type: Improvement >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > Fix For: SystemML 0.11 > > > The maven-failsafe-plugin entry in pom.xml is set to version 2.17 due to: > "Failsafe 2.18 has a bug in computing # cores, so use 2.17" > The maven central repo has the following later versions of > maven-failsafe-plugin available: > 2.19.1 > 2.19 > 2.18.1 > We should determine if a later version of the plugin can be used and the > "HACK ALERT" message can be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-708) Release checklist for 0.10.0-incubating-rc1
[
https://issues.apache.org/jira/browse/SYSTEMML-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409894#comment-15409894
]
Deron Eriksson commented on SYSTEMML-708:
-
Note that RC1 was rejected due to unexpected binaries in the src release. This
was fixed in RC2.
> Release checklist for 0.10.0-incubating-rc1
> ---
>
> Key: SYSTEMML-708
> URL: https://issues.apache.org/jira/browse/SYSTEMML-708
> Project: SystemML
> Issue Type: Task
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> || Task || Status || Notes ||
> | All Artifacts and Checksums Present | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Windows | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - OS X | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Windows |{panel:bgColor=#bfffba}Pass{panel} |
> SYSTEMML-712 opened for intermittent test failure |
> | Test Suite Passes - OS X| {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | All Binaries Execute| {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Check LICENSE and NOTICE Files | {panel:bgColor=#bfffba}Pass{panel} |
> non-blocker, SYSTEMML-711 filed |
> | Src Artifact Builds and Tests Pass | {panel:bgColor=#bfffba}Pass{panel} |
> 5037 of 5038 passed on OS X (RightIndexingMatrixTest failed) |
> | Single-Node Standalone - Windows| {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - OS X | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Spark | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X
> | Single-Node Hadoop | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X
> | Notebooks - Jupyter | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Notebooks - Zeppelin| {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Performance Suite - Spark | {panel:bgColor=#bfffba}Pass{panel} |
> Run on Spark 1.6.1 for data sizes {80MB, 800MB, 8GB, 80GB} |
> | Performance Suite - Hadoop | | |
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-708) Release checklist for 0.10.0-incubating-rc1
[
https://issues.apache.org/jira/browse/SYSTEMML-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson resolved SYSTEMML-708.
-
Resolution: Fixed
Fix Version/s: SystemML 0.10
> Release checklist for 0.10.0-incubating-rc1
> ---
>
> Key: SYSTEMML-708
> URL: https://issues.apache.org/jira/browse/SYSTEMML-708
> Project: SystemML
> Issue Type: Task
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> || Task || Status || Notes ||
> | All Artifacts and Checksums Present | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Windows | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - OS X | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Windows |{panel:bgColor=#bfffba}Pass{panel} |
> SYSTEMML-712 opened for intermittent test failure |
> | Test Suite Passes - OS X| {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | All Binaries Execute| {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Check LICENSE and NOTICE Files | {panel:bgColor=#bfffba}Pass{panel} |
> non-blocker, SYSTEMML-711 filed |
> | Src Artifact Builds and Tests Pass | {panel:bgColor=#bfffba}Pass{panel} |
> 5037 of 5038 passed on OS X (RightIndexingMatrixTest failed) |
> | Single-Node Standalone - Windows| {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - OS X | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Spark | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X
> | Single-Node Hadoop | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X
> | Notebooks - Jupyter | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Notebooks - Zeppelin| {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Performance Suite - Spark | {panel:bgColor=#bfffba}Pass{panel} |
> Run on Spark 1.6.1 for data sizes {80MB, 800MB, 8GB, 80GB} |
> | Performance Suite - Hadoop | | |
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-708) Release checklist for 0.10.0-incubating-rc1
[
https://issues.apache.org/jira/browse/SYSTEMML-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson closed SYSTEMML-708.
---
> Release checklist for 0.10.0-incubating-rc1
> ---
>
> Key: SYSTEMML-708
> URL: https://issues.apache.org/jira/browse/SYSTEMML-708
> Project: SystemML
> Issue Type: Task
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.10
>
>
> || Task || Status || Notes ||
> | All Artifacts and Checksums Present | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Windows | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - OS X | {panel:bgColor=#bfffba}Pass{panel} | |
> | Release Candidate Build - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Windows |{panel:bgColor=#bfffba}Pass{panel} |
> SYSTEMML-712 opened for intermittent test failure |
> | Test Suite Passes - OS X| {panel:bgColor=#bfffba}Pass{panel} | |
> | Test Suite Passes - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | All Binaries Execute| {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Check LICENSE and NOTICE Files | {panel:bgColor=#bfffba}Pass{panel} |
> non-blocker, SYSTEMML-711 filed |
> | Src Artifact Builds and Tests Pass | {panel:bgColor=#bfffba}Pass{panel} |
> 5037 of 5038 passed on OS X (RightIndexingMatrixTest failed) |
> | Single-Node Standalone - Windows| {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - OS X | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Standalone - Linux | {panel:bgColor=#bfffba}Pass{panel} | |
> | Single-Node Spark | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X
> | Single-Node Hadoop | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X
> | Notebooks - Jupyter | {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Notebooks - Zeppelin| {panel:bgColor=#bfffba}Pass{panel} |
> Verified on OS X |
> | Performance Suite - Spark | {panel:bgColor=#bfffba}Pass{panel} |
> Run on Spark 1.6.1 for data sizes {80MB, 800MB, 8GB, 80GB} |
> | Performance Suite - Hadoop | | |
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-707) diag not generate square matrix if given Nx1 matrix of zeroes
[
https://issues.apache.org/jira/browse/SYSTEMML-707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson closed SYSTEMML-707.
---
Verified that diag now generates a square matrix if given Nx1 matrix of zeroes
> diag not generate square matrix if given Nx1 matrix of zeroes
> -
>
> Key: SYSTEMML-707
> URL: https://issues.apache.org/jira/browse/SYSTEMML-707
> Project: SystemML
> Issue Type: Bug
> Components: APIs
>Reporter: Deron Eriksson
>Assignee: Matthias Boehm
> Fix For: SystemML 0.11
>
>
> Thank you Matthew Plourde for finding this!
> If an Nx1 matrix of 0's is given to the diag() function, an Nx1 matrix of 0's
> is returned. However, if an Nx1 matrix consists of any values that aren't
> 0's, an NxN diagonal matrix is returned. This is inconsistent and the Nx1
> matrix of 0's to diag() should probably return an NxN matrix.
> Example 1:
> {code}
> zeroes=matrix(0, 5, 1);
> print(toString(zeroes));
> print(toString(diag(zeroes)));
> {code}
> gives:
> {code}
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> 0.000
> {code}
> Example 2:
> {code}
> ones=matrix(1, 5, 1);
> print(toString(ones));
> print(toString(diag(ones)));
> {code}
> gives:
> {code}
> 1.000
> 1.000
> 1.000
> 1.000
> 1.000
> 1.000 0.000 0.000 0.000 0.000
> 0.000 1.000 0.000 0.000 0.000
> 0.000 0.000 1.000 0.000 0.000
> 0.000 0.000 0.000 1.000 0.000
> 0.000 0.000 0.000 0.000 1.000
> {code}
> Example 3:
> {code}
> nums=matrix("0 1 2 3 4", 5, 1);
> print(toString(nums));
> print(toString(diag(nums)));
> {code}
> gives:
> {code}
> 0.000
> 1.000
> 2.000
> 3.000
> 4.000
> 0.000 0.000 0.000 0.000 0.000
> 0.000 1.000 0.000 0.000 0.000
> 0.000 0.000 2.000 0.000 0.000
> 0.000 0.000 0.000 3.000 0.000
> 0.000 0.000 0.000 0.000 4.000
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-52) User Documentation Enhancement
[ https://issues.apache.org/jira/browse/SYSTEMML-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409871#comment-15409871 ] Deron Eriksson commented on SYSTEMML-52: The documentation has significantly been expanded since this issue was created. We now have a README, Quick Start Guide, Spark MLContext Programming Guide, Hadoop Batch Mode Guide, JMLC Guide, DML Language Reference, Beginner's Guide to DML and PyDML, Algorithms Reference, Debugger Guide, IDE Guide, Contributing to SystemML Guide, Troubleshooting Guide, and Release Process Guide. So, I will consider this issue resolved and closed. > User Documentation Enhancement > -- > > Key: SYSTEMML-52 > URL: https://issues.apache.org/jira/browse/SYSTEMML-52 > Project: SystemML > Issue Type: Epic >Reporter: Frederick Reiss >Assignee: Deron Eriksson > Fix For: SystemML 0.10 > > > Developer ramping up -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-52) User Documentation Enhancement
[ https://issues.apache.org/jira/browse/SYSTEMML-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson closed SYSTEMML-52. -- > User Documentation Enhancement > -- > > Key: SYSTEMML-52 > URL: https://issues.apache.org/jira/browse/SYSTEMML-52 > Project: SystemML > Issue Type: Epic >Reporter: Frederick Reiss >Assignee: Deron Eriksson > Fix For: SystemML 0.10 > > > Developer ramping up -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-52) User Documentation Enhancement
[ https://issues.apache.org/jira/browse/SYSTEMML-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-52. Resolution: Fixed Fix Version/s: SystemML 0.10 > User Documentation Enhancement > -- > > Key: SYSTEMML-52 > URL: https://issues.apache.org/jira/browse/SYSTEMML-52 > Project: SystemML > Issue Type: Epic >Reporter: Frederick Reiss >Assignee: Deron Eriksson > Fix For: SystemML 0.10 > > > Developer ramping up -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-56) Expand and organize documentation for R-like language syntax
[ https://issues.apache.org/jira/browse/SYSTEMML-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-56. Resolution: Fixed Fix Version/s: SystemML 0.10 The DML Language Reference has been ported to markdown and has undergone significant improvements since August 2015. Therefore I will close this issue, which I probably should have closed many months ago. > Expand and organize documentation for R-like language syntax > > > Key: SYSTEMML-56 > URL: https://issues.apache.org/jira/browse/SYSTEMML-56 > Project: SystemML > Issue Type: Task >Reporter: Frederick Reiss >Assignee: Deron Eriksson > Fix For: SystemML 0.10 > > Original Estimate: 40h > Remaining Estimate: 40h > > This task covers making a complete pass through the reference manual for the > R-like (DML) language syntax. After this task is complete, the reference > manual should have complete coverage of all major language features and > should have usable indexing and a complete table of contents. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-56) Expand and organize documentation for R-like language syntax
[ https://issues.apache.org/jira/browse/SYSTEMML-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson closed SYSTEMML-56. -- > Expand and organize documentation for R-like language syntax > > > Key: SYSTEMML-56 > URL: https://issues.apache.org/jira/browse/SYSTEMML-56 > Project: SystemML > Issue Type: Task >Reporter: Frederick Reiss >Assignee: Deron Eriksson > Fix For: SystemML 0.10 > > Original Estimate: 40h > Remaining Estimate: 40h > > This task covers making a complete pass through the reference manual for the > R-like (DML) language syntax. After this task is complete, the reference > manual should have complete coverage of all major language features and > should have usable indexing and a complete table of contents. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-649) JMLC/MLContext support for scalar output variables
[ https://issues.apache.org/jira/browse/SYSTEMML-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-649. - Resolution: Fixed Fix Version/s: SystemML 0.11 > JMLC/MLContext support for scalar output variables > -- > > Key: SYSTEMML-649 > URL: https://issues.apache.org/jira/browse/SYSTEMML-649 > Project: SystemML > Issue Type: Task > Components: APIs >Reporter: Matthias Boehm >Assignee: Deron Eriksson > Fix For: SystemML 0.11 > > > Right now neither JMLC nor MLContext supports scalar output variables. This > task aims to extend both APIs with the required primitives. > The workaround is to cast any output scalar on script-level with as.matrix to > a 1-1 matrix and handle it in the calling application. However, especially > with MLContext this puts an unnecessary burden on the user as he needs to > deal with RDDs for a simple scalar too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-649) JMLC/MLContext support for scalar output variables
[ https://issues.apache.org/jira/browse/SYSTEMML-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson closed SYSTEMML-649. --- > JMLC/MLContext support for scalar output variables > -- > > Key: SYSTEMML-649 > URL: https://issues.apache.org/jira/browse/SYSTEMML-649 > Project: SystemML > Issue Type: Task > Components: APIs >Reporter: Matthias Boehm >Assignee: Deron Eriksson > Fix For: SystemML 0.11 > > > Right now neither JMLC nor MLContext supports scalar output variables. This > task aims to extend both APIs with the required primitives. > The workaround is to cast any output scalar on script-level with as.matrix to > a 1-1 matrix and handle it in the calling application. However, especially > with MLContext this puts an unnecessary burden on the user as he needs to > deal with RDDs for a simple scalar too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-649) JMLC/MLContext support for scalar output variables
[ https://issues.apache.org/jira/browse/SYSTEMML-649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409817#comment-15409817 ] Deron Eriksson commented on SYSTEMML-649: - Scalars supported by new MLContext API (SYSTEMML-593). Fixed by [PR199|https://github.com/apache/incubator-systemml/pull/199]. > JMLC/MLContext support for scalar output variables > -- > > Key: SYSTEMML-649 > URL: https://issues.apache.org/jira/browse/SYSTEMML-649 > Project: SystemML > Issue Type: Task > Components: APIs >Reporter: Matthias Boehm >Assignee: Deron Eriksson > > Right now neither JMLC nor MLContext supports scalar output variables. This > task aims to extend both APIs with the required primitives. > The workaround is to cast any output scalar on script-level with as.matrix to > a 1-1 matrix and handle it in the calling application. However, especially > with MLContext this puts an unnecessary burden on the user as he needs to > deal with RDDs for a simple scalar too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SYSTEMML-848) Describe release candidate build and deployment
Deron Eriksson created SYSTEMML-848: --- Summary: Describe release candidate build and deployment Key: SYSTEMML-848 URL: https://issues.apache.org/jira/browse/SYSTEMML-848 Project: SystemML Issue Type: Task Components: Build, Documentation Reporter: Deron Eriksson Priority: Minor Describe the steps involved in building a release candidate and deploying the release candidate to Apache servers so that multiple project members can understand and help with the process. Add these steps to "Release Candidate Build and Deployment" section of Release Process document (release-process.md). See http://apache.github.io/incubator-systemml/release-process.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-848) Describe release candidate build and deployment
[ https://issues.apache.org/jira/browse/SYSTEMML-848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson updated SYSTEMML-848: Assignee: Luciano Resende > Describe release candidate build and deployment > --- > > Key: SYSTEMML-848 > URL: https://issues.apache.org/jira/browse/SYSTEMML-848 > Project: SystemML > Issue Type: Task > Components: Build, Documentation >Reporter: Deron Eriksson >Assignee: Luciano Resende >Priority: Minor > > Describe the steps involved in building a release candidate and deploying the > release candidate to Apache servers so that multiple project members can > understand and help with the process. > Add these steps to "Release Candidate Build and Deployment" section of > Release Process document (release-process.md). See > http://apache.github.io/incubator-systemml/release-process.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-713) Create release process document
[ https://issues.apache.org/jira/browse/SYSTEMML-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson closed SYSTEMML-713. --- > Create release process document > --- > > Key: SYSTEMML-713 > URL: https://issues.apache.org/jira/browse/SYSTEMML-713 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Deron Eriksson > Fix For: SystemML 0.11 > > > Describe the SystemML release process in a document so that the release > validation consists of well-defined, documented, reproducible steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-713) Create release process document
[ https://issues.apache.org/jira/browse/SYSTEMML-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-713. - Resolution: Fixed Fix Version/s: SystemML 0.11 The release-process.md document addresses the main steps involved in validating a release. See http://apache.github.io/incubator-systemml/release-process.html . See [PR170|https://github.com/apache/incubator-systemml/pull/170]. This document should be updated to include how to build a release candidate and how to deploy a release candidate to Apache servers. This can be addressed in a separate JIRA. > Create release process document > --- > > Key: SYSTEMML-713 > URL: https://issues.apache.org/jira/browse/SYSTEMML-713 > Project: SystemML > Issue Type: Task > Components: Documentation >Reporter: Deron Eriksson >Assignee: Deron Eriksson > Fix For: SystemML 0.11 > > > Describe the SystemML release process in a document so that the release > validation consists of well-defined, documented, reproducible steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-544) Create Convenience Abstractions for MLContext
[
https://issues.apache.org/jira/browse/SYSTEMML-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson closed SYSTEMML-544.
---
> Create Convenience Abstractions for MLContext
> -
>
> Key: SYSTEMML-544
> URL: https://issues.apache.org/jira/browse/SYSTEMML-544
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Mike Dusenberry
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Currently, our {{MLContext}} API has good capabilities, but requires
> boilerplate code that the end-user has to use. This JIRA aims to hide some
> of the boilerplate code under higher-level APIs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-544) Create Convenience Abstractions for MLContext
[
https://issues.apache.org/jira/browse/SYSTEMML-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson resolved SYSTEMML-544.
-
Resolution: Fixed
Fix Version/s: SystemML 0.11
MLContext boilerplate code has now mostly been eliminated through introduction
of the new MLContext API (SYSTEMML-593). Therefore I will close this JIRA since
all issues have been addressed.
> Create Convenience Abstractions for MLContext
> -
>
> Key: SYSTEMML-544
> URL: https://issues.apache.org/jira/browse/SYSTEMML-544
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Mike Dusenberry
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Currently, our {{MLContext}} API has good capabilities, but requires
> boilerplate code that the end-user has to use. This JIRA aims to hide some
> of the boilerplate code under higher-level APIs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-543) Refactor MLContext in Scala
[
https://issues.apache.org/jira/browse/SYSTEMML-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson resolved SYSTEMML-543.
-
Resolution: Won't Fix
Assignee: Deron Eriksson
Fix Version/s: SystemML 0.11
The MLContext API in Java has been refactored by SYSTEMML-593 (see
[PR199|https://github.com/apache/incubator-systemml/pull/199]) to remove
boilerplate code and to improve interaction via Scala (such as via Spark
Shell). Therefore a Scala API written in Scala current should not be needed.
> Refactor MLContext in Scala
> ---
>
> Key: SYSTEMML-543
> URL: https://issues.apache.org/jira/browse/SYSTEMML-543
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Mike Dusenberry
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Our {{MLContext}} API relies on a myriad of optional parameters as
> conveniences for end-users, which has led to our Java implementation growing
> in size. Moving to Scala will allow us to use default parameters and
> continue to expand the capabilities of the API in a clean way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-543) Refactor MLContext in Scala
[
https://issues.apache.org/jira/browse/SYSTEMML-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson closed SYSTEMML-543.
---
> Refactor MLContext in Scala
> ---
>
> Key: SYSTEMML-543
> URL: https://issues.apache.org/jira/browse/SYSTEMML-543
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Mike Dusenberry
>Assignee: Deron Eriksson
> Fix For: SystemML 0.11
>
>
> Our {{MLContext}} API relies on a myriad of optional parameters as
> conveniences for end-users, which has led to our Java implementation growing
> in size. Moving to Scala will allow us to use default parameters and
> continue to expand the capabilities of the API in a clean way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-847) Remove LogisticRegression and LogisticRegressionModel in api/javaml
[ https://issues.apache.org/jira/browse/SYSTEMML-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409609#comment-15409609 ] Glenn Weidner commented on SYSTEMML-847: This incorporates PR comments from [SYSTEMML-776] and also helps simplify potential issues between different Spark versions [SYSTEMML-774]. > Remove LogisticRegression and LogisticRegressionModel in api/javaml > --- > > Key: SYSTEMML-847 > URL: https://issues.apache.org/jira/browse/SYSTEMML-847 > Project: SystemML > Issue Type: Task > Components: APIs >Reporter: Glenn Weidner >Assignee: Glenn Weidner >Priority: Minor > > These classes have been replaced by corresponding versions in Scala and can > be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (SYSTEMML-847) Remove LogisticRegression and LogisticRegressionModel in api/javaml
[ https://issues.apache.org/jira/browse/SYSTEMML-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409609#comment-15409609 ] Glenn Weidner edited comment on SYSTEMML-847 at 8/5/16 3:57 PM: This incorporates PR comments from [SYSTEMML-776] and also helps simplify potential issues between different Spark versions [SYSTEMML-744]. was (Author: gweidner): This incorporates PR comments from [SYSTEMML-776] and also helps simplify potential issues between different Spark versions [SYSTEMML-774]. > Remove LogisticRegression and LogisticRegressionModel in api/javaml > --- > > Key: SYSTEMML-847 > URL: https://issues.apache.org/jira/browse/SYSTEMML-847 > Project: SystemML > Issue Type: Task > Components: APIs >Reporter: Glenn Weidner >Assignee: Glenn Weidner >Priority: Minor > > These classes have been replaced by corresponding versions in Scala and can > be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SYSTEMML-847) Remove LogisticRegression and LogisticRegressionModel in api/javaml
Glenn Weidner created SYSTEMML-847: -- Summary: Remove LogisticRegression and LogisticRegressionModel in api/javaml Key: SYSTEMML-847 URL: https://issues.apache.org/jira/browse/SYSTEMML-847 Project: SystemML Issue Type: Task Components: APIs Reporter: Glenn Weidner Assignee: Glenn Weidner Priority: Minor These classes have been replaced by corresponding versions in Scala and can be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SYSTEMML-846) Extend Python MLContext to register NumPy arrays as input/output
Niketan Pansare created SYSTEMML-846: Summary: Extend Python MLContext to register NumPy arrays as input/output Key: SYSTEMML-846 URL: https://issues.apache.org/jira/browse/SYSTEMML-846 Project: SystemML Issue Type: Task Reporter: Niketan Pansare -- This message was sent by Atlassian JIRA (v6.3.4#6332)
