[jira] [Resolved] (SYSTEMML-2004) Implement covariance kernels
[ https://issues.apache.org/jira/browse/SYSTEMML-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan resolved SYSTEMML-2004. - Resolution: Resolved addressed by [https://github.com/apache/systemml/pull/719|https://github.com/apache/systemml/pull/719,] , this pr contains a covariance matrix of a squared exponential kernel with out > Implement covariance kernels > > > Key: SYSTEMML-2004 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2004 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > Fix For: SystemML 1.1 > > > 1. square kernel > 2. 5/2 matern kernel -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SYSTEMML-1994) Implementation of Gaussian Process Regression
[ https://issues.apache.org/jira/browse/SYSTEMML-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1994: Description: Regression with Gaussian process. This script essentially takes in the input ( X, *y* ) (input is in matrix format), then # It calculates a covariance matrix, *K* # and a predictive mean and variance of a test point *x_star* (it is a single point). # a log marginal likelihood was: Regression with Gaussian process. This script essentially takes in the input \( X, *y* \) (input is in matrix format), then # It calculates a covariance matrix, *K* # and a predictive mean and variance of a test point *x_star* (it is a single point). > Implementation of Gaussian Process Regression > - > > Key: SYSTEMML-1994 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1994 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > Fix For: SystemML 1.1 > > > Regression with Gaussian process. > This script essentially takes in the input ( X, *y* ) (input is in matrix > format), then > # It calculates a covariance matrix, *K* > # and a predictive mean and variance of a test point *x_star* (it is a > single point). > # a log marginal likelihood -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SYSTEMML-1994) Implementation of Gaussian Process Regression
[ https://issues.apache.org/jira/browse/SYSTEMML-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1994: Description: Regression with Gaussian process. This script essentially takes in the input \( X, *y* \) (input is in matrix format), then # It calculates a covariance matrix, *K* # and a predictive mean and variance of a test point *x_star* (it is a single point). was:Regression with Gaussian process. > Implementation of Gaussian Process Regression > - > > Key: SYSTEMML-1994 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1994 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > Fix For: SystemML 1.1 > > > Regression with Gaussian process. > This script essentially takes in the input \( X, *y* \) (input is in matrix > format), then > # It calculates a covariance matrix, *K* > # and a predictive mean and variance of a test point *x_star* (it is a > single point). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SYSTEMML-1994) Implementation of Gaussian Process Regression
[ https://issues.apache.org/jira/browse/SYSTEMML-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-1994: --- Assignee: Janardhan > Implementation of Gaussian Process Regression > - > > Key: SYSTEMML-1994 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1994 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > Fix For: SystemML 1.1 > > > Regression with Gaussian process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1973: Comment: was deleted (was: Hi everyone, I am working on the hyperparameter optimization. This is the epic jira for the feature, with tasks and corresponding subtasks. a copy to : [~freiss] [~reinwald] [~dusenberrymw] [~prithvi_r_s] ) > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > This procedure constructs a probabilistic model for *f\(x\)*, and then > exploits this model to make decisions about where in input space to next > evaluate the function, while integrating out uncertainity. The essential > philosophy is to use all of the information available from previous > evaluations of *f\(x\)*. > > When performing Bayesian Optimization, [SYSTEMML-979] > 1. one must select a prior over functions that will express assumptions about > the function being optimized. – *We choose Gaussian Process Prior* > 2. need an acquisition function, which is used to construct a utility > function from the model posterior, allowing us to determine the next point to > evaluate. > > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > Once the bayesian optimization is supported, the module is integrated into > our API as described in SYSTEMML-1962 . By wrapping the dml functions in the > optimization algorithms and invoking them either by java or python scripts. > > *Phase 3:* Addition of Optimization test functions. > Testing of the training is done with the help of the well known benchmark > functions, SYSTEMML-1974 , which can be imported or can be invoked with the > help of python scripts or just by importing the function into the dml script > at hand. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1973: Comment: was deleted (was: This is the central jira for any or all of the feature related. cc: [~niketanpansare]) > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > This procedure constructs a probabilistic model for *f\(x\)*, and then > exploits this model to make decisions about where in input space to next > evaluate the function, while integrating out uncertainity. The essential > philosophy is to use all of the information available from previous > evaluations of *f\(x\)*. > > When performing Bayesian Optimization, [SYSTEMML-979] > 1. one must select a prior over functions that will express assumptions about > the function being optimized. – *We choose Gaussian Process Prior* > 2. need an acquisition function, which is used to construct a utility > function from the model posterior, allowing us to determine the next point to > evaluate. > > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > Once the bayesian optimization is supported, the module is integrated into > our API as described in SYSTEMML-1962 . By wrapping the dml functions in the > optimization algorithms and invoking them either by java or python scripts. > > *Phase 3:* Addition of Optimization test functions. > Testing of the training is done with the help of the well known benchmark > functions, SYSTEMML-1974 , which can be imported or can be invoked with the > help of python scripts or just by importing the function into the dml script > at hand. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1973: Description: This Epic tracks the algorithm optimization related improvements, and their testing. *Phase 1:* Addition of support for bayesian optimization. This procedure constructs a probabilistic model for *f\(x\)*, and then exploits this model to make decisions about where in input space to next evaluate the function, while integrating out uncertainity. The essential philosophy is to use all of the information available from previous evaluations of *f\(x\)*. When performing Bayesian Optimization, **1. one must select a prior over functions that will express assumptions about the function being optimized. – *We choose Gaussian Process Prior* 2. need an acquisition function, which is used to construct a utility function from the model posterior, allowing us to determine the next point to evaluate. *Phase 2:* Addition of Model selection & cross validation support at Engine level or API side. Once the bayesian optimization is supported, the module is integrated into our API as described in SYSTEMML-1962 . By wrapping the dml functions in the optimization algorithms and invoking them either by java or python scripts. *Phase 3:* Addition of Optimization test functions. Testing of the training is done with the help of the well known benchmark functions, SYSTEMML-1974 , which can be imported or can be invoked with the help of python scripts or just by importing the function into the dml script at hand. was: This Epic tracks the algorithm optimization related improvements, and their testing. *Phase 1:* Addition of support for bayesian optimization. This procedure constructs a probabilistic model for f(x), and then exploits this model to make decisions about where in input space to next evaluate the function, while integrating out uncertainity. The essential philosophy is to use all of the information available from previous evaluations of f(x). When performing Bayesian Optimization, **1. one must select a prior over functions that will express assumptions about the function being optimized. – *We choose Gaussian Process Prior* 2. need an acquisition function, which is used to construct a utility function from the model posterior, allowing us to determine the next point to evaluate. *Phase 2:* Addition of Model selection & cross validation support at Engine level or API side. Once the bayesian optimization is supported, the module is integrated into our API as described in SYSTEMML-1962 . By wrapping the dml functions in the optimization algorithms and invoking them either by java or python scripts. *Phase 3:* Addition of Optimization test functions. Testing of the training is done with the help of the well known benchmark functions, SYSTEMML-1974 , which can be imported or can be invoked with the help of python scripts or just by importing the function into the dml script at hand. > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > This procedure constructs a probabilistic model for *f\(x\)*, and then > exploits this model to make decisions about where in input space to next > evaluate the function, while integrating out uncertainity. The essential > philosophy is to use all of the information available from previous > evaluations of *f\(x\)*. > > When performing Bayesian Optimization, > **1. one must select a prior over functions that will express assumptions > about the function being optimized. – *We choose Gaussian Process Prior* > 2. need an acquisition function, which is used to construct a utility > function from the model posterior, allowing us to determine the next point to > evaluate. > > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > Once the bayesian optimization is supported, the module is integrated into > our API as described in SYSTEMML-1962 . By wrapping the dml functions in the > optimization algorithms and invoking them either by java or python scripts. > > *Phase 3:* Addition of Optimization test functions. > Testing of the training is done with the help of the well known benchmark > functions, SYSTEMML-1974 , which can be imported or can be invoked with the > help of python scripts or just by importing the function
[jira] [Updated] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1973: Description: This Epic tracks the algorithm optimization related improvements, and their testing. *Phase 1:* Addition of support for bayesian optimization. This procedure constructs a probabilistic model for f(x), and then exploits this model to make decisions about where in input space to next evaluate the function, while integrating out uncertainity. The essential philosophy is to use all of the information available from previous evaluations of f(x). When performing Bayesian Optimization, **1. one must select a prior over functions that will express assumptions about the function being optimized. – *We choose Gaussian Process Prior* 2. need an acquisition function, which is used to construct a utility function from the model posterior, allowing us to determine the next point to evaluate. *Phase 2:* Addition of Model selection & cross validation support at Engine level or API side. Once the bayesian optimization is supported, the module is integrated into our API as described in SYSTEMML-1962 . By wrapping the dml functions in the optimization algorithms and invoking them either by java or python scripts. *Phase 3:* Addition of Optimization test functions. Testing of the training is done with the help of the well known benchmark functions, SYSTEMML-1974 , which can be imported or can be invoked with the help of python scripts or just by importing the function into the dml script at hand. was: This Epic tracks the algorithm optimization related improvements, and their testing. *Phase 1:* Addition of support for bayesian optimization. *Phase 2:* Addition of Model selection & cross validation support at Engine level or API side. Once the bayesian optimization is supported, the module is integrated into our API, *Phase 3:* Addition of Optimization test functions. > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > This procedure constructs a probabilistic model for f(x), and then exploits > this model to make decisions about where in input space to next evaluate the > function, while integrating out uncertainity. The essential philosophy is to > use all of the information available from previous evaluations of f(x). > > When performing Bayesian Optimization, > **1. one must select a prior over functions that will express assumptions > about the function being optimized. – *We choose Gaussian Process Prior* > 2. need an acquisition function, which is used to construct a utility > function from the model posterior, allowing us to determine the next point to > evaluate. > > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > Once the bayesian optimization is supported, the module is integrated into > our API as described in SYSTEMML-1962 . By wrapping the dml functions in the > optimization algorithms and invoking them either by java or python scripts. > > *Phase 3:* Addition of Optimization test functions. > Testing of the training is done with the help of the well known benchmark > functions, SYSTEMML-1974 , which can be imported or can be invoked with the > help of python scripts or just by importing the function into the dml script > at hand. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1973: Description: This Epic tracks the algorithm optimization related improvements, and their testing. *Phase 1:* Addition of support for bayesian optimization. *Phase 2:* Addition of Model selection & cross validation support at Engine level or API side. Once the bayesian optimization is supported, the module is integrated into our API, *Phase 3:* Addition of Optimization test functions. was: This Epic tracks the algorithm optimization related improvements, and their testing. *Phase 1:* Addition of support for bayesian optimization. *Phase 2:* Addition of Model selection & cross validation support at Engine level or API side. *Phase 3:* Addition of Optimization test functions. > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > Once the bayesian optimization is supported, the module is integrated into > our API, > > *Phase 3:* Addition of Optimization test functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-2083:
Description:
SystemML already provides a rich set of execution strategies ranging from local
operations to large-scale computation on MapReduce or Spark. In this context,
we support both data-parallel (multi-threaded or distributed operations) as
well as task-parallel computation (multi-threaded or distributed parfor loops).
This epic aims to complement the existing execution strategies by language and
runtime primitives for parameter servers, i.e., model-parallel execution. We
use the terminology of model-parallel execution with distributed data and
distributed model to differentiate them from the existing data-parallel
operations. Target applications are distributed deep learning and mini-batch
algorithms in general. These new abstractions will help making SystemML a
unified framework for small- and large-scale machine learning that supports all
three major execution strategies in a single framework.
A major challenge is the integration of stateful parameter servers and their
common push/pull primitives into an otherwise functional (and thus, stateless)
language. We will approach this challenge via a new builtin function
{{paramserv}} which internally maintains state but at the same time fits into
the runtime framework of stateless operations.
Furthermore, we are interested in providing (1) different runtime backends
(local and distributed), (2) different parameter server modes (synchronous,
asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
(batch, multi-batch, epoch), as well as (4) different architectures for
distributed data (1 parameter server, k workers) and distributed model (k1
parameter servers, k2 workers).
*Note for GSOC students:* This is large project which will be broken down into
sub projects, so everybody will be having their share of pie.
*Prerequistes:* Java, machine learning experience is a plus but not required.
was:
SystemML already provides a rich set of execution strategies ranging from local
operations to large-scale computation on MapReduce or Spark. In this context,
we support both data-parallel (multi-threaded or distributed operations) as
well as task-parallel computation (multi-threaded or distributed parfor loops).
This epic aims to complement the existing execution strategies by language and
runtime primitives for parameter servers, i.e., model-parallel execution. We
use the terminology of model-parallel execution with distributed data and
distributed model to differentiate them from the existing data-parallel
operations. Target applications are distributed deep learning and mini-batch
algorithms in general. These new abstractions will help making SystemML a
unified framework for small- and large-scale machine learning that supports all
three major execution strategies in a single framework.
A major challenge is the integration of stateful parameter servers and their
common push/pull primitives into an otherwise functional (and thus, stateless)
language. We will approach this challenge via a new builtin function
{{paramserv}} which internally maintains state but at the same time fits into
the runtime framework of stateless operations.
Furthermore, we are interested in providing (1) different runtime backends
(local and distributed), (2) different parameter server modes (synchronous,
asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
(batch, multi-batch, epoch), as well as (4) different architectures for
distributed data (1 parameter server, k workers) and distributed model (k1
parameter servers, k2 workers).
*Note for GSOC students:* This is large project which will be broken down into
sub projects, so everybody will be having their share of pie.
*Prerequistes:* Java, machine learning experience experience is a plus but not
required.
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png, image-2018-02-14-12-31-37-563.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
[jira] [Updated] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-2083:
Description:
SystemML already provides a rich set of execution strategies ranging from local
operations to large-scale computation on MapReduce or Spark. In this context,
we support both data-parallel (multi-threaded or distributed operations) as
well as task-parallel computation (multi-threaded or distributed parfor loops).
This epic aims to complement the existing execution strategies by language and
runtime primitives for parameter servers, i.e., model-parallel execution. We
use the terminology of model-parallel execution with distributed data and
distributed model to differentiate them from the existing data-parallel
operations. Target applications are distributed deep learning and mini-batch
algorithms in general. These new abstractions will help making SystemML a
unified framework for small- and large-scale machine learning that supports all
three major execution strategies in a single framework.
A major challenge is the integration of stateful parameter servers and their
common push/pull primitives into an otherwise functional (and thus, stateless)
language. We will approach this challenge via a new builtin function
{{paramserv}} which internally maintains state but at the same time fits into
the runtime framework of stateless operations.
Furthermore, we are interested in providing (1) different runtime backends
(local and distributed), (2) different parameter server modes (synchronous,
asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
(batch, multi-batch, epoch), as well as (4) different architectures for
distributed data (1 parameter server, k workers) and distributed model (k1
parameter servers, k2 workers).
*Note for GSOC students:* This is large project which will be broken down into
sub projects, so everybody will be having their share of pie.
*Prerequistes:* Java, machine learning experience experience is a plus but not
required.
was:
SystemML already provides a rich set of execution strategies ranging from local
operations to large-scale computation on MapReduce or Spark. In this context,
we support both data-parallel (multi-threaded or distributed operations) as
well as task-parallel computation (multi-threaded or distributed parfor loops).
This epic aims to complement the existing execution strategies by language and
runtime primitives for parameter servers, i.e., model-parallel execution. We
use the terminology of model-parallel execution with distributed data and
distributed model to differentiate them from the existing data-parallel
operations. Target applications are distributed deep learning and mini-batch
algorithms in general. These new abstractions will help making SystemML a
unified framework for small- and large-scale machine learning that supports all
three major execution strategies in a single framework.
A major challenge is the integration of stateful parameter servers and their
common push/pull primitives into an otherwise functional (and thus, stateless)
language. We will approach this challenge via a new builtin function
{{paramserv}} which internally maintains state but at the same time fits into
the runtime framework of stateless operations.
Furthermore, we are interested in providing (1) different runtime backends
(local and distributed), (2) different parameter server modes (synchronous,
asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
(batch, multi-batch, epoch), as well as (4) different architectures for
distributed data (1 parameter server, k workers) and distributed model (k1
parameter servers, k2 workers).
*Note for GSOC students:* This is large project which will be broken down into
sub projects, so everybody will be having their share of pie.
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png, image-2018-02-14-12-31-37-563.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
>
[jira] [Updated] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-2083:
Description:
SystemML already provides a rich set of execution strategies ranging from local
operations to large-scale computation on MapReduce or Spark. In this context,
we support both data-parallel (multi-threaded or distributed operations) as
well as task-parallel computation (multi-threaded or distributed parfor loops).
This epic aims to complement the existing execution strategies by language and
runtime primitives for parameter servers, i.e., model-parallel execution. We
use the terminology of model-parallel execution with distributed data and
distributed model to differentiate them from the existing data-parallel
operations. Target applications are distributed deep learning and mini-batch
algorithms in general. These new abstractions will help making SystemML a
unified framework for small- and large-scale machine learning that supports all
three major execution strategies in a single framework.
A major challenge is the integration of stateful parameter servers and their
common push/pull primitives into an otherwise functional (and thus, stateless)
language. We will approach this challenge via a new builtin function
{{paramserv}} which internally maintains state but at the same time fits into
the runtime framework of stateless operations.
Furthermore, we are interested in providing (1) different runtime backends
(local and distributed), (2) different parameter server modes (synchronous,
asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
(batch, multi-batch, epoch), as well as (4) different architectures for
distributed data (1 parameter server, k workers) and distributed model (k1
parameter servers, k2 workers).
*Note for GSOC students:* This is large project which will be broken down into
sub projects, so everybody will be having their share of pie.
was:
SystemML already provides a rich set of execution strategies ranging from local
operations to large-scale computation on MapReduce or Spark. In this context,
we support both data-parallel (multi-threaded or distributed operations) as
well as task-parallel computation (multi-threaded or distributed parfor loops).
This epic aims to complement the existing execution strategies by language and
runtime primitives for parameter servers, i.e., model-parallel execution. We
use the terminology of model-parallel execution with distributed data and
distributed model to differentiate them from the existing data-parallel
operations. Target applications are distributed deep learning and mini-batch
algorithms in general. These new abstractions will help making SystemML a
unified framework for small- and large-scale machine learning that supports all
three major execution strategies in a single framework.
A major challenge is the integration of stateful parameter servers and their
common push/pull primitives into an otherwise functional (and thus, stateless)
language. We will approach this challenge via a new builtin function
\{{paramserv}} which internally maintains state but at the same time fits into
the runtime framework of stateless operations.
Furthermore, we are interested in providing (1) different runtime backends
(local and distributed), (2) different parameter server modes (synchronous,
asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
(batch, multi-batch, epoch), as well as (4) different architectures for
distributed data (1 parameter server, k workers) and distributed model (k1
parameter servers, k2 workers).
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png, image-2018-02-14-12-31-37-563.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new
[jira] [Comment Edited] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363568#comment-16363568
]
Janardhan edited comment on SYSTEMML-2083 at 2/14/18 7:04 AM:
--
The light weight parameter server interface is
[ps-lite|[https://github.com/dmlc/ps-lite].] ] as a simple example.
In simple terms (takes 7 mins to read this explanation), let's say we have
{code:java}
to caculate weights, with help of gradients.{code}
1. How parameter server looks? contains workers, server and data.
!image-2018-02-14-12-18-48-932.png!
2. What worker do? takes a little data & *calculates gradients* from it & sends
them to server.
!image-2018-02-14-12-21-00-932.png!
3. What server do? get the gradients from workers and *calculates weights*.
!image-2018-02-14-12-31-37-563.png!
was (Author: return_01):
The light weight parameter server interface is
[ps-lite|[https://github.com/dmlc/ps-lite].] ] as a simple example.
In simple terms, let's say we have (7 min read)
{code:java}
to caculate weights, with help of gradients.{code}
1. How parameter server looks? contains workers, server and data.
!image-2018-02-14-12-18-48-932.png!
2. What worker do? takes a little data & *calculates gradients* from it & sends
them to server.
!image-2018-02-14-12-21-00-932.png!
3. What server do? get the gradients from workers and *calculates weights*.
!image-2018-02-14-12-31-37-563.png!
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png, image-2018-02-14-12-31-37-563.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-2083:
Comment: was deleted
(was: The light weight parameter server interface is
[ps-lite|[https://github.com/dmlc/ps-lite|https://github.com/dmlc/ps-lite].] ]
as a simple example.
In simple terms, let's say we have (7 min read)
{code:java}
to caculate weights, with help of gradients.{code}
1. How parameter server looks? contains workers, server and data.
!image-2018-02-14-12-18-48-932.png!
2. What worker do? takes a little data & *calculates gradients* from it & sends
them to server.
!image-2018-02-14-12-21-00-932.png!
3. What server do? get the gradients from workers and *calculates weights*.
!image-2018-02-14-12-22-39-736.png!
)
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png, image-2018-02-14-12-31-37-563.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363568#comment-16363568
]
Janardhan edited comment on SYSTEMML-2083 at 2/14/18 7:01 AM:
--
The light weight parameter server interface is
[ps-lite|[https://github.com/dmlc/ps-lite].] ] as a simple example.
In simple terms, let's say we have (7 min read)
{code:java}
to caculate weights, with help of gradients.{code}
1. How parameter server looks? contains workers, server and data.
!image-2018-02-14-12-18-48-932.png!
2. What worker do? takes a little data & *calculates gradients* from it & sends
them to server.
!image-2018-02-14-12-21-00-932.png!
3. What server do? get the gradients from workers and *calculates weights*.
!image-2018-02-14-12-31-37-563.png!
was (Author: return_01):
The light weight parameter server interface is
[ps-lite|[https://github.com/dmlc/ps-lite|https://github.com/dmlc/ps-lite].] ]
as a simple example.
In simple terms, let's say we have (7 min read)
{code:java}
to caculate weights, with help of gradients.{code}
1. How parameter server looks? contains workers, server and data.
!image-2018-02-14-12-18-48-932.png!
2. What worker do? takes a little data & *calculates gradients* from it & sends
them to server.
!image-2018-02-14-12-21-00-932.png!
3. What server do? get the gradients from workers and *calculates weights*.
!image-2018-02-14-12-22-39-736.png!
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png, image-2018-02-14-12-31-37-563.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363567#comment-16363567
]
Janardhan commented on SYSTEMML-2083:
-
The light weight parameter server interface is
[ps-lite|[https://github.com/dmlc/ps-lite|https://github.com/dmlc/ps-lite].] ]
as a simple example.
In simple terms, let's say we have (7 min read)
{code:java}
to caculate weights, with help of gradients.{code}
1. How parameter server looks? contains workers, server and data.
!image-2018-02-14-12-18-48-932.png!
2. What worker do? takes a little data & *calculates gradients* from it & sends
them to server.
!image-2018-02-14-12-21-00-932.png!
3. What server do? get the gradients from workers and *calculates weights*.
!image-2018-02-14-12-22-39-736.png!
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363568#comment-16363568
]
Janardhan commented on SYSTEMML-2083:
-
The light weight parameter server interface is
[ps-lite|[https://github.com/dmlc/ps-lite|https://github.com/dmlc/ps-lite].] ]
as a simple example.
In simple terms, let's say we have (7 min read)
{code:java}
to caculate weights, with help of gradients.{code}
1. How parameter server looks? contains workers, server and data.
!image-2018-02-14-12-18-48-932.png!
2. What worker do? takes a little data & *calculates gradients* from it & sends
them to server.
!image-2018-02-14-12-21-00-932.png!
3. What server do? get the gradients from workers and *calculates weights*.
!image-2018-02-14-12-22-39-736.png!
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png,
> image-2018-02-14-12-21-00-932.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-2083:
Attachment: image-2018-02-14-12-18-48-932.png
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
> Attachments: image-2018-02-14-12-18-48-932.png
>
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (SYSTEMML-2110) Add support for relu operations (forward/backward)
[ https://issues.apache.org/jira/browse/SYSTEMML-2110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-2110: --- Assignee: Janardhan > Add support for relu operations (forward/backward) > -- > > Key: SYSTEMML-2110 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2110 > Project: SystemML > Issue Type: Sub-task >Reporter: Matthias Boehm >Assignee: Janardhan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SYSTEMML-2130) Primitives to check the validity of sparse block representations
[
https://issues.apache.org/jira/browse/SYSTEMML-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reassigned SYSTEMML-2130:
---
Assignee: Janardhan
> Primitives to check the validity of sparse block representations
>
>
> Key: SYSTEMML-2130
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2130
> Project: SystemML
> Issue Type: Task
>Reporter: Matthias Boehm
>Assignee: Janardhan
>Priority: Major
>
> This task aims to improve the debugging of our existing sparse block
> representations (MCSR, CSR, COO). We already have internal primitives such as
> {{ProgramBlock.checkSparsity}} and {{MatrixBlock.checkSparseRows}}, which are
> by default disabled but enabled on demand for debugging purposes.
> In detail, it would be useful to extend the {{SparseBlock}} abstraction by a
> method {{checkValidity(int rlen, int clen, boolean strict)}} in order to
> validate the correctness of the internal data structures of the different
> sparse block implementations. For example, for CSR this would entail checks
> for (1) correct meta data, (2) correct array lengths, (3) non-decreasing row
> pointers, (4) sorted column indexes per row, (5) non-existing zero values,
> and (6) a capacity that is no larger than nnz times resize factor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SYSTEMML-2102) Vectorize gradients for Factorization Machines function
[ https://issues.apache.org/jira/browse/SYSTEMML-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349092#comment-16349092 ] Janardhan commented on SYSTEMML-2102: - I will come to this shortly. Thanks Mike. > Vectorize gradients for Factorization Machines function > --- > > Key: SYSTEMML-2102 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2102 > Project: SystemML > Issue Type: Improvement >Reporter: Mike Dusenberry >Assignee: Janardhan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SYSTEMML-2102) Vectorize gradients for Factorization Machines function
[ https://issues.apache.org/jira/browse/SYSTEMML-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-2102: --- Assignee: Janardhan > Vectorize gradients for Factorization Machines function > --- > > Key: SYSTEMML-2102 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2102 > Project: SystemML > Issue Type: Improvement >Reporter: Mike Dusenberry >Assignee: Janardhan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (SYSTEMML-1695) Create design/investigation document for parameter server
[ https://issues.apache.org/jira/browse/SYSTEMML-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan resolved SYSTEMML-1695. - Resolution: Fixed Fix Version/s: SystemML 1.1 this design will be implemented actionable at https://issues.apache.org/jira/browse/SYSTEMML-2083 > Create design/investigation document for parameter server > - > > Key: SYSTEMML-1695 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1695 > Project: SystemML > Issue Type: Sub-task >Reporter: Niketan Pansare >Assignee: Janardhan >Priority: Major > Fix For: SystemML 1.1 > > > Here are relevant links: > - https://issues.apache.org/jira/browse/SPARK-4590 > - https://issues.apache.org/jira/browse/SPARK-6932 > - > https://docs.google.com/document/d/1SX3nkmF41wFXAAIr9BgqvrHSS5mW362fJ7roBXJm06o/edit > This document need not cover how to integrate a parameter server in SystemML > (which comes later), but should discuss following questions: > 1. Should we integrate existing parameter server implementation or build one > from scratch ? What are the pros and cons of each approaches ? For example: > https://github.com/rjagerman/glint > 2. If we plan to integrate existing parameter server implementation, what is > the interface ? (for exampe: update(matrix, new matrix) or update(matrix, > delta)). > 3. Can parameter server help in ML algorithms ? Some experiments might be > helpful. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345279#comment-16345279
]
Janardhan edited comment on SYSTEMML-2083 at 1/30/18 3:59 PM:
--
Hi all, this issue is a replica of the other two issues that I've assigned
myself for.
# the jiras are - https://issues.apache.org/jira/browse/SYSTEMML-739
was (Author: return_01):
Hi all, this issue is a replica of the other two issues that I've assigned
myself for.
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SYSTEMML-2083) Language and runtime for parameter servers
[
https://issues.apache.org/jira/browse/SYSTEMML-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345279#comment-16345279
]
Janardhan commented on SYSTEMML-2083:
-
Hi all, this issue is a replica of the other two issues that I've assigned
myself for.
> Language and runtime for parameter servers
> --
>
> Key: SYSTEMML-2083
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2083
> Project: SystemML
> Issue Type: Epic
>Reporter: Matthias Boehm
>Priority: Major
> Labels: gsoc2018
>
> SystemML already provides a rich set of execution strategies ranging from
> local operations to large-scale computation on MapReduce or Spark. In this
> context, we support both data-parallel (multi-threaded or distributed
> operations) as well as task-parallel computation (multi-threaded or
> distributed parfor loops). This epic aims to complement the existing
> execution strategies by language and runtime primitives for parameter
> servers, i.e., model-parallel execution. We use the terminology of
> model-parallel execution with distributed data and distributed model to
> differentiate them from the existing data-parallel operations. Target
> applications are distributed deep learning and mini-batch algorithms in
> general. These new abstractions will help making SystemML a unified framework
> for small- and large-scale machine learning that supports all three major
> execution strategies in a single framework.
>
> A major challenge is the integration of stateful parameter servers and their
> common push/pull primitives into an otherwise functional (and thus,
> stateless) language. We will approach this challenge via a new builtin
> function \{{paramserv}} which internally maintains state but at the same time
> fits into the runtime framework of stateless operations.
> Furthermore, we are interested in providing (1) different runtime backends
> (local and distributed), (2) different parameter server modes (synchronous,
> asynchronous, hogwild!, stale-synchronous), (3) different update frequencies
> (batch, multi-batch, epoch), as well as (4) different architectures for
> distributed data (1 parameter server, k workers) and distributed model (k1
> parameter servers, k2 workers).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SYSTEMML-1216) implement local svd function
[
https://issues.apache.org/jira/browse/SYSTEMML-1216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-1216:
Fix Version/s: SystemML 1.1
> implement local svd function
>
>
> Key: SYSTEMML-1216
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1216
> Project: SystemML
> Issue Type: New Feature
>Reporter: Imran Younus
>Assignee: Janardhan
>Priority: Major
> Fix For: SystemML 1.1
>
> Attachments: svd.txt
>
>
> SystemML currently provides several local matrix decompositions (qr(), lu(),
> cholesky()). But local version of svd is missing. This is also needed to
> scalable SVD implementation.
> Also, implement local {{svd()}} function with {{cusolver}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (SYSTEMML-1784) Predict Earthquakes and their magnitudes.
[
https://issues.apache.org/jira/browse/SYSTEMML-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-1784:
Comment: was deleted
(was: I will be working on this, as my academic project for the session July
2017, to July 2018. I hope this will also be worked on by the other folks from
the required specializations too.)
> Predict Earthquakes and their magnitudes.
> -
>
> Key: SYSTEMML-1784
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1784
> Project: SystemML
> Issue Type: Epic
> Components: Algorithms
>Reporter: Janardhan
>Assignee: Janardhan
>Priority: Major
> Attachments: Earthquakes magnitude predication using artificial
> neural network in northern Red Sea area..pdf
>
>
> h3. Predicting Earthquakes with Apache SystemML.
> _{color:#707070}Earthquakes take a dreadful human toll. Some 10,000 people
> die each year in earthquakes and their aftermath, but the toll can be much
> higher. Over 230,000 people died in the tsunami that followed the magnitude 9
> quake off the coast of Sumatra in 2004; more than 200,000 died in Haiti in
> 2010 after the country was hit by a magnitude 7 quake; and more than 800,000
> are thought to have died in a quake in China in 1556.^1^{color}_
> So a better way—any way—to forecast quakes would be hugely valuable.
> With the recent advancements in research in the fields and after some
> successful predictions, we strongly believe that it is possible to predict an
> earthquake.
> References:
> 1.
> https://www.technologyreview.com/s/603785/machine-learning-algorithm-predicts-laboratory-earthquakes/
> Goal & Approach:
> 1. dx.doi.org/10.1016/j.jksus.2011.05.002
> 2. https://arxiv.org/ftp/arxiv/papers/1702/1702.05774.pdf
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (SYSTEMML-1784) Predict Earthquakes and their magnitudes.
[
https://issues.apache.org/jira/browse/SYSTEMML-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-1784:
Comment: was deleted
(was: In some days, I will come up with a design report on how it is related to
SystemML and approach with required components.)
> Predict Earthquakes and their magnitudes.
> -
>
> Key: SYSTEMML-1784
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1784
> Project: SystemML
> Issue Type: Epic
> Components: Algorithms
>Reporter: Janardhan
>Assignee: Janardhan
>Priority: Major
> Attachments: Earthquakes magnitude predication using artificial
> neural network in northern Red Sea area..pdf
>
>
> h3. Predicting Earthquakes with Apache SystemML.
> _{color:#707070}Earthquakes take a dreadful human toll. Some 10,000 people
> die each year in earthquakes and their aftermath, but the toll can be much
> higher. Over 230,000 people died in the tsunami that followed the magnitude 9
> quake off the coast of Sumatra in 2004; more than 200,000 died in Haiti in
> 2010 after the country was hit by a magnitude 7 quake; and more than 800,000
> are thought to have died in a quake in China in 1556.^1^{color}_
> So a better way—any way—to forecast quakes would be hugely valuable.
> With the recent advancements in research in the fields and after some
> successful predictions, we strongly believe that it is possible to predict an
> earthquake.
> References:
> 1.
> https://www.technologyreview.com/s/603785/machine-learning-algorithm-predicts-laboratory-earthquakes/
> Goal & Approach:
> 1. dx.doi.org/10.1016/j.jksus.2011.05.002
> 2. https://arxiv.org/ftp/arxiv/papers/1702/1702.05774.pdf
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (SYSTEMML-2057) Support bitwise operators not, and, or, xor, & LShift, Rshift
[ https://issues.apache.org/jira/browse/SYSTEMML-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan resolved SYSTEMML-2057. - Resolution: Fixed Fix Version/s: SystemML 1.1 > Support bitwise operators not, and, or, xor, & LShift, Rshift > - > > Key: SYSTEMML-2057 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2057 > Project: SystemML > Issue Type: Task > Components: Parser, Runtime >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > Fix For: SystemML 1.1 > > > 1. bitwNot(a) > 2. bitwAnd(a, b) > 3. bitwOr(a, b) > 4. bitwXor(a, b) > 5. bitwShiftL(a, n) > 6. bitwShiftR(a, n) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SYSTEMML-1159) Enable Remote Hyperparameter Tuning
[
https://issues.apache.org/jira/browse/SYSTEMML-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reassigned SYSTEMML-1159:
---
Assignee: Janardhan
> Enable Remote Hyperparameter Tuning
> ---
>
> Key: SYSTEMML-1159
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1159
> Project: SystemML
> Issue Type: Improvement
>Affects Versions: SystemML 1.1
>Reporter: Mike Dusenberry
>Assignee: Janardhan
>Priority: Blocker
>
> Training a parameterized machine learning model (such as a large neural net
> in deep learning) requires learning a set of ideal model parameters from the
> data, as well as determining appropriate hyperparameters (or "settings") for
> the training process itself. In the latter case, the hyperparameters (i.e.
> learning rate, regularization strength, dropout percentage, model
> architecture, etc.) can not be learned from the data, and instead are
> determined via a search across a space for each hyperparameter. For large
> numbers of hyperparameters (such as in deep learning models), the current
> literature points to performing staged, randomized grid searches over the
> space to produce distributions of performance, narrowing the space after each
> search \[1]. Thus, for efficient hyperparameter optimization, it is
> desirable to train several models in parallel, with each model trained over
> the full dataset. For deep learning models, a mini-batch training approach
> is currently state-of-the-art, and thus separate models with different
> hyperparameters could, conceivably, be easily trained on each of the nodes in
> a cluster.
> In order to allow for the training of deep learning models, SystemML needs to
> determine a solution to enable this scenario with the Spark backend.
> Specifically, if the user has a {{train}} function that takes a set of
> hyperparameters and trains a model with a mini-batch approach (and thus is
> only making use of single-node instructions within the function), the user
> should be able to wrap this function with, for example, a remote {{parfor}}
> construct that samples hyperparameters and calls the {{train}} function on
> each machine in parallel.
> To be clear, each model would need access to the entire dataset, and each
> model would be trained independently.
> \[1]: http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SYSTEMML-1159) Enable Remote Hyperparameter Tuning
[
https://issues.apache.org/jira/browse/SYSTEMML-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341095#comment-16341095
]
Janardhan commented on SYSTEMML-1159:
-
This Jira is also a priority for me. And will be handling soon. I would like to
take help that I could get from the community. I am assigning myself, now.
> Enable Remote Hyperparameter Tuning
> ---
>
> Key: SYSTEMML-1159
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1159
> Project: SystemML
> Issue Type: Improvement
>Affects Versions: SystemML 1.1
>Reporter: Mike Dusenberry
>Priority: Blocker
>
> Training a parameterized machine learning model (such as a large neural net
> in deep learning) requires learning a set of ideal model parameters from the
> data, as well as determining appropriate hyperparameters (or "settings") for
> the training process itself. In the latter case, the hyperparameters (i.e.
> learning rate, regularization strength, dropout percentage, model
> architecture, etc.) can not be learned from the data, and instead are
> determined via a search across a space for each hyperparameter. For large
> numbers of hyperparameters (such as in deep learning models), the current
> literature points to performing staged, randomized grid searches over the
> space to produce distributions of performance, narrowing the space after each
> search \[1]. Thus, for efficient hyperparameter optimization, it is
> desirable to train several models in parallel, with each model trained over
> the full dataset. For deep learning models, a mini-batch training approach
> is currently state-of-the-art, and thus separate models with different
> hyperparameters could, conceivably, be easily trained on each of the nodes in
> a cluster.
> In order to allow for the training of deep learning models, SystemML needs to
> determine a solution to enable this scenario with the Spark backend.
> Specifically, if the user has a {{train}} function that takes a set of
> hyperparameters and trains a model with a mini-batch approach (and thus is
> only making use of single-node instructions within the function), the user
> should be able to wrap this function with, for example, a remote {{parfor}}
> construct that samples hyperparameters and calls the {{train}} function on
> each machine in parallel.
> To be clear, each model would need access to the entire dataset, and each
> model would be trained independently.
> \[1]: http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (SYSTEMML-1992) Implemenation of Mode finding for Gaussian Process
[ https://issues.apache.org/jira/browse/SYSTEMML-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan resolved SYSTEMML-1992. - Resolution: Implemented addressed by https://github.com/apache/systemml/commit/997eb2aa261b11857983b1eca8bc43dc33719589 > Implemenation of Mode finding for Gaussian Process > -- > > Key: SYSTEMML-1992 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1992 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan >Priority: Major > Fix For: SystemML 1.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SYSTEMML-2068) Add support for logical and bitwise logical operations
[
https://issues.apache.org/jira/browse/SYSTEMML-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reassigned SYSTEMML-2068:
---
Assignee: Janardhan
> Add support for logical and bitwise logical operations
> --
>
> Key: SYSTEMML-2068
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2068
> Project: SystemML
> Issue Type: Sub-task
>Reporter: Matthias Boehm
>Assignee: Janardhan
>
> This task aims to add code generation support for the recently introduced
> XOR, BW_AND, BW_OR, BW_XOR, BW_LSHIFT, and BW_RSHIFT operators. In detail,
> this entails:
> 1) Extend {{CNodeBinary.BinType}} with the new operator codes for
> vector-vector, vector-scalar, scalar-scalar operations, as well as related
> modifications in all places where these operator codes are handled.
> 2) Extend {{CPlanVectorPrimitivesTest}} with tests for all new vector
> primitives.
> 3) Add new tests to {{RowAggTmplTest}} and {{CellwiseTmplTest}} that uses
> these operations.
> 4) Implement all necessary operations in {{LibSpoofPrimitives}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (SYSTEMML-2057) Support bitwise operators not, and, or, xor, & LShift, Rshift
[ https://issues.apache.org/jira/browse/SYSTEMML-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323597#comment-16323597 ] Janardhan commented on SYSTEMML-2057: - bitwAnd, bitwOr, bitwXor, bitwShiftL, & bitwShiftR are addressed by https://github.com/apache/systemml/commit/7dbbaaa76f79061d8918cca389a51ac955001ffa > Support bitwise operators not, and, or, xor, & LShift, Rshift > - > > Key: SYSTEMML-2057 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2057 > Project: SystemML > Issue Type: Task > Components: Parser, Runtime >Reporter: Janardhan >Assignee: Janardhan > > 1. bitwNot(a) > 2. bitwAnd(a, b) > 3. bitwOr(a, b) > 4. bitwXor(a, b) > 5. bitwShiftL(a, n) > 6. bitwShiftR(a, n) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1996) Implementation of Bayesian module
[ https://issues.apache.org/jira/browse/SYSTEMML-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-1996: --- Assignee: Janardhan > Implementation of Bayesian module > - > > Key: SYSTEMML-1996 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1996 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan > Fix For: SystemML 1.1 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (SYSTEMML-1931) Support logical operators AND, OR, XOR, NOT over matrices
[ https://issues.apache.org/jira/browse/SYSTEMML-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan closed SYSTEMML-1931. --- Resolution: Implemented Fix Version/s: SystemML 1.1 > Support logical operators AND, OR, XOR, NOT over matrices > - > > Key: SYSTEMML-1931 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1931 > Project: SystemML > Issue Type: Bug >Reporter: Matthias Boehm >Assignee: Matthias Boehm > Fix For: SystemML 1.1 > > > So far, all logical operators (AND, OR, XOR, NOT) are only supported over > scalars. This task aims to add support for logical operators over matrices. > In detail, this entails: > * New test cases for all logical operators over dense and sparse matrices and > CP, MR, and SPARK execution types. > * Parser/compiler integration: (a) validation of boolean expressions to check > the validity of inputs (scalar or matrices), (b) propagation of data type and > value types (double if at least one matrix input), (b) sparsity estimates for > the individual operators. > * Runtime integration: Extend the function objects of AND, OR, XOR, NOT to > combinations of double, and boolean inputs. > * Fix (extend) opcode checks of individual instructions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1931) Support logical operators AND, OR, XOR, NOT over matrices
[ https://issues.apache.org/jira/browse/SYSTEMML-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311226#comment-16311226 ] Janardhan commented on SYSTEMML-1931: - Resolved by [~mboehm7]. & commit address: https://github.com/apache/systemml/commit/ea77cb456008c99276ebbae021ea8fe1246338ea > Support logical operators AND, OR, XOR, NOT over matrices > - > > Key: SYSTEMML-1931 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1931 > Project: SystemML > Issue Type: Bug >Reporter: Matthias Boehm >Assignee: Matthias Boehm > > So far, all logical operators (AND, OR, XOR, NOT) are only supported over > scalars. This task aims to add support for logical operators over matrices. > In detail, this entails: > * New test cases for all logical operators over dense and sparse matrices and > CP, MR, and SPARK execution types. > * Parser/compiler integration: (a) validation of boolean expressions to check > the validity of inputs (scalar or matrices), (b) propagation of data type and > value types (double if at least one matrix input), (b) sparsity estimates for > the individual operators. > * Runtime integration: Extend the function objects of AND, OR, XOR, NOT to > combinations of double, and boolean inputs. > * Fix (extend) opcode checks of individual instructions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1931) Support logical operators AND, OR, XOR, NOT over matrices
[ https://issues.apache.org/jira/browse/SYSTEMML-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-1931: --- Assignee: Matthias Boehm (was: Janardhan) > Support logical operators AND, OR, XOR, NOT over matrices > - > > Key: SYSTEMML-1931 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1931 > Project: SystemML > Issue Type: Bug >Reporter: Matthias Boehm >Assignee: Matthias Boehm > > So far, all logical operators (AND, OR, XOR, NOT) are only supported over > scalars. This task aims to add support for logical operators over matrices. > In detail, this entails: > * New test cases for all logical operators over dense and sparse matrices and > CP, MR, and SPARK execution types. > * Parser/compiler integration: (a) validation of boolean expressions to check > the validity of inputs (scalar or matrices), (b) propagation of data type and > value types (double if at least one matrix input), (b) sparsity estimates for > the individual operators. > * Runtime integration: Extend the function objects of AND, OR, XOR, NOT to > combinations of double, and boolean inputs. > * Fix (extend) opcode checks of individual instructions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-2057) Support bitwise operators not, and, or, xor, & LShift, Rshift
[ https://issues.apache.org/jira/browse/SYSTEMML-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-2057: --- Assignee: Janardhan > Support bitwise operators not, and, or, xor, & LShift, Rshift > - > > Key: SYSTEMML-2057 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2057 > Project: SystemML > Issue Type: Task > Components: Parser, Runtime >Reporter: Janardhan >Assignee: Janardhan > > 1. bitwNot(a) > 2. bitwAnd(a, b) > 3. bitwOr(a, b) > 4. bitwXor(a, b) > 5. bitwShiftL(a, n) > 6. bitwShiftR(a, n) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-2057) Support bitwise operators not, and, or, xor, & LShift, Rshift
Janardhan created SYSTEMML-2057: --- Summary: Support bitwise operators not, and, or, xor, & LShift, Rshift Key: SYSTEMML-2057 URL: https://issues.apache.org/jira/browse/SYSTEMML-2057 Project: SystemML Issue Type: Task Components: Parser, Runtime Reporter: Janardhan 1. bitwNot(a) 2. bitwAnd(a, b) 3. bitwOr(a, b) 4. bitwXor(a, b) 5. bitwShiftL(a, n) 6. bitwShiftR(a, n) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-2004) Implement covariance kernels
[ https://issues.apache.org/jira/browse/SYSTEMML-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-2004: --- Assignee: Janardhan > Implement covariance kernels > > > Key: SYSTEMML-2004 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2004 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan > Fix For: SystemML 1.1 > > > 1. square kernel > 2. 5/2 matern kernel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1995) Implementation of Surrogate data slice sampling
[ https://issues.apache.org/jira/browse/SYSTEMML-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-1995: --- Assignee: Janardhan > Implementation of Surrogate data slice sampling > --- > > Key: SYSTEMML-1995 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1995 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan > Fix For: SystemML 1.1 > > > The sampling of hyperparameters by surrogate data slice method. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1992) Implemenation of Mode finding for Gaussian Process
[ https://issues.apache.org/jira/browse/SYSTEMML-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-1992: --- Assignee: Janardhan > Implemenation of Mode finding for Gaussian Process > -- > > Key: SYSTEMML-1992 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1992 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan > Fix For: SystemML 1.1 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-2041) Implement Block-Sparse GPU Kernels
[ https://issues.apache.org/jira/browse/SYSTEMML-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-2041: Description: Sparsity enables, for example, training of neural networks that are much wider and deeper than otherwise possible with a given parameter budget and computational budget, such as LSTMs with tens of thousands of hidden units. (The largest LSTMs trained today are only thousands of hidden units.) *Resource:* TensorFlow implemented repo - https://github.com/openai/blocksparse *Best Supported architectures:* Maxwell, Pascal with Kepler & Volta support for limited functionality was: Sparsity enables, for example, training of neural networks that are much wider and deeper than otherwise possible with a given parameter budget and computational budget, such as LSTMs with tens of thousands of hidden units. (The largest LSTMs trained today are only thousands of hidden units.) *Resource:* TensorFlow implemented repo - https://github.com/openai/blocksparse > Implement Block-Sparse GPU Kernels > -- > > Key: SYSTEMML-2041 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2041 > Project: SystemML > Issue Type: New Feature > Components: Infrastructure >Reporter: Janardhan > Attachments: GPU Kernels for Block-Sparse Weights.pdf > > > Sparsity enables, for example, training of neural networks that are much > wider and deeper than otherwise possible with a given parameter budget and > computational budget, such as LSTMs with tens of thousands of hidden units. > (The largest LSTMs trained today are only thousands of hidden units.) > *Resource:* TensorFlow implemented repo - > https://github.com/openai/blocksparse > *Best Supported architectures:* Maxwell, Pascal with Kepler & Volta support > for limited functionality -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-2018) Fixing Weight Decay Regularization in ADAM
[ https://issues.apache.org/jira/browse/SYSTEMML-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254926#comment-16254926 ] Janardhan commented on SYSTEMML-2018: - cc [~dusenberrymw] > Fixing Weight Decay Regularization in ADAM > -- > > Key: SYSTEMML-2018 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2018 > Project: SystemML > Issue Type: Improvement > Components: Algorithms >Reporter: Janardhan > > The common implementations of adaptive gradient algorithms, such > as Adam, limit the potential benefit of weight decay regularization, because > the > weights do not decay multiplicatively (as would be expected for standard > weight > decay) but by an additive constant factor. > This following paper found a way to fix regularization in Adam Optimization > with one addition step(+ wx) to the gradient step : > https://arxiv.org/pdf/1711.05101.pdf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-2018) Fixing Weight Decay Regularization in ADAM
[ https://issues.apache.org/jira/browse/SYSTEMML-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-2018: Summary: Fixing Weight Decay Regularization in ADAM (was: FIXING WEIGHT DECAY REGULARIZATION IN ADAM) > Fixing Weight Decay Regularization in ADAM > -- > > Key: SYSTEMML-2018 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2018 > Project: SystemML > Issue Type: Improvement > Components: Algorithms >Reporter: Janardhan > > The common implementations of adaptive gradient algorithms, such > as Adam, limit the potential benefit of weight decay regularization, because > the > weights do not decay multiplicatively (as would be expected for standard > weight > decay) but by an additive constant factor. > This following paper found a way to fix regularization in Adam Optimization > with one addition step(+ wx) to the gradient step : > https://arxiv.org/pdf/1711.05101.pdf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-2018) FIXING WEIGHT DECAY REGULARIZATION IN ADAM
Janardhan created SYSTEMML-2018: --- Summary: FIXING WEIGHT DECAY REGULARIZATION IN ADAM Key: SYSTEMML-2018 URL: https://issues.apache.org/jira/browse/SYSTEMML-2018 Project: SystemML Issue Type: Improvement Components: Algorithms Reporter: Janardhan The common implementations of adaptive gradient algorithms, such as Adam, limit the potential benefit of weight decay regularization, because the weights do not decay multiplicatively (as would be expected for standard weight decay) but by an additive constant factor. This following paper found a way to fix regularization in Adam Optimization with one addition step(+ wx) to the gradient step : https://arxiv.org/pdf/1711.05101.pdf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240504#comment-16240504 ] Janardhan commented on SYSTEMML-1973: - Hi everyone, I am working on the hyperparameter optimization. This is the epic jira for the feature, with tasks and corresponding subtasks. a copy to : [~freiss] [~reinwald] [~dusenberrymw] [~prithvi_r_s] > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > *Phase 3:* Addition of Optimization test functions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-2006) Add gradient matching functionality
[ https://issues.apache.org/jira/browse/SYSTEMML-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-2006: Issue Type: New Feature (was: Bug) > Add gradient matching functionality > --- > > Key: SYSTEMML-2006 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2006 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Janardhan > > As described in this paper : https://arxiv.org/pdf/1706.04859.pdf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-2006) Add gradient matching functionality
Janardhan created SYSTEMML-2006: --- Summary: Add gradient matching functionality Key: SYSTEMML-2006 URL: https://issues.apache.org/jira/browse/SYSTEMML-2006 Project: SystemML Issue Type: Bug Components: Algorithms Reporter: Janardhan As described in this paper : https://arxiv.org/pdf/1706.04859.pdf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-2003) Addition of parallel monte carlo acquisition function
Janardhan created SYSTEMML-2003: --- Summary: Addition of parallel monte carlo acquisition function Key: SYSTEMML-2003 URL: https://issues.apache.org/jira/browse/SYSTEMML-2003 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240397#comment-16240397 ] Janardhan commented on SYSTEMML-1973: - This is the central jira for any or all of the feature related. cc: [~niketanpansare] > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > *Phase 3:* Addition of Optimization test functions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1973: Description: This Epic tracks the algorithm optimization related improvements, and their testing. *Phase 1:* Addition of support for bayesian optimization. *Phase 2:* Addition of Model selection & cross validation support at Engine level or API side. *Phase 3:* Addition of Optimization test functions. was:This Epic tracks the algorithm optimization related improvements, and their testing. > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan > > This Epic tracks the algorithm optimization related improvements, and their > testing. > *Phase 1:* Addition of support for bayesian optimization. > *Phase 2:* Addition of Model selection & cross validation support at Engine > level or API side. > *Phase 3:* Addition of Optimization test functions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1973) Optimization of parameters, Hyperparameters, and testing.
[ https://issues.apache.org/jira/browse/SYSTEMML-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1973: Summary: Optimization of parameters, Hyperparameters, and testing. (was: Optimization of parameters & Hyperparameters, and testing.) > Optimization of parameters, Hyperparameters, and testing. > - > > Key: SYSTEMML-1973 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 > Project: SystemML > Issue Type: Epic > Components: Algorithms, APIs, Documentation, Test >Reporter: Janardhan >Assignee: Janardhan > > This Epic tracks the algorithm optimization related improvements, and their > testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1974) Implementation of Optimization test functions
[ https://issues.apache.org/jira/browse/SYSTEMML-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1974: Description: Functions such as Buckin, Branin Hoo, etc., useful for testing of the implemented optimization algorithms in tough scenarios such as multiple global minima, etc., *Why these functions?* These functions help us understand the gradient behaviour, on different terrains. Continuity and differentiability effects can also be tested. was:Functions such as Buckin, Branin Hoo, etc., useful for testing of the implemented optimization algorithms in tough scenarios such as multiple global minima, etc., > Implementation of Optimization test functions > - > > Key: SYSTEMML-1974 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1974 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Janardhan >Assignee: Janardhan > > Functions such as Buckin, Branin Hoo, etc., useful for testing of the > implemented optimization algorithms in tough scenarios such as multiple > global minima, etc., > *Why these functions?* > These functions help us understand the gradient behaviour, on different > terrains. Continuity and differentiability effects can also be tested. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-2002) some special functions implementation
Janardhan created SYSTEMML-2002: --- Summary: some special functions implementation Key: SYSTEMML-2002 URL: https://issues.apache.org/jira/browse/SYSTEMML-2002 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-2000) Valley shaped functions implementation
Janardhan created SYSTEMML-2000: --- Summary: Valley shaped functions implementation Key: SYSTEMML-2000 URL: https://issues.apache.org/jira/browse/SYSTEMML-2000 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-2001) Functions for steep ridges or drops, implementation
Janardhan created SYSTEMML-2001: --- Summary: Functions for steep ridges or drops, implementation Key: SYSTEMML-2001 URL: https://issues.apache.org/jira/browse/SYSTEMML-2001 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1999) Plate shaped functions implementation
Janardhan created SYSTEMML-1999: --- Summary: Plate shaped functions implementation Key: SYSTEMML-1999 URL: https://issues.apache.org/jira/browse/SYSTEMML-1999 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1998) Bowl shaped functions implementation
Janardhan created SYSTEMML-1998: --- Summary: Bowl shaped functions implementation Key: SYSTEMML-1998 URL: https://issues.apache.org/jira/browse/SYSTEMML-1998 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1997) local minima functions
Janardhan created SYSTEMML-1997: --- Summary: local minima functions Key: SYSTEMML-1997 URL: https://issues.apache.org/jira/browse/SYSTEMML-1997 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan The local minima functions -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1996) Implementation of Bayesian module
Janardhan created SYSTEMML-1996: --- Summary: Implementation of Bayesian module Key: SYSTEMML-1996 URL: https://issues.apache.org/jira/browse/SYSTEMML-1996 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1995) Implementation of Surrogate data slice sampling
Janardhan created SYSTEMML-1995: --- Summary: Implementation of Surrogate data slice sampling Key: SYSTEMML-1995 URL: https://issues.apache.org/jira/browse/SYSTEMML-1995 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan The sampling of hyperparameters by surrogate data slice method. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (SYSTEMML-979) Add support for bayesian optimization
[ https://issues.apache.org/jira/browse/SYSTEMML-979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-979: --- Comment: was deleted (was: Hi,:) It will be great, if the bayesian optimization is implemented. Please provide some thoughts on how to best proceed on this. also a copy to: [~dusenberrymw] [~prithvi_r_s] [~freiss] [~mboehm7] [~deron] and sorry if I missed out on some one.) > Add support for bayesian optimization > - > > Key: SYSTEMML-979 > URL: https://issues.apache.org/jira/browse/SYSTEMML-979 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Niketan Pansare >Assignee: Janardhan > Labels: optimization > Fix For: SystemML 1.0 > > Original Estimate: 336h > Time Spent: 702h > Remaining Estimate: 0h > > Main paper: > https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf > supplements: > * selecting the next point to evaluate > [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] > * sobol sequence generator [P Bratley, BL > Fox|http://dl.acm.org/citation.cfm?id=214372] > * Handling Sparsity via the Horseshoe [Carlos M. > Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] > *Design document:* http://www.bit.do/systemml-bayesian -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1993) Implementation of Gaussian Process Classification
Janardhan created SYSTEMML-1993: --- Summary: Implementation of Gaussian Process Classification Key: SYSTEMML-1993 URL: https://issues.apache.org/jira/browse/SYSTEMML-1993 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan The classification script with Gaussian process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1992) Implemenation of Mode finding for Gaussian Process
Janardhan created SYSTEMML-1992: --- Summary: Implemenation of Mode finding for Gaussian Process Key: SYSTEMML-1992 URL: https://issues.apache.org/jira/browse/SYSTEMML-1992 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1991) Implementation of Sobol quasi random sequence generator
Janardhan created SYSTEMML-1991: --- Summary: Implementation of Sobol quasi random sequence generator Key: SYSTEMML-1991 URL: https://issues.apache.org/jira/browse/SYSTEMML-1991 Project: SystemML Issue Type: Sub-task Components: Algorithms Reporter: Janardhan Assignee: Janardhan The quasi random sobol sequence generator, samples points uniformly from a unit hypercube or unit sphere. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-979) Add support for bayesian optimization
[ https://issues.apache.org/jira/browse/SYSTEMML-979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-979: --- Description: Main paper: https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf supplements: * selecting the next point to evaluate [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] * sobol sequence generator [P Bratley, BL Fox|http://dl.acm.org/citation.cfm?id=214372] * Handling Sparsity via the Horseshoe [Carlos M. Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] *Design document:* http://www.bit.do/systemml-bayesian was: Main paper: https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf supplements: * selecting the next point to evaluate [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] * sobol sequence generator [P Bratley, BL Fox|http://dl.acm.org/citation.cfm?id=214372] * Handling Sparsity via the Horseshoe [Carlos M. Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] *Design document:* http://www.bit.do/systemml-bayesian > Add support for bayesian optimization > - > > Key: SYSTEMML-979 > URL: https://issues.apache.org/jira/browse/SYSTEMML-979 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Niketan Pansare >Assignee: Janardhan > Labels: optimization > Fix For: SystemML 1.0 > > Original Estimate: 336h > Time Spent: 702h > Remaining Estimate: 0h > > Main paper: > https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf > supplements: > * selecting the next point to evaluate > [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] > * sobol sequence generator [P Bratley, BL > Fox|http://dl.acm.org/citation.cfm?id=214372] > * Handling Sparsity via the Horseshoe [Carlos M. > Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] > *Design document:* http://www.bit.do/systemml-bayesian -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1973) Optimization of parameters & Hyperparameters, and testing.
Janardhan created SYSTEMML-1973: --- Summary: Optimization of parameters & Hyperparameters, and testing. Key: SYSTEMML-1973 URL: https://issues.apache.org/jira/browse/SYSTEMML-1973 Project: SystemML Issue Type: Epic Components: Algorithms, APIs, Documentation, Test Reporter: Janardhan Assignee: Janardhan This Epic tracks the algorithm optimization related improvements, and their testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SYSTEMML-1648) Verify whether SVM scripts work with MLContext
[ https://issues.apache.org/jira/browse/SYSTEMML-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan resolved SYSTEMML-1648. - Resolution: Resolved Fix Version/s: SystemML 1.0 > Verify whether SVM scripts work with MLContext > -- > > Key: SYSTEMML-1648 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1648 > Project: SystemML > Issue Type: Improvement > Components: Algorithms >Reporter: Imran Younus >Assignee: Jerome > Fix For: SystemML 1.0 > > > This jira plans to verify whether existing SVM scripts work properly with new > MLContext. These scripts include l2-svm.dml, l2-svm-predict.dml, m-svm.dml, > m-svm-predict.dml. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1648) Verify whether SVM scripts work with MLContext
[ https://issues.apache.org/jira/browse/SYSTEMML-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217249#comment-16217249 ] Janardhan commented on SYSTEMML-1648: - Addressed by [PR 687|https://github.com/apache/systemml/pull/687] > Verify whether SVM scripts work with MLContext > -- > > Key: SYSTEMML-1648 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1648 > Project: SystemML > Issue Type: Improvement > Components: Algorithms >Reporter: Imran Younus >Assignee: Jerome > Fix For: SystemML 1.0 > > > This jira plans to verify whether existing SVM scripts work properly with new > MLContext. These scripts include l2-svm.dml, l2-svm-predict.dml, m-svm.dml, > m-svm-predict.dml. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1962) Support model-selection via mllearn APIs
[
https://issues.apache.org/jira/browse/SYSTEMML-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reassigned SYSTEMML-1962:
---
Assignee: Janardhan
> Support model-selection via mllearn APIs
>
>
> Key: SYSTEMML-1962
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1962
> Project: SystemML
> Issue Type: New Feature
>Reporter: Niketan Pansare
>Assignee: Janardhan
>
> The end goal of this JIRA is to support model selection facility similar to
> [http://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection].
> Currently, we support model selection using MLPipeline's cross-validator. For
> example: please replace `from pyspark.ml.classification import
> LogisticRegression` with `from systemml.mllearn import LogisticRegression` in
> the example
> http://spark.apache.org/docs/2.1.1/ml-tuning.html#example-model-selection-via-cross-validation.
>
> However, this invokes k-seperate and independent mlcontext calls. This PR
> proposes to add a new class `GridSearchCV`, `RandomizedSearchCV` and possibly
> bayesian optimization which like mllearn has methods `fit` and `predict`.
> These methods internally generate a script that wraps the external script
> with a `parfor` when the fit method is called. For example:
> {code}
> from sklearn import datasets
> from systemml.mllearn import GridSearchCV, SVM
> iris = datasets.load_iris()
> parameters = {'C':[1, 10]}
> svm = SVM()
> clf = GridSearchClassifierCV(svm, parameters)
> clf.fit(iris.data, iris.target)
> {code}
> would execute the script:
> {code}
> CVals = matrix("1; 10", rows=2, cols=1)
> parfor(i in seq(1, nrow(CVals))) {
>C = CVals[i, 1]
>reg = 1 / C
> # SVM script
> }
> {code}
> This will require:
> 1. Functionization of the script (for example: L2SVM)
> {code}
> svm = function(matrix[double] X, matrix[double] Y, double icpt, double tol,
> double reg, double maxiter) returns (matrix[double] w) {
>if(nrow(X) < 2)
> stop("Stopping due to invalid inputs: Not possible to learn a binary
> class classifier without at least 2 rows")
>check_min = min(Y)
>
>w = t(cbind(t(w), t(extra_model_params)))
> }
> {code}
> 2. Adding two new java classes in the package `org.apache.sysml.api.ml`
> called `GridSearchClassifierCV` which extends
> `Estimator[GridSearchClassifierCVModel]` and `GridSearchClassifierCVModel`
> which `extends Model[GridSearchClassifierCVModel] with
> BaseSystemMLClassifierModel`. Then you will have to implement the abstract
> methods: fit and transform respectively.
> 3. Add a python class GridSearchClassifierCV that invokes the above java
> classes.
> For more details on step 2 and step 3, please read the design documentation
> of mllearn API:
> https://github.com/apache/systemml/blob/master/src/main/scala/org/apache/sysml/api/ml/BaseSystemMLClassifier.scala#L42
> [~dusenberrymw] may be, this can be part of
> https://issues.apache.org/jira/browse/SYSTEMML-1159
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (SYSTEMML-822) Gradient Boosted Trees
[ https://issues.apache.org/jira/browse/SYSTEMML-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-822: --- Description: It would be great to have an implementation of gradient boosted trees in SystemML, similar to scikit-learn's gradient boosting machine [1] or DMLC's XGBoost [2]. [1] http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html [2] https://github.com/dmlc/xgboost/ [3] http://homes.cs.washington.edu/~tqchen/2016/03/10/story-and-lessons-behind-the-evolution-of-xgboost.html For some inspiration, implementation for MLlib - https://github.com/apache/spark/pull/2607/files was: It would be great to have an implementation of gradient boosted trees in SystemML, similar to scikit-learn's gradient boosting machine [1] or DMLC's XGBoost [2]. [1] http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html [2] https://github.com/dmlc/xgboost/ [3] http://homes.cs.washington.edu/~tqchen/2016/03/10/story-and-lessons-behind-the-evolution-of-xgboost.html > Gradient Boosted Trees > -- > > Key: SYSTEMML-822 > URL: https://issues.apache.org/jira/browse/SYSTEMML-822 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Affects Versions: SystemML 0.11 >Reporter: Abhinav Maurya > Labels: Hacktoberfest, features > > It would be great to have an implementation of gradient boosted trees in > SystemML, similar to scikit-learn's gradient boosting machine [1] or DMLC's > XGBoost [2]. > [1] > http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html > [2] https://github.com/dmlc/xgboost/ > [3] > http://homes.cs.washington.edu/~tqchen/2016/03/10/story-and-lessons-behind-the-evolution-of-xgboost.html > For some inspiration, implementation for MLlib - > https://github.com/apache/spark/pull/2607/files -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1938) Regularized Greedy Forest (RGF) Implementation
[ https://issues.apache.org/jira/browse/SYSTEMML-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180790#comment-16180790 ] Janardhan commented on SYSTEMML-1938: - This ( if needed) can be implemented after adding the gradient boost support. > Regularized Greedy Forest (RGF) Implementation > -- > > Key: SYSTEMML-1938 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1938 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Janardhan >Priority: Minor > > RGF is a machine learning method for building decision forests. Based on > 1. Learning Nonlinear Functions UsingRegularized Greedy Forest - > https://arxiv.org/pdf/1109.0887.pdf > 2. Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost - > https://arxiv.org/ftp/arxiv/papers/1203/1203.3491.pdf > 3. A general boosting method and its application to learning ranking > functions for web search - > https://papers.nips.cc/paper/3305-a-general-boosting-method-and-its-application-to-learning-ranking-functions-for-web-search.pdf > A C++ implementation is at https://github.com/baidu/fast_rgf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1938) Regularized Greedy Forest (RGF) Implementation
[ https://issues.apache.org/jira/browse/SYSTEMML-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1938: Description: RGF is a machine learning method for building decision forests. Based on 1. Learning Nonlinear Functions UsingRegularized Greedy Forest - https://arxiv.org/pdf/1109.0887.pdf 2. Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost - https://arxiv.org/ftp/arxiv/papers/1203/1203.3491.pdf 3. A general boosting method and its application to learning ranking functions for web search - https://papers.nips.cc/paper/3305-a-general-boosting-method-and-its-application-to-learning-ranking-functions-for-web-search.pdf A C++ implementation is at https://github.com/baidu/fast_rgf was: RGF is a machine learning method for building decision forests. Based on Learning Nonlinear Functions Using Regularized Greedy Forest https://arxiv.org/pdf/1109.0887.pdf A C++ implementation is at https://github.com/baidu/fast_rgf > Regularized Greedy Forest (RGF) Implementation > -- > > Key: SYSTEMML-1938 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1938 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Janardhan >Priority: Minor > > RGF is a machine learning method for building decision forests. Based on > 1. Learning Nonlinear Functions UsingRegularized Greedy Forest - > https://arxiv.org/pdf/1109.0887.pdf > 2. Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost - > https://arxiv.org/ftp/arxiv/papers/1203/1203.3491.pdf > 3. A general boosting method and its application to learning ranking > functions for web search - > https://papers.nips.cc/paper/3305-a-general-boosting-method-and-its-application-to-learning-ranking-functions-for-web-search.pdf > A C++ implementation is at https://github.com/baidu/fast_rgf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1938) Regularized Greedy Forest (RGF) Implementation
Janardhan created SYSTEMML-1938: --- Summary: Regularized Greedy Forest (RGF) Implementation Key: SYSTEMML-1938 URL: https://issues.apache.org/jira/browse/SYSTEMML-1938 Project: SystemML Issue Type: New Feature Components: Algorithms Reporter: Janardhan Priority: Minor RGF is a machine learning method for building decision forests. A C++ implementation is at https://github.com/baidu/fast_rgf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1938) Regularized Greedy Forest (RGF) Implementation
[ https://issues.apache.org/jira/browse/SYSTEMML-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-1938: Description: RGF is a machine learning method for building decision forests. Based on Learning Nonlinear Functions Using Regularized Greedy Forest https://arxiv.org/pdf/1109.0887.pdf A C++ implementation is at https://github.com/baidu/fast_rgf was: RGF is a machine learning method for building decision forests. A C++ implementation is at https://github.com/baidu/fast_rgf > Regularized Greedy Forest (RGF) Implementation > -- > > Key: SYSTEMML-1938 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1938 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Janardhan >Priority: Minor > > RGF is a machine learning method for building decision forests. Based on > Learning Nonlinear Functions Using > Regularized Greedy Forest https://arxiv.org/pdf/1109.0887.pdf > A C++ implementation is at https://github.com/baidu/fast_rgf -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SYSTEMML-1937) Vector Free L-BFGS implementation
Janardhan created SYSTEMML-1937: --- Summary: Vector Free L-BFGS implementation Key: SYSTEMML-1937 URL: https://issues.apache.org/jira/browse/SYSTEMML-1937 Project: SystemML Issue Type: New Feature Components: Algorithms, ParFor Reporter: Janardhan This is for the implementation of vector free L-BFGS, as in the paper http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf , to avoid the parameter server. Example implementation for spark-ml lib : @ https://github.com/yanboliang/spark-vlbfgs -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1649) Verify whether GLM scripts work with MLContext
[
https://issues.apache.org/jira/browse/SYSTEMML-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179423#comment-16179423
]
Janardhan commented on SYSTEMML-1649:
-
Hi [~nilmeier], I have added a test class for {{GLM.dml}} and
{{GML-predict.dml}}. I believe, I took into all the individual cases by setting
up combinations of arguments, can you have look at that?. Thanks.
> Verify whether GLM scripts work with MLContext
> --
>
> Key: SYSTEMML-1649
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1649
> Project: SystemML
> Issue Type: Improvement
> Components: Algorithms
>Reporter: Imran Younus
>Assignee: Janardhan
>
> This jira will verify whether GLM scripts work properly with new MLContext.
> These scripts include GLM.dml and GLM-predict.dml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1865) Add MLContext test class for SVM scripts
[
https://issues.apache.org/jira/browse/SYSTEMML-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reassigned SYSTEMML-1865:
---
Assignee: Janardhan
> Add MLContext test class for SVM scripts
>
>
> Key: SYSTEMML-1865
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1865
> Project: SystemML
> Issue Type: Sub-task
> Components: Algorithms, Test
>Reporter: Janardhan
>Assignee: Janardhan
>
> This jira plans to add a test class for all the operations and functions in
> {{l2-svm-predict.dml}}, {{l2-svm.dml}}, {{m-svm-predict.dml}} and
> {{m-svm.dml}} scripts.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (SYSTEMML-888) Add PNMF algorithm to SystemML
[
https://issues.apache.org/jira/browse/SYSTEMML-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178154#comment-16178154
]
Janardhan edited comment on SYSTEMML-888 at 9/24/17 11:04 AM:
--
Reopening this issue to address
[convergence|https://github.com/apache/systemml/blob/47ce14fc632303ef38675e1c9b274b93cafbeee2/scripts/staging/PNMF.dml#L32]
needs with the ALS, & Eucledian distance minimization. And addition of some
little features..
I believe this can be modularized, so that a general {{NMF}} script can be
possible, with {{optim}} layers. I have gone through the literature when I was
first assigned for this issue. I'll give a patch over this utilizing the
{{l2_loss.dml}}. Thanks.
was (Author: return_01):
Reopening this issue to address
[convergence|https://github.com/apache/systemml/blob/47ce14fc632303ef38675e1c9b274b93cafbeee2/scripts/staging/PNMF.dml#L32]
needs with the ALS, & Eucledian distance minimization. And addition of some
little features..
I believe this can be modularized, so that a general {{NMF}} script can be
possible, with {{optim}} layers. I have gone through the literature when I was
first assigned for this issue. I'll give a patch over this utilizing the
{{l2_loss.dml}}.
> Add PNMF algorithm to SystemML
> --
>
> Key: SYSTEMML-888
> URL: https://issues.apache.org/jira/browse/SYSTEMML-888
> Project: SystemML
> Issue Type: Task
> Components: Algorithms
>Reporter: Deron Eriksson
>Assignee: Matthias Boehm
> Fix For: SystemML 0.15
>
>
> Add the Poisson Nonnegative Matrix Factorization algorithm to the SystemML
> algorithms.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (SYSTEMML-888) Add PNMF algorithm to SystemML
[
https://issues.apache.org/jira/browse/SYSTEMML-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reopened SYSTEMML-888:
Reopening this issue to address
[convergence|https://github.com/apache/systemml/blob/47ce14fc632303ef38675e1c9b274b93cafbeee2/scripts/staging/PNMF.dml#L32]
needs with the ALS, & Eucledian distance minimization. And addition of some
little features..
I believe this can be modularized, so that a general {{NMF}} script can be
possible, with {{optim}} layers. I have gone through the literature when I was
first assigned for this issue. I'll give a patch over this utilizing the
{{l2_loss.dml}}.
> Add PNMF algorithm to SystemML
> --
>
> Key: SYSTEMML-888
> URL: https://issues.apache.org/jira/browse/SYSTEMML-888
> Project: SystemML
> Issue Type: Task
> Components: Algorithms
>Reporter: Deron Eriksson
>Assignee: Matthias Boehm
> Fix For: SystemML 0.15
>
>
> Add the Poisson Nonnegative Matrix Factorization algorithm to the SystemML
> algorithms.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1649) Verify whether GLM scripts work with MLContext
[
https://issues.apache.org/jira/browse/SYSTEMML-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158494#comment-16158494
]
Janardhan commented on SYSTEMML-1649:
-
Hi Jerome, the unit test for {{GLM.dml}} is passing now, and is resolving input
matrix issues for {{GLM-predict.dml}}. But, can you check whether the random
matrices against which I am checking is sufficient?. Thanks.
> Verify whether GLM scripts work with MLContext
> --
>
> Key: SYSTEMML-1649
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1649
> Project: SystemML
> Issue Type: Improvement
> Components: Algorithms
>Reporter: Imran Younus
>Assignee: Janardhan
>
> This jira will verify whether GLM scripts work properly with new MLContext.
> These scripts include GLM.dml and GLM-predict.dml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1883) Add XOR built in operator
[
https://issues.apache.org/jira/browse/SYSTEMML-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-1883:
Description: (Based on discussion in list) Let us try a java-based UDF to
efficiently implement this recursive {{XOR}} operator, although we can
implement this with existing {{AND}} and {{NOR}}. Also, we might want to
consider both GPU enablement. (was: (Based on discussion in list) Try a
java-based UDF to efficiently implement this recursive {{XOR}} operator. )
> Add XOR built in operator
> -
>
> Key: SYSTEMML-1883
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1883
> Project: SystemML
> Issue Type: Sub-task
> Components: Algorithms, Compiler
>Affects Versions: SystemML 1.0
>Reporter: Janardhan
>Priority: Minor
> Fix For: SystemML 1.0
>
>
> (Based on discussion in list) Let us try a java-based UDF to efficiently
> implement this recursive {{XOR}} operator, although we can implement this
> with existing {{AND}} and {{NOR}}. Also, we might want to consider both GPU
> enablement.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1883) Add XOR built in operator
[
https://issues.apache.org/jira/browse/SYSTEMML-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151472#comment-16151472
]
Janardhan commented on SYSTEMML-1883:
-
cc [~nakul02] [~niketanpansare].
> Add XOR built in operator
> -
>
> Key: SYSTEMML-1883
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1883
> Project: SystemML
> Issue Type: Sub-task
> Components: Algorithms, Compiler
>Affects Versions: SystemML 1.0
>Reporter: Janardhan
>Priority: Minor
> Fix For: SystemML 1.0
>
>
> (Based on discussion in list) Try a java-based UDF to efficiently implement
> this recursive {{XOR}} operator.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (SYSTEMML-979) Add support for bayesian optimization
[ https://issues.apache.org/jira/browse/SYSTEMML-979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-979: --- Description: Main paper: https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf supplements: * selecting the next point to evaluate [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] * sobol sequence generator [P Bratley, BL Fox|http://dl.acm.org/citation.cfm?id=214372] * Handling Sparsity via the Horseshoe [Carlos M. Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] *Design document:* http://www.bit.do/systemml-bayesian was: Main paper: https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf supplements: * selecting the next point to evaluate [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] * sobol sequence generator [P Bratley, BL Fox|http://dl.acm.org/citation.cfm?id=214372] * Handling Sparsity via the Horseshoe [Carlos M. Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] Design document: http://www.bit.do/systemml-bayesian > Add support for bayesian optimization > - > > Key: SYSTEMML-979 > URL: https://issues.apache.org/jira/browse/SYSTEMML-979 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Niketan Pansare >Assignee: Janardhan > Labels: optimization > Fix For: SystemML 1.0 > > Original Estimate: 336h > Remaining Estimate: 336h > > Main paper: > https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf > supplements: > * selecting the next point to evaluate > [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] > * sobol sequence generator [P Bratley, BL > Fox|http://dl.acm.org/citation.cfm?id=214372] > * Handling Sparsity via the Horseshoe [Carlos M. > Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] > *Design document:* http://www.bit.do/systemml-bayesian -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-979) Add support for bayesian optimization
[ https://issues.apache.org/jira/browse/SYSTEMML-979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan updated SYSTEMML-979: --- Description: Main paper: https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf supplements: * selecting the next point to evaluate [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] * sobol sequence generator [P Bratley, BL Fox|http://dl.acm.org/citation.cfm?id=214372] * Handling Sparsity via the Horseshoe [Carlos M. Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] Design document: http://www.bit.do/systemml-bayesian was: Main paper: https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf supplements: * selecting the next point to evaluate [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] * sobol sequence generator [P Bratley, BL Fox|http://dl.acm.org/citation.cfm?id=214372] * Handling Sparsity via the Horseshoe [Carlos M. Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] > Add support for bayesian optimization > - > > Key: SYSTEMML-979 > URL: https://issues.apache.org/jira/browse/SYSTEMML-979 > Project: SystemML > Issue Type: New Feature > Components: Algorithms >Reporter: Niketan Pansare >Assignee: Janardhan > Labels: optimization > Fix For: SystemML 1.0 > > Original Estimate: 336h > Remaining Estimate: 336h > > Main paper: > https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf > supplements: > * selecting the next point to evaluate > [Jasper|http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips_appendix.pdf] > * sobol sequence generator [P Bratley, BL > Fox|http://dl.acm.org/citation.cfm?id=214372] > * Handling Sparsity via the Horseshoe [Carlos M. > Carvalho|http://ftp.isds.duke.edu/WorkingPapers/09-03.pdf] > Design document: http://www.bit.do/systemml-bayesian -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1883) Add XOR built in operator
[
https://issues.apache.org/jira/browse/SYSTEMML-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-1883:
Description: (Based on discussion in list) Try a java-based UDF to
efficiently implement this recursive {{XOR}} operator. (was: A java-based UDF
to efficiently implement this recursive {{XOR}} operator.)
> Add XOR built in operator
> -
>
> Key: SYSTEMML-1883
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1883
> Project: SystemML
> Issue Type: Sub-task
> Components: Algorithms, Compiler
>Affects Versions: SystemML 1.0
>Reporter: Janardhan
>Priority: Minor
> Fix For: SystemML 1.0
>
>
> (Based on discussion in list) Try a java-based UDF to efficiently implement
> this recursive {{XOR}} operator.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (SYSTEMML-1883) Add XOR built in operator
Janardhan created SYSTEMML-1883:
---
Summary: Add XOR built in operator
Key: SYSTEMML-1883
URL: https://issues.apache.org/jira/browse/SYSTEMML-1883
Project: SystemML
Issue Type: Sub-task
Components: Algorithms, Compiler
Affects Versions: SystemML 1.0
Reporter: Janardhan
Priority: Minor
A java-based UDF to efficiently implement this recursive {{XOR}} operator.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1868) Add MLContext test class for ALS scripts
[
https://issues.apache.org/jira/browse/SYSTEMML-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reassigned SYSTEMML-1868:
---
Assignee: Janardhan
> Add MLContext test class for ALS scripts
>
>
> Key: SYSTEMML-1868
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1868
> Project: SystemML
> Issue Type: Sub-task
> Components: Algorithms
>Reporter: Janardhan
>Assignee: Janardhan
> Fix For: SystemML 1.0
>
>
> This jira tries to incorporate test through MLContext for {{ALS-DS.dml}},
> {{ALS-CG.dml}} and {{ALS-predict.dml}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (SYSTEMML-1866) Add MLContext test class for GLM scripts
[
https://issues.apache.org/jira/browse/SYSTEMML-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan reassigned SYSTEMML-1866:
---
Assignee: Janardhan
> Add MLContext test class for GLM scripts
>
>
> Key: SYSTEMML-1866
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1866
> Project: SystemML
> Issue Type: Sub-task
> Components: Algorithms, Test
>Reporter: Janardhan
>Assignee: Janardhan
>
> This jira plans to add a test class for all the operations and functions in
> {{GLM.dml}} and {{GLM-predict.dml}} scripts.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (SYSTEMML-280) Scalable implementation of eigen()
[ https://issues.apache.org/jira/browse/SYSTEMML-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143061#comment-16143061 ] Janardhan commented on SYSTEMML-280: Hi, this paper might be of some help here [Stable and Efficient Spectral Divide and Conquer Algorithms for the Symmetric Eigenvalue Decomposition and the SVD|http://epubs.siam.org/doi/pdf/10.1137/120876605]. Thanks. :) > Scalable implementation of eigen() > -- > > Key: SYSTEMML-280 > URL: https://issues.apache.org/jira/browse/SYSTEMML-280 > Project: SystemML > Issue Type: New Feature >Reporter: Berthold Reinwald >Assignee: Frederick Reiss > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SYSTEMML-984) Add mllearn and scala wrappers for PCA
[
https://issues.apache.org/jira/browse/SYSTEMML-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143059#comment-16143059
]
Janardhan commented on SYSTEMML-984:
Recently, when I am trying to find the literature for implementing scalable
{{eigen}}, I came across this paper "Parallelization of Principal Component
Analysis (using Eigen Value Decomposition) on Scalable Multi-core
Architecture".
[~niketanpansare] can you please comment on whether we need scalar wrapper, if
we could find a situation where we can implement PCA scalably with scalable
{{eigen}}.
> Add mllearn and scala wrappers for PCA
> --
>
> Key: SYSTEMML-984
> URL: https://issues.apache.org/jira/browse/SYSTEMML-984
> Project: SystemML
> Issue Type: Task
> Components: APIs
>Reporter: Niketan Pansare
> Labels: Hacktoberfest, starter
>
> See
> https://apache.github.io/incubator-systemml/algorithms-matrix-factorization.html#principle-component-analysis
> for usage
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1216) implement local svd function
[
https://issues.apache.org/jira/browse/SYSTEMML-1216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142868#comment-16142868
]
Janardhan commented on SYSTEMML-1216:
-
CPU version is addressed by https://github.com/apache/systemml/pull/605 .
[~nakul02] , I am interested in GPU implementation with cusolver once, we find
that distributed svd() with local svd() is successful.
> implement local svd function
>
>
> Key: SYSTEMML-1216
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1216
> Project: SystemML
> Issue Type: New Feature
>Reporter: Imran Younus
>Assignee: Janardhan
> Attachments: svd.txt
>
>
> SystemML currently provides several local matrix decompositions (qr(), lu(),
> cholesky()). But local version of svd is missing. This is also needed to
> scalable SVD implementation.
> Also, implement local {{svd()}} function with {{cusolver}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (SYSTEMML-1216) implement local svd function
[
https://issues.apache.org/jira/browse/SYSTEMML-1216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan updated SYSTEMML-1216:
Description:
SystemML currently provides several local matrix decompositions (qr(), lu(),
cholesky()). But local version of svd is missing. This is also needed to
scalable SVD implementation.
Also, implement local {{svd()}} function with {{cusolver}}.
was:SystemML currently provides several local matrix decompositions (qr(),
lu(), cholesky()). But local version of svd is missing. This is also needed to
scalable SVD implementation.
> implement local svd function
>
>
> Key: SYSTEMML-1216
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1216
> Project: SystemML
> Issue Type: New Feature
>Reporter: Imran Younus
>Assignee: Janardhan
> Attachments: svd.txt
>
>
> SystemML currently provides several local matrix decompositions (qr(), lu(),
> cholesky()). But local version of svd is missing. This is also needed to
> scalable SVD implementation.
> Also, implement local {{svd()}} function with {{cusolver}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (SYSTEMML-1869) Add MLContext test class for Logistic regression algorithm
Janardhan created SYSTEMML-1869:
---
Summary: Add MLContext test class for Logistic regression algorithm
Key: SYSTEMML-1869
URL: https://issues.apache.org/jira/browse/SYSTEMML-1869
Project: SystemML
Issue Type: Sub-task
Reporter: Janardhan
This jira tries to add a test class set through MLContext for
{{MultiLogReg.dml}} script.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (SYSTEMML-1868) Add MLContext test class for ALS scripts
Janardhan created SYSTEMML-1868:
---
Summary: Add MLContext test class for ALS scripts
Key: SYSTEMML-1868
URL: https://issues.apache.org/jira/browse/SYSTEMML-1868
Project: SystemML
Issue Type: Sub-task
Reporter: Janardhan
This jira tries to incorporate test through MLContext for {{ALS-DS.dml}},
{{ALS-CG.dml}} and {{ALS-predict.dml}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (SYSTEMML-1867) Add MLContext test class for survival analysis algorithms
Janardhan created SYSTEMML-1867:
---
Summary: Add MLContext test class for survival analysis algorithms
Key: SYSTEMML-1867
URL: https://issues.apache.org/jira/browse/SYSTEMML-1867
Project: SystemML
Issue Type: Sub-task
Reporter: Janardhan
This jira tries to add tests with MLContext for survival analysis algorithms.
The algorithms are {{Cox.dml}}, {{Cox-predict.dml}} and {{KM.dml}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (SYSTEMML-1865) Add MLContext test class for SVM scripts
Janardhan created SYSTEMML-1865:
---
Summary: Add MLContext test class for SVM scripts
Key: SYSTEMML-1865
URL: https://issues.apache.org/jira/browse/SYSTEMML-1865
Project: SystemML
Issue Type: Sub-task
Components: Algorithms, Test
Reporter: Janardhan
This jira plans to add a test class for all the operations and functions in
{{l2-svm-predict.dml}}, {{l2-svm.dml}}, {{m-svm-predict.dml}} and {{m-svm.dml}}
scripts.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (SYSTEMML-1811) Can we Implement X%*%t(X) in a better way?
[ https://issues.apache.org/jira/browse/SYSTEMML-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121208#comment-16121208 ] Janardhan commented on SYSTEMML-1811: - Thanks Matthias Boehm for helping me understand these operations. I am not aware these operations when I created this jira. > Can we Implement X%*%t(X) in a better way? > -- > > Key: SYSTEMML-1811 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1811 > Project: SystemML > Issue Type: Improvement >Reporter: Janardhan >Assignee: Matthias Boehm > Fix For: Not Applicable > > > A matrix multiplied by its self transpose is a frequent occurrence in many > algorithms ( a lot of them). There is definitely a way to take into > consideration the special properties of this matrix operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
