[
https://issues.apache.org/jira/browse/SINGA-19?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangwei updated SINGA-19:
-------------------------
Description:
Some Param objects in deep learning models are much larger than other Param
objects. For example, a weight matrix is usually 100 times larger than a bias
vector. The difference in Param size causes two problems,
1. if there are multiple servers in one server group, then the servers may be
assigned different number of parameters to update.
2. if there are multiple server groups, e.g., in distributed Hogwild framework,
then these server groups may be assigned different number of parameters to
maintain.
This ticket its to slice large Param objects to solve the load-balance problem.
The slicing operations are done in the stub thread to make them transparent to
both workers and servers.
was:
Some Param objects in deep learning model are much larger than other Param
objects. For example, the weight matrix is usually 100 times larger than the
bias vector. The difference in Param size causes two problems,
1. if there are multiple servers in one server group, then the servers may be
assigned different number of parameters to update.
2. if there are multiple server groups, e.g., in distributed Hogwild framework,
then these server groups may be assigned different number of parameters to
maintain.
This ticket its to slice large Param objects to solve the load-balance problem.
The slicing operations are done in the stub thread to make them transparent to
both workers and servers.
> Slice large Param objects for load-balance
> ------------------------------------------
>
> Key: SINGA-19
> URL: https://issues.apache.org/jira/browse/SINGA-19
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
> Assignee: wangwei
>
> Some Param objects in deep learning models are much larger than other Param
> objects. For example, a weight matrix is usually 100 times larger than a bias
> vector. The difference in Param size causes two problems,
> 1. if there are multiple servers in one server group, then the servers may be
> assigned different number of parameters to update.
> 2. if there are multiple server groups, e.g., in distributed Hogwild
> framework, then these server groups may be assigned different number of
> parameters to maintain.
> This ticket its to slice large Param objects to solve the load-balance
> problem. The slicing operations are done in the stub thread to make them
> transparent to both workers and servers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
