wangwei created SINGA-19:
----------------------------
Summary: Slice large Param objects for load-balance
Key: SINGA-19
URL: https://issues.apache.org/jira/browse/SINGA-19
Project: Singa
Issue Type: New Feature
Reporter: wangwei
Assignee: wangwei
Some Param objects in deep learning model are much larger than other Param
objects. For example, the weight matrix is usually 100 times larger than the
bias vector. The difference in Param size causes two problems,
1. if there are multiple servers in one server group, then the servers may be
assigned different number of parameters to update.
2. if there are multiple server groups, e.g., in distributed Hogwild framework,
then these server groups may be assigned different number of parameters to
maintain.
This ticket its to slice large Param objects to solve the load-balance problem.
The slicing operations are done in the stub thread to make them transparent to
both workers and servers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
