[
https://issues.apache.org/jira/browse/SINGA-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangwei resolved SINGA-71.
--------------------------
Resolution: Fixed
> Fix a buf from repeated averaging gradients from workers
> --------------------------------------------------------
>
> Key: SINGA-71
> URL: https://issues.apache.org/jira/browse/SINGA-71
> Project: Singa
> Issue Type: Bug
> Reporter: wangwei
> Assignee: wangwei
>
> For workers from the same group, if they share parameters, gradients of
> parameters should be averaged. SINGA averages the gradients by
> * aggregates gradients from all workers
> * calls Updater::Update() passing the scale_grad = 1.0 / worker_group_size.
> * The Update function multiplies the gradient with scale_grad.
> The bug was from the Trainer::HandleUpdate() which averaged the gradients
> from local workers. The bug has been fixed by removing the averaging
> operation. Gradients from local workers are (only) summed before sending to
> servers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
