Here's the diagram about my idea. https://docs.google.com/drawings/d/1cjz50sGbpnFp2oab30cZ5MNYsaD3PtaBRVsUWuLiglI/edit?usp=sharing
On Wed, May 27, 2015 at 5:04 PM, Edward J. Yoon <[email protected]> wrote: > Oh wait. a DNN BSPJob partitions the input data (examples) into the > number of tasks, then each task executes SGD graph job using > LocalBSPRunner as a mini-batch, and uses external pm server to avoid > the heavy cost of synchronization of shared variables such as > gradients or parameters. Does it make sense? > > On Wed, May 27, 2015 at 9:34 AM, Edward J. Yoon <[email protected]> > wrote: >> According to their architecture, DNN job is consist of multiple (mini-batch) >> SGD job groups and 'ParameterServer' is used for swapping parameters among >> them. Since these mini-batch jobs need to be work asynchronously, I guess we >> can't do this in single job of Hama BSP framework. Hama can be used as a sub >> component. >> >> -- >> Best Regards, Edward J. Yoon >> >> -----Original Message----- >> From: Chia-Hung Lin [mailto:[email protected]] >> Sent: Tuesday, May 26, 2015 5:40 PM >> To: [email protected] >> Subject: Re: DNN at Tencent >> >> That's why I am working on refactoring core module stuff : ) >> >> BSP is suitable for iterative applications and is for general purpose >> parallelism. So as long as the application can be expressed as >> iterative algorithm, ideally our framework (in terms of higher >> abstraction level) should have no problem for such applications. >> >> >> On 26 May 2015 at 16:08, Edward J. Yoon <[email protected]> wrote: >>> Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf >>> >>> According to them, they used BSP model (and additional Parameter >>> server) like google's distbelief. Do you think we can do this? >>> >>> >>> -- >>> Best Regards, Edward J. Yoon >> >> > > > > -- > Best Regards, Edward J. Yoon -- Best Regards, Edward J. Yoon
