[
https://issues.apache.org/jira/browse/SINGA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231527#comment-15231527
]
ASF subversion and git services commented on SINGA-130:
-------------------------------------------------------
Commit 4344ab7b2a17711d77465a6c52e50b41cad0dfef in incubator-singa's branch
refs/heads/master from [~ug93tad]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=4344ab7 ]
SINGA-130 Data prefetching
Move fetch_data thread to ComputeFeature()
Pass cpplint check
> Implement a layer subclass for data prefetching
> -----------------------------------------------
>
> Key: SINGA-130
> URL: https://issues.apache.org/jira/browse/SINGA-130
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
> Assignee: Anh Dinh
> Labels: data, multi-threading, prefetch
>
> Data prefetching is important for training with GPU, because the IO would
> become the bottleneck when the computation is very fast.
> One idea is to create a general prefetch layer which embeds the application
> specific data loading layers.
> {code}
> PrefetchLayer::ComptueFeature() {
> wait until the pretch thread finishes.
> swap the prefeth_data_ and data_ blobs.
> if (first time)
> load data into data_ blobs
> spawn a new thread to call functions from data loading layers for loading
> data into prefetch_data_.
> }
> {code}
>
> If the prefetch layer has multiple loading layers and is connected to
> multiple destination layers, then different destination layer may want data
> loaded by different loading layers. This case should be handled properly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
