[
https://issues.apache.org/jira/browse/SINGA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangwei resolved SINGA-130.
---------------------------
Resolution: Implemented
> Implement a layer subclass for data prefetching
> -----------------------------------------------
>
> Key: SINGA-130
> URL: https://issues.apache.org/jira/browse/SINGA-130
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
> Assignee: Anh Dinh
> Labels: data, multi-threading, prefetch
>
> Data prefetching is important for training with GPU, because the IO would
> become the bottleneck when the computation is very fast.
> One idea is to create a general prefetch layer which embeds the application
> specific data loading layers.
> {code}
> PrefetchLayer::ComptueFeature() {
> wait until the pretch thread finishes.
> swap the prefeth_data_ and data_ blobs.
> if (first time)
> load data into data_ blobs
> spawn a new thread to call functions from data loading layers for loading
> data into prefetch_data_.
> }
> {code}
>
> If the prefetch layer has multiple loading layers and is connected to
> multiple destination layers, then different destination layer may want data
> loaded by different loading layers. This case should be handled properly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
