wangwei created SINGA-130:
-----------------------------
Summary: Implement a layer subclass for data prefetching
Key: SINGA-130
URL: https://issues.apache.org/jira/browse/SINGA-130
Project: Singa
Issue Type: New Feature
Reporter: wangwei
Data prefetching is important for training with GPU, because the IO would
become the bottleneck when the computation is very fast.
One idea is to create a general prefetch layer which embeds the application
specific data loading layers.
{code}
PrefetchLayer::ComptueFeature() {
wait until the pretch thread finishes.
swap the prefeth_data_ and data_ blobs.
if (first time)
load data into data_ blobs
spawn a new thread to call functions from data loading layers for loading
data into prefetch_data_.
}
{code}
If the prefetch layer has multiple loading layers and is connected to multiple
destination layers, then different destination layer may want data loaded by
different loading layers. This case should be handled properly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
