Re: Range and poolTask
On Monday, 6 June 2016 at 11:25:00 UTC, Rene Zwanenburg wrote:
Could you elaborate a bit?
Yes.
I have an InputRange and need to pass it throughout a couple of
iteration and manipulation functions such as filter, map and
finishing by grouping with fold. Like:
myrange
.filter!xxx
.map!yyy
.tee!zzz
.fold!www
This process take a lot of time, so I decide to put it in
parallel. I did something like this:
Appender!www output;
foreach(iii; parallel(myrange))
{
only(iii)
.filter!xxx
.map!yyy
.tee!zzz
.copy(output)
}
output.fold!www;
only() will create a Range with one item in it. Also, I can't
group directly. I
need to use a second Range.
It works but it's not elegant as the non-working example above is.
Re: Range and poolTask
On Monday, 6 June 2016 at 10:26:11 UTC, moechofe wrote: The functions passed to map or amap take the type of the element of the range as argument, but not a range itself. Right. I don't think I understand what the semantics of your example would be though.. Could you elaborate a bit?
Re: Range and poolTask
On Monday, 6 June 2016 at 09:38:32 UTC, Rene Zwanenburg wrote: http://dlang.org/phobos/std_parallelism.html#.TaskPool Or, more specifically, http://dlang.org/phobos/std_parallelism.html#.TaskPool.amap http://dlang.org/phobos/std_parallelism.html#.TaskPool.map The functions passed to map or amap take the type of the element of the range as argument, but not a range itself.
Re: Range and poolTask
On Monday, 6 June 2016 at 09:32:30 UTC, moechofe wrote:
I wonder if it is possible to write something like this:
---
// taskPool.distribute -- take a range and distribute entries
to different threads.
dirEntries().distribute(function(R1,R2)(R1 from,R2 to){
from
.filter!xxx
.map!yyy
.tee!zzz(to);
})
.each!www;
---
This would be great.
This might be useful:
http://dlang.org/phobos/std_parallelism.html#.TaskPool
Or, more specifically,
http://dlang.org/phobos/std_parallelism.html#.TaskPool.amap
http://dlang.org/phobos/std_parallelism.html#.TaskPool.map
Range and poolTask
I wonder if it is possible to write something like this:
---
// taskPool.distribute -- take a range and distribute entries to
different threads.
dirEntries().distribute(function(R1,R2)(R1 from,R2 to){
from
.filter!xxx
.map!yyy
.tee!zzz(to);
})
.each!www;
---
This would be great.
