> On 13 Nov 2018, at 10:35 AM, Weijun Wang <[email protected]> wrote:
>
> I'm scanning a file and downloading links inside:
>
> lines.flapMap(x -> Stream.ofNullable(findURIFrom(x)))
> .map(l -> download(c, l))
> .forEach(f -> f.join());
>
> CompletableFuture<HttpResponse<Path>> download(HttpClient c, URI link) {
> return c.sendAsync(HttpRequest.newBuilder(link).build(),
> HttpResponse.BodyHandlers.ofFile(Path.of(link.getPath())));
> }
>
> However, it seems the download is one by one and not parallel. I guess maybe
> map() is lazy and each request is only send when forEach() is called and the
> next one is only sent after join().
>
> I can only collect the jobs into a list and then call join() on
> CompletableFuture.allOf(list). Is there a simpler way?
Just guess your Stream (lines) is not parallel, so either use parallel stream
or collect into list before join should work?
lines.parallel().flapMap(x -> Stream.ofNullable(findURIFrom(x)))
.map(l -> download(c, l))
.forEach(f -> f.join());
Or
lines.flapMap(x -> Stream.ofNullable(findURIFrom(x)))
.map(l -> download(c, l))
.collect(Collectors.toList())
.forEach(f -> f.join());
Regards,
Chris
>
> Thanks
> Max
