On Jul 14, 2014, at 5:00 PM, Remi Forax <fo...@univ-mlv.fr> wrote: > and imagine the case where you know the size of a stream returned by flatMap, > in that case, > you may want to split before pumping the values of the stream. >
Since you don't know the characteristics of the flat mapped streams until you get them (i.e. traversing) any global optimizations (such as that related to exact size, or even sorted or distrinct) cannot be applied *unless* you turn flatMap into a stateful barrier. It's much easier to optimize concatenation when one explicitly knows upfront the characteristics of what is to be concatenated. FWIW I did ponder how in general flatMap could be modified for parallel computation e.g. with a flatMap wrapping spliterator, but i am a very skeptical of it's benefits. It's probably only of questionable benefit for small source streams where the size is less than the parallelism of the common pool and each upstream element is mapped to large number of downstream elements or the flat mapping cost is high (to overcome the cost of wrapping). Paul.
