2014-09-19 13:50 GMT+02:00 Marko Topolnik <[email protected]>:
> > > On Friday, September 19, 2014 1:40:30 PM UTC+2, Lukas Eder wrote: >> >> >> 2014-09-19 13:35 GMT+02:00 Marko Topolnik <[email protected]>: >> >>> Hi Lukas, >>> >>> I understand your concern for backward compatibility. BTW I already saw >>> your jOOλ work and immediately recognized all the stream operators I have >>> missed from the Streams API :) I may start working with it as my default >>> approach to Streams. >>> >> >> Great! If you feel that anything's missing, just create a pull request! >> :-) >> >> >>> As for use-cases of parellelization, there is really nothing specific to >>> a result set about it: pick your favorite example of computation-intensive >>> processing of a large dataset. Whenever I/O speed overtakes CPU speed, >>> you've got a parallelization target. My specific use case are Lucene >>> queries executed against each row of DB data. The key advantage of a lazy >>> stream is that it turns O(n) space into O(1), making your code scalable to >>> datasets of indefinite size. >>> >> >> (... if you have infinite cores ;-) ) >> > > Actually, whenever your server is not at 100% utilization by concurrent > client pressure, and that's basically almost all the time. At peak request > intensity it can be true that a single thread per request is better, but it > is by no means obvious. If all the requests are CPU-limited, then serving > one request at a time, using parellel processing, would again be the best > option (less DB connections and less state maintenance for exactly the same > overall throughput). > I meant that the O(1) assumption is only correct with an infinite number of cores. Otherwise, you'll get something like O(n / c), which is still O(n) > > >> The stream can either be reduced to some O(1) result, or mapped to an >>> output form which is written directly to an output stream. >>> >> >> I see. So a pretty standard use-case. Be aware though that jOOQ's Cursor >> operates on an open JDBC ResultSet internally. You must ensure that a row >> is completely consumed by one thread, before another thread advances the >> ResultSet to the next row. >> >> By default, jOOQ's Cursor is not thread-safe, i.e. there is no internal >> synchronization to prevent rows from being skipped in such use-cases. >> > > And that's where the beauty of the Spliterator API comes in :) > > Despite their obvious utility in parallel algorithms, spliterators are not >> expected to be thread-safe; instead, implementations of parallel algorithms >> using spliterators should ensure that the spliterator is only used by one >> thread at a time. > > OK, I overlooked this important detail so far! Thanks for sharing! -- You received this message because you are subscribed to the Google Groups "jOOQ User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
