Hi Marko, Thanks for sharing, and off to Reddit with the article - this is very interesting!
I do believe that such streams are very easy to achieve if the jOOQ API were allowed to make use of the JDK 8 libraries. JDBC's ResultSet really is a Stream, apart from the fact that they're not formally integrated, and that ResultSet doesn't provide any type information for Stream elements. But jOOQ provides that type information bridge and is capable of streaming the ResultSet. So, we're very close :-) 2014-11-20 11:03 GMT+01:00 Marko Topolnik <[email protected]>: > I wrote a piece on exploiting lazy streams in REST responses with Spring > MVC. You may be interested in similar ideas with respect to the integration > of jOOQ with Spring JDBC. The most sensitive aspect are holdable cursors, I > haven't explored how to achieve them without Hibernate sessions. Could turn > out to be easier than with Hibernate. > > https://www.airpair.com/java/posts/spring-streams-memory-efficiency > > On Monday, November 10, 2014 7:46:07 AM UTC+1, Lukas Eder wrote: >> >> Very interesting thanks for the explanation. We'll consider that for jOOQ >> 3.6: >> https://github.com/jOOQ/jOOQ/issues/3753 >> >> Cheers >> Lukas >> >> 2014-11-09 21:57 GMT+01:00 Marko Topolnik <[email protected]>: >> >> This is how I came about my information: I am submitting a pull request >>> to the Spring Framework. To that end I have checked out their complete >>> project, which includes a self-contained build system (gradle). The project >>> settings are such that the source-level compatibility is set to 1.6, but >>> dependencies on Java 8-specific APIs are allowed. There is also an >>> annotation @UsesJava8, supposed to mark classes which make use of these >>> APIs (I'm not 100% sure on the semantics of these annotations). >>> >>> I have successfully built the entire Spring Core project with my Java >>> 8-specific class, I just wasn't allowed to use the lambda syntax. >>> >>> On Sunday, November 9, 2014 6:09:54 PM UTC+1, Lukas Eder wrote: >>>> >>>> Hi Marko, >>>> >>>> That's interesting - I wasn't aware of Spring doing that. Are you aware >>>> of any blog post / documentation explaining how exactly they're doing it? I >>>> can't imagine that it would be possible to achieve both binary and *source* >>>> compatibility with Java 6 while integrating with Java 8 libraries - without >>>> some clever trickery. Also, I suspect that these techniques will be limited >>>> to method arguments and result types. Nonetheless, yes, why not think about >>>> this? >>>> >>>> Cheers >>>> Lukas >>>> >>>> 2014-11-09 10:03 GMT+01:00 Marko Topolnik <[email protected]>: >>>> >>>> Hi Lukas, >>>>> >>>>> I'm still involved with various considerations with respect to support >>>>> for Streams in legacy code. For example, Spring does accept support for >>>>> Java 8 APIs even while strictly maintaining runtime compatibility with >>>>> Java >>>>> 6. Can you explain what exactly is a showstopper in your case when e.g. >>>>> you >>>>> try to introduce a Stream-returning method to a class? My thinking is that >>>>> nothing breaks unless you actually call the method. Is there something >>>>> else >>>>> I'm overlooking? I'm asking this because I'm considering to propose a >>>>> similar thing in Hibernate. >>>>> >>>>> Thanks, >>>>> Marko >>>>> >>>>> On Friday, September 19, 2014 1:15:58 PM UTC+2, Lukas Eder wrote: >>>>>> >>>>>> Hi Marko, >>>>>> >>>>>> jOOQ currently doesn't support any Java 8 API as we want to maintain >>>>>> backwards compatibility with Java 6 for our customers. However, with jOOQ >>>>>> 3.5, we're going to have Cursor extend Iterable (and perhaps also >>>>>> Iterator) >>>>>> for better interoperability with Java 8 Streams. >>>>>> >>>>>> Note that we've also recently published jOOλ ( >>>>>> https://github.com/jOOQ/jOOL), which is an enhancement to the JDK's >>>>>> Stream API, supporting a variety of convenience methods for purely >>>>>> sequential streams. >>>>>> >>>>>> The nice thing about Streams is that the above can be used without >>>>>>> further ado to parallelize the result set processing. In actual >>>>>>> projects I >>>>>>> do provide my own splitting logic, but that's a relatively minor detail. >>>>>> >>>>>> >>>>>> Interesting. What would be a use-case for parallelised result set >>>>>> processing? >>>>>> >>>>>> Cheers >>>>>> Lukas >>>>>> >>>>>> 2014-09-19 12:42 GMT+02:00 Marko Topolnik <[email protected]>: >>>>>> >>>>>>> I'm working with JOOQ 3.4.1 and don't see any direct support for >>>>>>> making a lazy Java 8 Stream from JOOQ's Cursor. Is such a thing planned >>>>>>> or >>>>>>> am I missing something already provided? For reference, this is my >>>>>>> current >>>>>>> solution which works for me: >>>>>>> >>>>>>> public class JooqResultSpliterator<R extends Record> extends >>>>>>> AbstractSpliterator<R> >>>>>>> { >>>>>>> private final Cursor<R> cursor; >>>>>>> >>>>>>> public JooqResultSpliterator(ResultQuery<R> q) { >>>>>>> super(Long.MAX_VALUE, NONNULL | ORDERED); >>>>>>> this.cursor = q.fetchSize(50).fetchLazy(); >>>>>>> } >>>>>>> >>>>>>> @Override public boolean tryAdvance(Consumer<? super R> action) { >>>>>>> final R r = cursor.fetchOne(); >>>>>>> if (r == null) return false; >>>>>>> action.accept(r); >>>>>>> return true; >>>>>>> } >>>>>>> >>>>>>> @Override public void forEachRemaining(Consumer<? super R> action) >>>>>>> { >>>>>>> cursor.fetchInto(action::accept); >>>>>>> } >>>>>>> >>>>>>> public static <R extends Record> Stream<R> >>>>>>> jooqStream(ResultQuery<R> q) { >>>>>>> final JooqResultSpliterator<R> s = new >>>>>>> JooqResultSpliterator<>(q); >>>>>>> return StreamSupport.stream(s, false).onClose(s.cursor::close); >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> The nice thing about Streams is that the above can be used without >>>>>>> further ado to parallelize the result set processing. In actual >>>>>>> projects I >>>>>>> do provide my own splitting logic, but that's a relatively minor detail. >>>>>>> >>>>>>> >>>>>>> Marko >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "jOOQ User Group" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "jOOQ User Group" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> 2014-11-09 10:03 GMT+01:00 Marko Topolnik <[email protected]>: >>>> >>>> Hi Lukas, >>>>> >>>>> I'm still involved with various considerations with respect to support >>>>> for Streams in legacy code. For example, Spring does accept support for >>>>> Java 8 APIs even while strictly maintaining runtime compatibility with >>>>> Java >>>>> 6. Can you explain what exactly is a showstopper in your case when e.g. >>>>> you >>>>> try to introduce a Stream-returning method to a class? My thinking is that >>>>> nothing breaks unless you actually call the method. Is there something >>>>> else >>>>> I'm overlooking? I'm asking this because I'm considering to propose a >>>>> similar thing in Hibernate. >>>>> >>>>> Thanks, >>>>> Marko >>>>> >>>>> On Friday, September 19, 2014 1:15:58 PM UTC+2, Lukas Eder wrote: >>>>>> >>>>>> Hi Marko, >>>>>> >>>>>> jOOQ currently doesn't support any Java 8 API as we want to maintain >>>>>> backwards compatibility with Java 6 for our customers. However, with jOOQ >>>>>> 3.5, we're going to have Cursor extend Iterable (and perhaps also >>>>>> Iterator) >>>>>> for better interoperability with Java 8 Streams. >>>>>> >>>>>> Note that we've also recently published jOOλ ( >>>>>> https://github.com/jOOQ/jOOL), which is an enhancement to the JDK's >>>>>> Stream API, supporting a variety of convenience methods for purely >>>>>> sequential streams. >>>>>> >>>>>> The nice thing about Streams is that the above can be used without >>>>>>> further ado to parallelize the result set processing. In actual >>>>>>> projects I >>>>>>> do provide my own splitting logic, but that's a relatively minor detail. >>>>>> >>>>>> >>>>>> Interesting. What would be a use-case for parallelised result set >>>>>> processing? >>>>>> >>>>>> Cheers >>>>>> Lukas >>>>>> >>>>>> 2014-09-19 12:42 GMT+02:00 Marko Topolnik <[email protected]>: >>>>>> >>>>>>> I'm working with JOOQ 3.4.1 and don't see any direct support for >>>>>>> making a lazy Java 8 Stream from JOOQ's Cursor. Is such a thing planned >>>>>>> or >>>>>>> am I missing something already provided? For reference, this is my >>>>>>> current >>>>>>> solution which works for me: >>>>>>> >>>>>>> public class JooqResultSpliterator<R extends Record> extends >>>>>>> AbstractSpliterator<R> >>>>>>> { >>>>>>> private final Cursor<R> cursor; >>>>>>> >>>>>>> public JooqResultSpliterator(ResultQuery<R> q) { >>>>>>> super(Long.MAX_VALUE, NONNULL | ORDERED); >>>>>>> this.cursor = q.fetchSize(50).fetchLazy(); >>>>>>> } >>>>>>> >>>>>>> @Override public boolean tryAdvance(Consumer<? super R> action) { >>>>>>> final R r = cursor.fetchOne(); >>>>>>> if (r == null) return false; >>>>>>> action.accept(r); >>>>>>> return true; >>>>>>> } >>>>>>> >>>>>>> @Override public void forEachRemaining(Consumer<? super R> action) >>>>>>> { >>>>>>> cursor.fetchInto(action::accept); >>>>>>> } >>>>>>> >>>>>>> public static <R extends Record> Stream<R> >>>>>>> jooqStream(ResultQuery<R> q) { >>>>>>> final JooqResultSpliterator<R> s = new >>>>>>> JooqResultSpliterator<>(q); >>>>>>> return StreamSupport.stream(s, false).onClose(s.cursor::close); >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> The nice thing about Streams is that the above can be used without >>>>>>> further ado to parallelize the result set processing. In actual >>>>>>> projects I >>>>>>> do provide my own splitting logic, but that's a relatively minor detail. >>>>>>> >>>>>>> >>>>>>> Marko >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "jOOQ User Group" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "jOOQ User Group" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "jOOQ User Group" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "jOOQ User Group" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "jOOQ User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
