John, Oddly enough one of my use cases was for a client using Grails before we spun that out and the Grails engineering team said that GORM support for Gemfire had gone into mothballs for just this lack of functionality. The generated controllers relied on streaming and paginated data, which is a common requirement against data stores. I’ll go back and resurrect that project and upgrade the gemfire-plugin to the Grails 3.0 when it comes. I’ll just need to understand how we’re implementing it.
Thanks, Wayne Wayne Lund Advisory Platform Architect 916.296.1893 [email protected] <mailto:[email protected]> <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> > On Nov 30, 2017, at 1:16 PM, John Blum <[email protected]> wrote: > > Wayne- > > The following 2 things are upcoming and coming for the next SD Release Train > (Lovelace), for both Spring Data for Apache Geode, and Spring Data GemFire... > > https://jira.spring.io/browse/SGF-452 <https://jira.spring.io/browse/SGF-452> > - Function Executions from a SD Repository > > https://jira.spring.io/browse/SGF-377 <https://jira.spring.io/browse/SGF-377> > - which is essentially Reactive Repository support with CQ-based query > methods, like this... > > Flux<Customer> customers = customerRepository.findByLastName("doe"); > > ... > > Yes, much more to come! > > > On Thu, Nov 30, 2017 at 1:05 PM, Wayne Lund <[email protected] > <mailto:[email protected]>> wrote: > I have wanted this from the early days of my using gemfire. Keep pushing. > > Wayne Lund > Advisory Platform Architect > 916.296.1893 <tel:(916)%20296-1893> > [email protected] <mailto:[email protected]> > > <PastedGraphic-1.tiff> > <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> > > <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> > >> On Nov 30, 2017, at 12:51 PM, Amit Pandey <[email protected] >> <mailto:[email protected]>> wrote: >> >> Thanks John..this is helpful and yes a fully reactive result collector from >> Geode will be awesome...I do think for large data sets this a must. >> >> On Fri, Dec 1, 2017 at 2:05 AM, John Blum <[email protected] >> <mailto:[email protected]>> wrote: >> 1 last thing... >> >> If you run the PeerCacheFunctionExecutionResultStreamingIntegrationTests >> test class, you will see the effects of Function Stream Result processing in >> you IDE (or on the command-line). >> >> Enjoy! >> -j >> >> >> On Thu, Nov 30, 2017 at 12:31 PM, John Blum <[email protected] >> <mailto:[email protected]>> wrote: >> Hi Amit, all- >> >> 2 things... >> >> >> 1. Have a look at... DATAGEODE-36 - "Add Async, Reactive and Streaming >> capabilities to the Function Annotation support." [1] >> >> I already started experimenting with Streaming Function results here [2]. >> Essentially the only thing preventing Apache Geode from Streaming results >> now is the default o.a.g.cache.execute.ResultCollector implementation (i.e. >> o.a.g.internal.cache.execute.DefaultResultCollector [3]) provided by Apache >> Geode OOTB. >> >> >> 2. Dan is correct about the batching functionality provided in SDG! >> >> You can see how it work by looking at the >> o.s.d.g.function.BatchingResultSender class [4], which is used by the >> o.s.d.g.function.PojoFunctionWrapper class [5] (which is how the entire >> Function "implementation" annotated POJO methods work, BTW! Well, that >> class and some Spring (Data Geode) infrastructure to enable it, ;-). >> >> >> Hope this helps! >> >> -John >> >> [1] https://jira.spring.io/browse/DATAGEODE-36 >> <https://jira.spring.io/browse/DATAGEODE-36> >> [2] >> https://github.com/jxblum/spring-gemfire-tests/blob/master/src/test/java/org/spring/data/gemfire/cache/PeerCacheFunctionExecutionResultStreamingIntegrationTests.java >> >> <https://github.com/jxblum/spring-gemfire-tests/blob/master/src/test/java/org/spring/data/gemfire/cache/PeerCacheFunctionExecutionResultStreamingIntegrationTests.java> >> [3] >> https://github.com/apache/geode/blob/rel/v1.3.0/geode-core/src/main/java/org/apache/geode/internal/cache/execute/DefaultResultCollector.java >> >> <https://github.com/apache/geode/blob/rel/v1.3.0/geode-core/src/main/java/org/apache/geode/internal/cache/execute/DefaultResultCollector.java> >> [4] >> https://github.com/spring-projects/spring-data-gemfire/blob/2.0.2.RELEASE/src/main/java/org/springframework/data/gemfire/function/BatchingResultSender.java >> >> <https://github.com/spring-projects/spring-data-gemfire/blob/2.0.2.RELEASE/src/main/java/org/springframework/data/gemfire/function/BatchingResultSender.java> >> [5] >> https://github.com/spring-projects/spring-data-gemfire/blob/2.0.2.RELEASE/src/main/java/org/springframework/data/gemfire/function/PojoFunctionWrapper.java#L143-L148 >> >> <https://github.com/spring-projects/spring-data-gemfire/blob/2.0.2.RELEASE/src/main/java/org/springframework/data/gemfire/function/PojoFunctionWrapper.java#L143-L148> >> >> >> On Thu, Nov 30, 2017 at 7:59 AM, Anthony Baker <[email protected] >> <mailto:[email protected]>> wrote: >> FWIW, I really like this suggestion. There is some interaction with the >> `isHA()` behavior but I would love to have a reactive Function that returns >> a mono / flux [1]. >> >> Anthony >> >> [1] https://spring.io/blog/2016/04/19/understanding-reactive-types >> <https://spring.io/blog/2016/04/19/understanding-reactive-types> >> >> >>> On Nov 28, 2017, at 10:23 PM, Amit Pandey <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hey Guys, >>> >>> I filed this JIRA ticket :- >>> https://issues.apache.org/jira/browse/GEODE-4027 >>> <https://issues.apache.org/jira/browse/GEODE-4027> >>> >>> Regards >>> >>> On Wed, Nov 29, 2017 at 2:01 AM, Amit Pandey <[email protected] >>> <mailto:[email protected]>> wrote: >>> Sure. >>> >>> On Wed, Nov 29, 2017 at 1:43 AM, Swapnil Bawaskar <[email protected] >>> <mailto:[email protected]>> wrote: >>> Amit, >>> Can you please file a JIRA? >>> >>> Thanks! >>> >>> On Tue, Nov 28, 2017 at 12:02 PM Amit Pandey <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hey Udo, >>> >>> if I was confident I wouldn't have griped about the non availability. I >>> think though that it will be a good addition. Because for large data >>> collections it's not enough to batch it from the server side. >>> >>> Without a streaming result collector its futile >>> >>> On Wed, Nov 29, 2017 at 1:05 AM, Udo Kohlmeyer <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hi there Amit. >>> >>> Geode does also work for PRs.... We actually look forward to the community >>> contributing to the product... :) >>> >>> But on a more serious note... a streaming results collector does require >>> more thought and potentially significant (positive) changes to the system >>> behavior. >>> >>> --Udo >>> >>> On Tue, Nov 28, 2017 at 11:21 AM, Amit Pandey <[email protected] >>> <mailto:[email protected]>> wrote: >>> Thanks Dan. >>> >>> A Streaming Result Collector in Geode is needed now . This will be great >>> addition. I know we can make it, but its pretty error prone to do it >>> >>> On Wed, Nov 29, 2017 at 12:05 AM, Dan Smith <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hi Amit, >>> >>> Looking at the code, it looks like if your annotated method returns an >>> array, or an Iteratable (like a List), and you have batchSize set, then >>> spring will send your data in batches back to the caller of the function. >>> Underneath the covers is it using geode's ResultSender and passing the >>> batches to the ResultSender sendResult method. >>> >>> So it looks like this would reduce the amount of data that gets sent to the >>> client in a single message. >>> >>> -Dan >>> >>> On Sun, Nov 26, 2017 at 9:11 PM, Amit Pandey <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hi John and everyone, >>> >>> How does Spring Data Geode Function annotation batch size work? >>> >>> So if we set it up like @GemfireFunction(id="lotMapService",HA=true, >>> batchSize =1000) >>> >>> >>> what are the implications ? What happens if there is a million data to be >>> streamed on the server as well as on the client side. >>> >>> Regards >>> >>> >>> >>> >>> >>> -- >>> Kindest Regards >>> ----------------------------- >>> Udo Kohlmeyer | Pivotal >>> [email protected] <mailto:[email protected]> >>> <http://www.gopivotal.com/> >>> www.pivotal.io <http://www.pivotal.io/> >>> >>> >>> >> >> >> >> >> -- >> -John >> john.blum10101 (skype) >> >> >> >> -- >> -John >> john.blum10101 (skype) >> > > > > > -- > -John > john.blum10101 (skype)
signature.asc
Description: Message signed with OpenPGP
