Such an awesome response. Can’t wait for the geode update so that I can try it out.
Sent from my iPhone > On Dec 15, 2017, at 7:37 PM, John Blum <[email protected]> wrote: > > Hi Dennis, all- > > Alright, 1 more update... > > I filed DATAGEODE-76 [1], which is an "improvement" to automatically register > SDG's internal, "administrative" Functions (e.g. > o.s.d.g.config.admin.functions.CreateRegionFunction [2]) when using SDG's > Annotation configuration support to configure and bootstrap Apache Geode > server's with Spring (Data Geode + Spring Boot) [3]. > > This is particularly useful since the @EnableClusterConfiguration annotation, > useHttp attributes is disabled (i.e. false) by default, primarily because it > requires a full installation of Apache Geode on the server. I may change > this default in the future, but for now, I will just automatically register > the SDG administrative Functions. > > This is already in progress, so this improvement will also make its way into > SDG 2.0.3.RELEASE (Kay-SR3). > > Let me know if you have any questions/feedback. > > Thanks, > John > > > [1] https://jira.spring.io/browse/DATAGEODE-76 > [2] > https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateRegionFunction.html > [3] > https://docs.spring.io/spring-data/geode/docs/current/reference/html/#bootstrap-annotation-config-client-server-applications > > >> On Fri, Dec 15, 2017 at 7:13 PM, John Blum <[email protected]> wrote: >> Hi Dennis- >> >> As promised, I am following up with a concrete example (function-example >> [1]); yay! >> >> Unfortunately, I have not documented this example yet (nor are any of the >> other examples in the Contacts Application SDG RI documented for that >> matter; 1 of my SDG goals in the New Year), but it demonstrates your UC and >> then some. >> >> Like DATAGEODE-68 [2], I found several other issues (argh!) prior to >> SpringOne Platform (S1P) 2017 when I was preparing my examples (i.e. [3] & >> [4]) for my talks. 1 of the more important issues include DATAGEODE-73 [5], >> which involves CQ and @EnableClusterConfiguration annotation. So, I am >> using a BUILD-SNAPSHOT for the function-example [1] to get this to work. >> This 2.0.3.BUILD-SNAPSHOT does not currently contain the fixes yet since I >> have not checked in, but will soon. However, the issues are fixed and know >> that DATAGEODE-68 [2], DATAGEODE-73 [5] along with several other issues are >> going to be included in the upcoming SDG 2.0.3.RELEASE (Kay-SR3 service >> release), probably due in the new year now. >> >> I only mention this because I threw everything along with the kitchen sink >> into this new function-example [1], ;-). >> >> This example includes Entity-defined Regions [6], CQs [7] (with a CQ event >> handler [8]), a client-side Function execution [9] (enabled with [10]) along >> with a server-side Function implementation [11] (enabled by [12]), a GemFire >> Repository [13] (enabled with [14]) and enables cluster configuration push >> (using HTTP) from the client [15] to the Spring Boot-configured/bootstrapped >> Apache Geode cache server application [16]. Additionally, on the server, I >> have enabled both an embedded Locator [17] and Manager [18], which starts >> immediately. By enabling both a Locator and Manager, this allows me to >> connect to this Spring Boot-configured/bootstrapped Apache Geode cache >> server using Gfsh, simply by doing this... >> >> gfsh> connect >> >> Both the embedded Locator (10334) and Manager (1099) are running with their >> default ports, as is the CacheServer (40404). The Spring Boot cache client >> application will connect directly to the CacheServer on port 40404 (the >> default), but, of course, all of this is configurable. >> >> Because I am using HTTP for the client-side cluster configuration push, I >> need to have a full installation of Apache Geode when I run the Spring Boot, >> Apache Geode cache server application. Therefore, I set the GEODE_HOME >> environment variable to the installation directory of Apache Geode 1.2.1 (on >> my machine) in my IDE run profile. >> >> Of course, I do not need to absolutely do this either, since I also created >> a nested Spring @Configuration class in the server code to manually register >> the internal SDG "administrative" Functions [19] I explained previously. >> When you enable this Spring profile (by using the Java System property on >> the command-line: -Dspring.profiles.active=non-http-cluster-config), then >> you can remove the useHttp attribute in the @EnableClusterConfiguration >> annotation on the client [15], since then the client will rely on the Apache >> Geode server being a Spring (Data Geode) enabled server with these Functions >> registered, which is why the server also @EnableGemfireFunctions [12], that >> and to register the AddressGeocodingFunction implementation on the server >> [20], which is used by the application, and specifically the client to, >> well, geocode Addresses. >> >> To geocode addresses, I included a dependency [21] on the caching-example >> [22], which demonstrates using Apache Geode as a caching provider in >> Spring's Cache Abstraction. Clearly, geocoding is a good candidate (and >> example ;-) of caching. There were several application service components >> that I recycled for this example in my server Function implementation. >> >> Anyway, it all works as expected! Hooray! Of course, as I mentioned, I >> need to check in fixes for some of the issues I encountered before you will >> be able to run it. Otherwise, you will encounter some of the issues I >> uncovered for S1P. As I said, they are already resolved and I am currently >> on the commit path so that the fixes are included in SDG 2.0.3.RELEASE >> (Kay-SR3). >> >> I will follow-up on this thread again when I have the fixes committed and >> when the SDG 2.0.3.BUILD-SNAPSHOT available repo.spring.io/libs-snapshot in >> should include the changes. >> >> Regarding... >> >> > I also discovered a few minor typos in the documentation, where should I >> > report those? >> >> Please file a JIRA ticket in the DATAGEODE JIRA project [23]. If you'd >> like, you are also welcome to submit a PR [24] with the doc edits. >> Contributions are always welcomed and appreciated. >> >> I am going to follow up with 1 more thing in just a minute, :-) >> >> But, I hope this sample clears up the muddy waters a bit, along with showing >> you (and others) the power of these new annotations. >> >> Cheers! >> -John >> >> >> [1] >> https://github.com/jxblum/contacts-application/tree/master/function-example >> [2] https://jira.spring.io/browse/DATAGEODE-68 >> [3] https://github.com/jxblum/scaling-springbootapplications-realtime >> [4] https://github.com/jxblum/simplifying-apache-geode-with-spring-data >> [5] https://jira.spring.io/browse/DATAGEODE-73 >> [6] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L61-L63 >> [7] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L60 >> [8] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L94-L104 >> [9] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/functions/AddressGeocodingFunctionExecution.java#L35-L39 >> [10] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L64 >> [11] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/functions/AddressGeocodingFunction.java#L58-L84 >> [12] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L48 >> [13] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/repo/AddressRepository.java >> [14] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L65 >> [15] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L66 >> [16] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L44-L50 >> [17] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L47 >> [18] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L46 >> [19] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L76-L94 >> [20] >> https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L60-L63 >> [21] >> https://github.com/jxblum/contacts-application/blob/master/function-example/pom.xml#L63-L67 >> [22] >> https://github.com/jxblum/contacts-application/tree/master/caching-example >> [23] https://jira.spring.io/issues/?jql=project%20%3D%20DATAGEODE >> [24] https://github.com/spring-projects/spring-data-geode/pulls >> >> >>> On Fri, Dec 15, 2017 at 1:07 PM, Dennis Dai <[email protected]> wrote: >>> Wow, thanks a lot for the detailed explanations and references! I was >>> slowly going through the documentation and trying out different things. >>> FYI, I did hit the bug you reported in DATAGEODE-68 along the way and >>> resolved it with your suggestions. I also discovered a few minor typos in >>> the documentation, where should I report those? >>> >>> Thanks, >>> Dennis >>> >>> >>>> On Dec 15, 2017, at 12:07 PM, John Blum <[email protected]> wrote: >>>> >>>> Hi Dennis- >>>> >>>> I apologize for your troubles. I possibly need to refine the docs; I am >>>> open to any suggestions here on how I might make it more clear. >>>> >>>> I sense there is possibly some confusion around how multiple annotations >>>> work in tandem (e.g. @EnableEntityDefinedRegions with >>>> @EnableClusterConfiguration along with using SDG's Function annotation >>>> support, enabled with @EnableGemfireFunctions). Without seeing your >>>> code/configuration, I am basing my understanding solely on your provided >>>> description and the log messages. >>>> >>>> Let's start with @EnableClusterConfiguration. >>>> >>>> >>>> 1) First, this annotation only sends Region and Index (both OQL and Lucene >>>> Indexes are supported) schema object definitions from the client to the >>>> server. Functions are not included in the cluster configuration push from >>>> the client. >>>> >>>> NOTE: Functions are not supported in the client to server cluster >>>> configuration push using @EnableClusterConfiguration since this would >>>> require the Function class definition(s) to be on the servers' classpath >>>> when the Function(s) get created/registered, which would entail uploading >>>> a JAR file. Additionally, Apache Geode does not recognize SDG POJO >>>> annotated Function "implementations" [1] (i.e. with @GemfireFunction). See >>>> here [2] for more details. >>>> >>>> >>>> 2) Second, the default behavior of @EnableClusterConfiguration when >>>> "pushing" schema object definitions (i.e. only Regions/Indexes) from the >>>> client to the server is to invoke "internal", "administrative" Functions >>>> provided by SDG. E.g. to create Regions on the servers from a client SDG >>>> uses the o.s.d.g.config.admin.functions.CreateRegionFunction [3]. To >>>> create Indexes on the servers from a client SDG uses the >>>> o.s.d.g.config.admin.functions.CreateIndexFunction [4]. Again, these are >>>> "internal" SDG "administrative" Functions and the default behavior for >>>> @EnableClusterConfiguration when pushing schema object definitions from >>>> the client to the server. >>>> >>>> Additionally, these internal SDG administrative Functions are not >>>> automatically registered by SDG on the servers since the servers may not >>>> be Spring enabled; e.g. they may have been started with Gfsh or via some >>>> other approach (although, in your case, the servers are Spring enabled >>>> since you bootstrapped the servers with Spring Boot; +1). >>>> >>>> Furthermore, these SDG provided Functions are not recommended for >>>> production purposes either since their "actions" are not recorded by >>>> Apache Geode's Cluster Configuration Service [5] (yet). Therefore a user >>>> must register these Functions manually (although, I am already getting >>>> ideas here). For instance, see my SDG integration test for >>>> @EnableClusterConfiguration based configuration [6]. Specially, have a >>>> look at the "forked" Apache Geode server (Spring) configuration here [7] >>>> and here [8]. >>>> >>>> >>>> 3) Third, the alternative to using these internal SDG administrative >>>> Functions is to set the @EnableClusterConfiguration(useHttp = true), By >>>> setting the useHttp annotation attribute to true, the SDG >>>> @EnableClusterConfiguration push will use Apache Geode's Management REST >>>> API to push the schema object definitions from the client to the servers. >>>> >>>> This approach has several advantages. One, the internal SDG >>>> administrative Functions are not used and therefore are not required to be >>>> manually registered on the servers in the cluster. Two, by using HTTP to >>>> push configuration from the client to the server, Apache Geode's Cluster >>>> Configuration Service will record the "actions" sent from the client. >>>> Thus, when you add additional members to the cluster, either with Spring >>>> Boot, or event with Gfsh, they will have the same/consistent >>>> configuration. If you shutdown the entire cluster and bring it back up, >>>> it will retain the configuration from before; all compliments of Apache >>>> Geode's Cluster Configuration Service., which SDG uses when useHttp is set >>>> to true. >>>> >>>> However, using HTTP does require a full installation of Apache Geode to be >>>> present on the system where the server is running. When using Spring Boot >>>> to bootstrap your servers, you can point these Spring Boot >>>> configured/bootstrapped servers to a full Apache Geode installation by >>>> setting the GEODE_HOME environment variable to the installation directory. >>>> >>>> FYI, though the Management REST API is not publicly advertised (like the >>>> "Developer REST API" [9]), it does exist because 1) I created it when I >>>> was working the GemFire/Geode engineering team, and 2) it is the same >>>> functionality used by Gfsh when you connect via HTTP, like so... >>>> >>>> gfsh> connect --use-http --url=http://localhost:7070/gemfire/v1 >>>> >>>> This "Management REST API" was created for the purposes of managing Apache >>>> Geode clusters running in a Cloud environment (e.g. AWS, or GCP) so users >>>> would not have to worry about firewalls when connecting Gfsh locally to >>>> the Apache Geode Manager in the cluster running in the Cloud. SDG's >>>> @EnableClusterConfiguration uses this vary Management REST API to carry >>>> out its function when useHttp is set to true. >>>> >>>> >>>> As for your application-defined Apache Geode Functions using SDG's >>>> Function annotation support, these will be registered and executable >>>> (using SDG's Function execution annotation support [10]) as you would >>>> expect. >>>> >>>> To make all of this more concrete, let me put a quick example together for >>>> you showcasing your UC. >>>> >>>> I will follow up in a short bit once I have completed the example; stay >>>> tuned. >>>> >>>> Cheers, >>>> John >>>> >>>> >>>> [1] >>>> https://docs.spring.io/spring-data/geode/docs/current/reference/html/#function-implementation >>>> [2] >>>> https://docs.spring.io/spring-data/geode/docs/current/reference/html/#_implementation_vs_execution >>>> [3] >>>> https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateRegionFunction.html >>>> [4] >>>> https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateIndexFunction.html >>>> [5] >>>> http://geode.apache.org/docs/guide/12/configuring/cluster_config/gfsh_persist.html >>>> [6] >>>> https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java >>>> [7] >>>> https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java#L195-L197 >>>> [8] >>>> https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java#L207-L220 >>>> [9] http://geode.apache.org/docs/guide/12/rest_apps/book_intro.html >>>> [10] >>>> https://docs.spring.io/spring-data/geode/docs/current/reference/html/#function-execution >>>> >>>> >>>>> On Thu, Dec 14, 2017 at 5:35 PM, Dennis Dai <[email protected]> wrote: >>>>> Hello, >>>>> >>>>> I am running a Geode server instance from spring boot (with >>>>> @CacheServerApplication annotation). Then on the client side (also spring >>>>> boot applications with @ClientCacheApplication annotation), I am trying >>>>> to define some regions (with @Region annotation). Together with >>>>> @EnableEntityDefinedRegions and @EnableClusterConfiguration on the client >>>>> side and @EnableGemfireFuctions on the server side, I was hoping the >>>>> regions would be automatically created on the server. But when I ran the >>>>> client app, I am getting an error saying: >>>>> >>>>> org.apache.geode.cache.client.ServerOperationException: remote server on >>>>> 192.168.1.100(SpringBasedCacheClientApplication:50794:loner):55570:4310ba57:SpringBasedCacheClientApplication: >>>>> The function is not registered for function id CreateRegionFunction >>>>> >>>>> On the server side, I got a similar error: >>>>> >>>>> 18:11:40.636 [ServerConnection on port 40405 Thread 1] WARN >>>>> org.apache.geode.internal.cache.tier.sockets.BaseCommand - Server >>>>> connection from >>>>> [identity(192.168.1.100(SpringBasedCacheClientApplication:50794:loner):55570:4310ba57:SpringBasedCacheClientApplication,connection=1; >>>>> port=55570]: The function is not registered for function id >>>>> CreateRegionFunction >>>>> >>>>> From @EnableGemfireFunctions API doc: >>>>> >>>>> Enables GemFire annotated Function implementations. Causes the container >>>>> to discover any beans that are annotated with >>>>> {code}@GemfireFunction{code}, wrap them in a PojoFunctionWrapper, and >>>>> register them with the cache. >>>>> >>>>> I thought that is all I need to get the spring data geode’s functions >>>>> (e.g., CreateRegionFunction) registered on the server side. What am I >>>>> missing here? >>>>> >>>>> Thanks, >>>>> Dennis >>>> >>>> >>>> >>>> -- >>>> -John >>>> john.blum10101 (skype) >>> >> >> >> >> -- >> -John >> john.blum10101 (skype) > > > > -- > -John > john.blum10101 (skype)
