Hi All,
Thanks a bunch for your replies. I never liked static fields myself either,
and it looks that code like this (eg. GemFireCacheImpl.getInstance())
always bites you sooner or later.
John,
Appreciate your code with examples. Unfortunately, I couldn't find anywhere
dynamic region creation (on server side triggered by client). Most of the
time, server process starts with static configuration (either Spring Config
/ annotations or cache.xml). Did I look in the wrong places ?
Another non-intuitive behavior is when creating client region (as PROXY). I
expect the code to fail on create() method not put() (is that expected?)
ClientCache cache = new ClientCacheFactory()
.set("log-level", "warning")
.addPoolServer("127.0.0.1", 1234)
.create();
// shouldn't it fail here ?
Region<String, String> region = cache.<String,
String>createClientRegionFactory(ClientRegionShortcut.PROXY)
.create("foo");
region.put("aaa", "bbb"); // instead it fails on this line
assertThat(region.get("aaa")).isEqualTo("bbb");
Caused by: org.apache.geode.cache.RegionDestroyedException: Server
> connection from
> [identity(192.168.0.4(6382:loner):57357:a5c820e2,connection=1; port=57358]:
> Region
> named /foo was not found during put request
Regards,
Andrei
On Thu, Dec 8, 2016 at 7:59 PM, Wayne Lund <[email protected]> wrote:
> What a great answer. I think your test names are just too short though :)
>
> *Wayne Lund*
> Platform Architect
> 916.296.1893 <(916)%20296-1893>
> [email protected] <[email protected]>
> www.pivotal.io
>
> https://spring.io
>
> On Dec 8, 2016, at 3:44 PM, John Blum <[email protected]> wrote:
>
> Hi Andrei-
>
> I don't have much time to give a complete explanation, but I do very
> similar testing in *Spring Data GemFire/Geode* for both *Pivotal GemFire*
> and *Apache Geode*.
>
> I also have done very similar things in *Spring Session* with the GemFire
> support.
>
> Here are a few examples...
>
> 1. Most recently, I created a client/server test
> <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java>
> [1]
> where I test *Geode's* Security infrastructure/framework in my *Contact
> Application* RI
> <https://github.com/jxblum/contacts-application/tree/apache-geode> [2].
> The test "forks
> <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L131-L141>"
> [3] a GemFire Server for testing purposes (note: the test acts as the
> cache client, of course).
>
> This example is unique since it showcases many of the new features in SDG
> (like
> using and building on my new Annotation configuration model for SDG
> GemFire/Geode applications, *Spring Boot* style, for both clients
> <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L177-L182>
> [12]
> and servers
> <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L196-L208>
> [13])
> that I am developing to greatly simplify the "*getting started*"
> experience, even above and beyond the "Geode in 5 min
> <https://cwiki.apache.org/confluence/display/GEODE/Index#Index-Geodein5minutesGeodein5minutes>"
> [4], which is well, cumbersome since it does not bode well for creating an
> actual "automated", "client/server, integration" test.
>
> 2. The SDG test suite has many client/server integration tests, most
> notably in the org.springframework.data.gemfire.client
> <https://github.com/spring-projects/spring-data-gemfire/tree/master/src/test/java/org/springframework/data/gemfire/client>
> package
> [5].
>
> 3. There are also some CQ related tests in org.springframework.data.ge
> mfire.listener
> <https://github.com/spring-projects/spring-data-gemfire/tree/master/src/test/java/org/springframework/data/gemfire/listener>
> package
> [6].
>
> 4. In fact, if you search the SDG test suite source for uses of ForkUtil
> <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/ForkUtil.java>
> [7]
> (old) and ProcessExecutor
> <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/process/ProcessExecutor.java>
> [8]
> (relatively new), you will see all the (integration) tests that fork
> processes.
>
> Some tests like GemfireTemplateQueriesOnGroupedPooledClientCacheRegions
> IntegrationTests
> <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/GemfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests.java>
> [9]
> even fork and coordinate
> <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/GemfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests.java#L105-L108>
> [10]
> (via Locator) multiple Geode Servers.
>
> 5. Additionally, you can see the integration test infrastructure support
> <https://github.com/spring-projects/spring-session/tree/1.3.0.RC1/spring-session/src/integration-test/java/org/springframework/session/data/gemfire>
> [11]
> in the *Spring Session* GemFire support.
>
> Unfortunately, I have learned a lot since then and need to really clean
> all this up and standardize around a common test programming model for
> [*Spring
> Data*] *GemFire/Geode*, which is already in the works and is based on
> several things... first, my new Annotation configuration model (which will
> greatly simplify boilerplate test setup/tearDown, ugh),
> *Spring TestContext* and abstraction and *JUnit 5*.
>
> Anyway, even if you are not using *Spring* [*Data GemFire/Geode*],
> hopefully this gives you some ideas of how to organize and create your own
> (integration) tests.
>
>
> Cheers!
> John
>
> [1] https://github.com/jxblum/contacts-application/blob/apache
> -geode/security-example/src/test/java/example/app/geode/security/
> GeodeSecurityIntegrationTests.java
> [2] https://github.com/jxblum/contacts-application/tree/apache-geode
> [3] https://github.com/jxblum/contacts-application/blob/apache
> -geode/security-example/src/test/java/example/app/geode/security/
> GeodeSecurityIntegrationTests.java#L131-L141
> [4] https://cwiki.apache.org/confluence/display/GEODE/Index#Index-
> Geodein5minutesGeodein5minutes
> [5] https://github.com/spring-projects/spring-data-gemfire/tree/master/src
> /test/java/org/springframework/data/gemfire/client
> [6] https://github.com/spring-projects/spring-data-gemfire/tree/master/src
> /test/java/org/springframework/data/gemfire/listener
> [7] https://github.com/spring-projects/spring-data-gemfire/blob/master/src
> /test/java/org/springframework/data/gemfire/ForkUtil.java
> [8] https://github.com/spring-projects/spring-data-gemfire/blob/master/src
> /test/java/org/springframework/data/gemfire/process/ProcessExecutor.java
> [9] https://github.com/spring-projects/spring-data-gemfire/blob/master/src
> /test/java/org/springframework/data/gemfire/G
> emfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests
> .java
> [10] https://github.com/spring-projects/spring-data-gemfire/blob/master/
> src/test/java/org/springframework/data/gemfire/GemfireTemplateQueries
> OnGroupedPooledClientCacheRegionsIntegrationTests.java#L105-L108
> [11] https://github.com/spring-projects/spring-
> session/tree/1.3.0.RC1/spring-session/src/integration-test/java/org/
> springframework/session/data/gemfire
> [12] https://github.com/jxblum/contacts-application/blob/apache
> -geode/security-example/src/test/java/example/app/geode/security/
> GeodeSecurityIntegrationTests.java#L177-L182
> [13] https://github.com/jxblum/contacts-application/blob/apache
> -geode/security-example/src/test/java/example/app/geode/security/
> GeodeSecurityIntegrationTests.java#L196-L208
>
>
> On Thu, Dec 8, 2016 at 3:38 PM, Udo Kohlmeyer <[email protected]>
> wrote:
>
>> Hi Andrei,
>>
>> #1 The reason why this restriction is there is because currently the
>> Cache is a singleton. I suppose the correct way of stating this is, legacy.
>> There are some JIRA's relating to changing all of this, but it will take a
>> little more time. But you are most welcome to join the cause to help us
>> improve the Geode product. :D
>>
>> #2 Not really a follow up question here... But I imagine you'd want to
>> know how mimic this... A CacheListener is a "post-commit" event processor.
>> Which means, it will be invoke after the entry "commit". So in order to
>> best mimic similar behavior on a server you would have to implement a
>> CacheListener (or extend CacheListenerAdapter). Then you implement the
>> afterCreate, afterUpdate, afterInvalidate and afterDestroy methods. Most
>> likely your code in each of the methods would be a simple filter statement
>> to determine if the data entry matches your "query".
>>
>> Please note, you cannot use OQL or the Query engine here, so it is a
>> little more manual and labor intensive to write some code that could take
>> an OQL statement and convert it to a filter.
>>
>> Once your filter code returns with a true, you could invoke a callback
>> method for a registered processor object (look at the init() method on the
>> CacheListener
>> <https://cwiki.apache.org/confluence/display/GEODE/CacheWriter+and+CacheListener+Best+Practices>)
>> to trigger some other behavior. (there are many ways to skin a cat, so
>> please bear with me)
>>
>> #3 Correct. You'd have to imagine that a client is a "Server Lite". It
>> does not have redundancy or FT but you can still create regions on a client
>> and use them. So creating a region from a client is "harder". In order to
>> do something on the server (from a client), it would almost be best to use
>> a Function to invoke the behavior/action on the server(s).
>>
>> --Udo
>>
>> On 12/8/16 15:13, Andrei Sereda wrote:
>>
>> Hi Udo,
>>
>> Thanks for your reply. I looked at JUnit4BasicDUnitTest
>> <https://github.com/apache/geode/blob/develop/geode-core/src/test/java/org/apache/geode/test/dunit/tests/JUnit4BasicDUnitTest.java>
>> and
>> it seems it is using RMI to create regions on the server cache (similar to
>> Function concept you proposed). Some follow-ups:
>>
>> #1 : I would like to avoid writing infrastructure code around managing
>> individual VMs. But it seems for our use-case (single client, one cache)
>> there is no other way. Any reason why there is such restriction of distinct
>> VMs for client/server ?
>>
>> #2 : Our code makes extensive use of Queries and CQs already, so I wan't
>> be able to simulate this functionality with CacheListeners.
>>
>> #3 : We can't create region just on the client (via ClientRegionFactory
>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/client/ClientRegionFactory.html>)
>> since put / query / CQ will not work together (have tried both PROXY /
>> PROXY_CACHING shortcuts) "Create region" call must originate from client
>> (before test runs) therefore one doesn't have access to Server Region
>> Factory
>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/Cache.html#createRegionFactory-->.
>> I will try the Function approach.
>>
>> Please let me know if above reasoning is correct.
>>
>> Regards,
>> Andreo.
>>
>>
>> On Thu, Dec 8, 2016 at 4:25 PM, Udo Kohlmeyer <[email protected]>
>> wrote:
>>
>>> Hi there Andrei.
>>>
>>> #1: Clients and Servers cannot coexist within the same JVM. What has
>>> been done for testing within Geode, is that we use the concept of a VM
>>> <https://github.com/apache/geode/blob/develop/geode-core/src/test/java/org/apache/geode/test/dunit/VM.java>.
>>> Which allows us to start multiple servers/clients in the same JVM. You can
>>> review the JUnit4BasicDUnitTest
>>> <https://github.com/apache/geode/blob/develop/geode-core/src/test/java/org/apache/geode/test/dunit/tests/JUnit4BasicDUnitTest.java>
>>> to get an idea how we use it.
>>>
>>> #2: Correct. CQ's is a client-side concept. If you want similar
>>> behavior, you could try writing a custom CacheListener (or
>>> AsyncEventListener for a more batch oriented approach) if you need to
>>> trigger something on the server-side.
>>>
>>> #3: We do have a Java API that you could use to create Regions. Geode
>>> JavaDoc <http://geode.apache.org/releases/latest/javadoc/index.html>.
>>> You could also look at the Geode in 5min
>>> <https://cwiki.apache.org/confluence/display/GEODE/Index#Index-Geodein5minutesGeodein5minutes>,
>>> as the Java API creates a region on the Client. If you were to use that
>>> approach on the Server you would use Server Region Factory
>>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/Cache.html#createRegionFactory-->
>>> to create a region (in Java code) on the server. If you needed to trigger
>>> this from a client, you could always use a Function
>>> <http://geode.apache.org/docs/guide/developing/function_exec/chapter_overview.html>
>>> that could create a Region on the server. Once, the regions are create on
>>> the server(s) those regions will be usable for queries and cq's.
>>>
>>> --Udo
>>>
>>>
>>> On 12/8/16 10:28, Andrei Sereda wrote:
>>>
>>> Hello,
>>>
>>> I'm trying to bootstrap my application with prefabricated geode
>>> (gemfire) cache for simple tests.
>>>
>>> Basically what is necessary is to create a couple of regions, populate
>>> them and run some queries (including CQ) on the top of existing data,
>>> perhaps also validate transaction functionality.
>>>
>>> Ideally one should be able to re-create regions (recycle cache) multiple
>>> times. This also, has to run independently (no external process
>>> dependencies) as part of existing build
>>>
>>> I can't find an easy way to do so for the following reasons (please
>>> correct me if I'm wrong) :
>>>
>>> 1) Client and Server caches can't coexists on the same JVM. Does it mean
>>> one has to start a separate process for server ?
>>>
>>> 2) It seems that CQs can't be executed on server cache.
>>>
>>> 3) No easy way to create (non-local) regions programmatically on a
>>> remote server cache (see (1)). They should be visible for queries / CQs.
>>> Preferably avoiding starting a locator (another process) ?
>>>
>>> Please point me to some examples if they exist.
>>>
>>> Current version of product is gemfire v7 but looking at the code of
>>> geode concepts remained the same.
>>>
>>> Thanks,
>>> Andrei.
>>>
>>>
>>>
>>>
>>
>>
>
>
> --
> -John
> 503-504-8657 <(503)%20504-8657>
> john.blum10101 (skype)
>
>
>
