I believe the issue is really C++’s lack of reflection. In the Java client the resolution of a PDX type from another system is done by the class name embedded in the type metadata. Reflection is used to instantiate that class and all is well. On the C++ side we obviously don’t have reflection. The C++ lacks the ability to make a PDX class name to a C++ class factory.
So what happens when you register that PDX type on both sides of the WAN is that they both register a handler for the type in with their own system ID and there is no mapping that would account for the type from the other system ID. I don’t think it would be too hard to resolve. PdxSerializable already has a method to get the class name of the C++ class. As long as we had a mapping based on that it should work. I would also double check that the C++ type maps keep track of the fully qualified ID which includes the system ID, I am not sure that is currently the case. Its been so long since I have stuck my nose in this area of the client, it is possible it is attempting to do this but a bug exists too. Is there any chance you could produce an integration test based on your findings using the framework under cppcache/integration/test. Currently it doesn’t handle WAN but it shouldn’t be too hard to add that to the Cluster class and connect two Clusters via WAN to test this scenario. -Jake On Jun 30, 2020, at 2:45 AM, Mateusz Rys <[email protected]<mailto:[email protected]>> wrote: Hi Anil, Thanks for the response. You are of course right, this isn't connected to CQ. Yesterday we implemented CacheListener and the behavior is the same. Unfortunately, we do have unique distributed ids set in gemfire.properties. #on geodeA mcast-port=0 locators=GeodeA[10334] distributed-system-id=1 remote-locators=GeodeB[10334] #on geodeB mcast-port=0 locators=GeodeB[10334] distributed-system-id=2 remote-locators=GeodeA[10334] Same IDs are used when creating gateway senders. We also noticed that on both clients we have "Deserialized distributed member Id 1" output, even after we changed IDs in gemfire.properties to 50 and 60 for better visualization. [debug 2020/06/28 16:44:35.904586 CEST geodeA:3628 140700250048512] Deserializing distributed member Id [debug 2020/06/28 16:44:35.904675 CEST geodeA:3628 140700250048512] SerializationRegistry::deserialize typeId = -1 dsCode = 1 [debug 2020/06/28 16:44:35.904815 CEST geodeA:3628 140700250048512] SerializationRegistry::deserialize typeId = -1 dsCode = 87 [debug 2020/06/28 16:44:35.905000 CEST geodeA:3628 140700250048512] SerializationRegistry::deserialize typeId = -1 dsCode = 87 [debug 2020/06/28 16:44:35.905108 CEST geodeA:3628 140700250048512] SerializationRegistry::deserialize typeId = -1 dsCode = 87 [debug 2020/06/28 16:44:35.905209 CEST geodeA:3628 140700250048512] SerializationRegistry::deserialize typeId = -1 dsCode = 87 [debug 2020/06/28 16:44:35.905308 CEST geodeA:3628 140700250048512] ClientProxyMembershipID::readVersion ordinal = 110 [debug 2020/06/28 16:44:35.905466 CEST geodeA:3628 140700250048512] ClientProxyMembershipID::writeVersion ordinal = 45 [debug 2020/06/28 16:44:35.905573 CEST geodeA:3628 140700250048512] GethashKey :192:168:56:201:41001:server::1 client id: 192.168.56.201(3371:loner):2::server [debug 2020/06/28 16:44:35.905719 CEST geodeA:3628 140700250048512] Adding a new member to the member list maintained for version stamps member Ids. HashKey: :192:168:56:201:41001:server::1 MemberCounter: 1 [debug 2020/06/28 16:44:35.905867 CEST geodeA:3628 140700250048512] Deserialized distributed member Id 1 This would suggest that there is another place to provide IDs on the client, but I fail to see anything relevant in the API. Do you think this can be the problem? Regards Mateusz pon., 29 cze 2020 o 21:04 Anilkumar Gingade <[email protected]<mailto:[email protected]>> napisał(a): Hi Mateusz, The issue is not related to CQ functionality, its related to serialization and de-serialization of the PDX type. As you can see from documentation, you need to configure distributed Ids in each cluster to get the PDX working across the WAN sites. https://geode.apache.org/docs/guide/110/developing/data_serialization/use_pdx_high_level_steps.html Have you configured the cluster sites with unique distributed ids? -Anil. From: Mateusz Rys <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, June 29, 2020 at 2:26 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: CQ Event Notification in Multisite (WAN) Hi Geode Users, Together with my friends I'm trying to setup a simple Native Client (C++) and Server configuration. We decided to use Continuous Query (CQ) and overwrite onEvent() method to have an easy way to be informed about the updates in the database. So far so good, everything works as expected. But the main reason we are doing all of this is to have two or more such setups - all connected using Geode Multisite (WAN). We successfully created Multisite configuration, we can see (with gfsh commands) that if something is written to SiteA, it is being replicated to SiteB. After reading the documentation, it also became obvious that we must provide a way to serialize objects before they are sent over the network. We decided that the simplest way should be to inherit from PdxSerializable class. Unfortunately, we ran into a problem that we can't get rid of: [error 2020/06/25 16:44:36.344952 CEST geodeA:3604 139864301352704] Exception while receiving subscription event for endpoint geodeB:40404:: apache::geode::client::IllegalStateException: Unregistered type in deserialization The exception is visible if the event comes from a remote (SiteB) source. If we update the database locally on SiteA we have normal onEvent() invocation in ClientA, but the exception is present on ClientB connected to SiteB. We register our type using cache.getTypeRegistry().registerPdxType(Order::createDeserializable); We tried to have only the first client to register the type, only the second client, or both of them, but that didn't seem to change anything. We also tried to toy with setPdxReadSerialized(true) but to no avail. Have you met this exception before? Do you have any CQ+Multisite implementation examples? Thank you Regards Mateusz
