Re: Non-persistent parallel gateway sender on non-persistent region (collocated with persistent region)
Hi Barrett, We have suspected that this could be bug, and it is great that you confirmed it and this quickly created a fix. Thank you very much for your effort! BRs, Jakov On 16. 07. 2020. 21:39, Barrett Oglesby wrote: I think you've found a bug in this scenario. The ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR method currently compares the data policy of the input region's leader region with the sender's persistence policy. It assumes the input region and the leader region have the same data policy. In this scenario, that is not the case. The input region is 'part_a' which is not persistent, and the leader region is '_part_hidden' which is persistent. The sender is 'sender' which is not persistent. So, instead of comparing the data policy of 'part_a' to the sender which would succeed since they are both not persistent, it compares the data policy of '_part_hidden' to the sender which fails since one is persistent and one is not. I made a small change to ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR to address the issue. I'll file a JIRA and run CI on it to see if it is a valid change. I'll also add a test for this scenario. From: Jakov Varenina Sent: Friday, July 10, 2020 3:34 AM To: dev@geode.apache.org Subject: Re: Non-persistent parallel gateway sender on non-persistent region (collocated with persistent region) Hi devs, just a kind reminder. We would be really grateful if you could take look at question in below mail. BRs, Jakov On 06. 07. 2020. 15:50, Jakov Varenina wrote: Hi all, We are trying to setup non-persistent parallel gateway sender (‘sender’) on a non-persistent partitioned region (‘part_a’). This works OK. But when this same region ‘part_a’ is colocated with another persistent region (‘_part_hidden’), Geode throws an exception: Exception in thread "main" org.apache.geode.internal.cache.wan.GatewaySenderException: Non persistent gateway sender sender can not be attached to persistent region /_part_hidden at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:461) at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:451) at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderEventProcessor.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderEventProcessor.java:191) at org.apache.geode.internal.cache.wan.parallel.ConcurrentParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ConcurrentParallelGatewaySenderQueue.java:177) at org.apache.geode.internal.cache.PartitionedRegion.postCreateRegion(PartitionedRegion.java:1174) at org.apache.geode.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:3010) at org.apache.geode.internal.cache.GemFireCacheImpl.basicCreateRegion(GemFireCacheImpl.java:2869) at org.apache.geode.internal.cache.xmlcache.RegionCreation.createRoot(RegionCreation.java:237) at org.apache.geode.internal.cache.xmlcache.CacheCreation.initializeRegions(CacheCreation.java:658) at org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:592) at org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:338) at org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4081) at org.apache.geode.internal.cache.GemFireCacheImpl.initializeDeclarativeCache(GemFireCacheImpl.java:1535) at org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1374) at org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:191) at org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:158) at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:142) at org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52) at org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:894) at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:809) at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:739) at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:256) This is cache.xml used: https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgeode.apache.org%2Fschema%2Fcache&data=02%7C01%7Cboglesby%40vmware.com%7C6b93b4396d44490cb36c08d824bce120%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637299740942592531&sdata=sJIqN%2BcbftenSAKI3SDiZSgQiuLrq98UrdZbASDo3gY%3D&reserved=0"; xml
Re: Non-persistent parallel gateway sender on non-persistent region (collocated with persistent region)
I think you've found a bug in this scenario. The ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR method currently compares the data policy of the input region's leader region with the sender's persistence policy. It assumes the input region and the leader region have the same data policy. In this scenario, that is not the case. The input region is 'part_a' which is not persistent, and the leader region is '_part_hidden' which is persistent. The sender is 'sender' which is not persistent. So, instead of comparing the data policy of 'part_a' to the sender which would succeed since they are both not persistent, it compares the data policy of '_part_hidden' to the sender which fails since one is persistent and one is not. I made a small change to ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR to address the issue. I'll file a JIRA and run CI on it to see if it is a valid change. I'll also add a test for this scenario. From: Jakov Varenina Sent: Friday, July 10, 2020 3:34 AM To: dev@geode.apache.org Subject: Re: Non-persistent parallel gateway sender on non-persistent region (collocated with persistent region) Hi devs, just a kind reminder. We would be really grateful if you could take look at question in below mail. BRs, Jakov On 06. 07. 2020. 15:50, Jakov Varenina wrote: > Hi all, > > > We are trying to setup non-persistent parallel gateway sender > (‘sender’) on a non-persistent partitioned region (‘part_a’). This > works OK. > But when this same region ‘part_a’ is colocated with another > persistent region (‘_part_hidden’), Geode throws an exception: > > Exception in thread "main" > org.apache.geode.internal.cache.wan.GatewaySenderException: Non > persistent gateway sender sender can not be attached to persistent > region /_part_hidden > at > org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:461) > at > org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:451) > at > org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderEventProcessor.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderEventProcessor.java:191) > at > org.apache.geode.internal.cache.wan.parallel.ConcurrentParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ConcurrentParallelGatewaySenderQueue.java:177) > at > org.apache.geode.internal.cache.PartitionedRegion.postCreateRegion(PartitionedRegion.java:1174) > at > org.apache.geode.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:3010) > at > org.apache.geode.internal.cache.GemFireCacheImpl.basicCreateRegion(GemFireCacheImpl.java:2869) > at > org.apache.geode.internal.cache.xmlcache.RegionCreation.createRoot(RegionCreation.java:237) > at > org.apache.geode.internal.cache.xmlcache.CacheCreation.initializeRegions(CacheCreation.java:658) > at > org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:592) > at > org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:338) > at > org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4081) > at > org.apache.geode.internal.cache.GemFireCacheImpl.initializeDeclarativeCache(GemFireCacheImpl.java:1535) > at > org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1374) > at > org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:191) > at > org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:158) > at > org.apache.geode.cache.CacheFactory.create(CacheFactory.java:142) > at > org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52) > at > org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:894) > at > org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:809) > at > org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:739) > at > org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:256) > > > This is cache.xml used: > > xmlns="https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgeode.apache.org%2Fschema%2Fcache&data=02%7C01%7Cboglesby%40vmware.com%7C6b93b4396d44490cb36c08d824bce120%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637299740942592531&sdata=sJIqN%2BcbftenSAKI3
Re: Non-persistent parallel gateway sender on non-persistent region (collocated with persistent region)
Hi devs, just a kind reminder. We would be really grateful if you could take look at question in below mail. BRs, Jakov On 06. 07. 2020. 15:50, Jakov Varenina wrote: Hi all, We are trying to setup non-persistent parallel gateway sender (‘sender’) on a non-persistent partitioned region (‘part_a’). This works OK. But when this same region ‘part_a’ is colocated with another persistent region (‘_part_hidden’), Geode throws an exception: Exception in thread "main" org.apache.geode.internal.cache.wan.GatewaySenderException: Non persistent gateway sender sender can not be attached to persistent region /_part_hidden at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:461) at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:451) at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderEventProcessor.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderEventProcessor.java:191) at org.apache.geode.internal.cache.wan.parallel.ConcurrentParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ConcurrentParallelGatewaySenderQueue.java:177) at org.apache.geode.internal.cache.PartitionedRegion.postCreateRegion(PartitionedRegion.java:1174) at org.apache.geode.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:3010) at org.apache.geode.internal.cache.GemFireCacheImpl.basicCreateRegion(GemFireCacheImpl.java:2869) at org.apache.geode.internal.cache.xmlcache.RegionCreation.createRoot(RegionCreation.java:237) at org.apache.geode.internal.cache.xmlcache.CacheCreation.initializeRegions(CacheCreation.java:658) at org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:592) at org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:338) at org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4081) at org.apache.geode.internal.cache.GemFireCacheImpl.initializeDeclarativeCache(GemFireCacheImpl.java:1535) at org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1374) at org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:191) at org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:158) at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:142) at org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52) at org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:894) at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:809) at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:739) at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:256) This is cache.xml used: xmlns="http://geode.apache.org/schema/cache"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://geode.apache.org/schema/cache http://geode.apache.org/schema/cache/cache-1.0.xsd"; version="1.0" copy-on-read="true"> org.apache.geode.cache.util.StringPrefixPartitionResolver redundant-copies="1"> org.apache.geode.cache.util.StringPrefixPartitionResolver redundant-copies="1"> org.apache.geode.cache.util.StringPrefixPartitionResolver There is nothing explicitly said about this in documentation, and It is not clear why this is not allowed. Non-persistent parallel gateway sender is attached only to non-persistent region ‘part_a’ (and not to persistent region ‘_part_hidden’) . Why is this not allowed by Geode? Is there any way around this issue? Geode version: 1.12, 1.11 BRs, Jakov
Non-persistent parallel gateway sender on non-persistent region (collocated with persistent region)
Hi all, We are trying to setup non-persistent parallel gateway sender (‘sender’) on a non-persistent partitioned region (‘part_a’). This works OK. But when this same region ‘part_a’ is colocated with another persistent region (‘_part_hidden’), Geode throws an exception: Exception in thread "main" org.apache.geode.internal.cache.wan.GatewaySenderException: Non persistent gateway sender sender can not be attached to persistent region /_part_hidden at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:461) at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderQueue.java:451) at org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderEventProcessor.addShadowPartitionedRegionForUserPR(ParallelGatewaySenderEventProcessor.java:191) at org.apache.geode.internal.cache.wan.parallel.ConcurrentParallelGatewaySenderQueue.addShadowPartitionedRegionForUserPR(ConcurrentParallelGatewaySenderQueue.java:177) at org.apache.geode.internal.cache.PartitionedRegion.postCreateRegion(PartitionedRegion.java:1174) at org.apache.geode.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:3010) at org.apache.geode.internal.cache.GemFireCacheImpl.basicCreateRegion(GemFireCacheImpl.java:2869) at org.apache.geode.internal.cache.xmlcache.RegionCreation.createRoot(RegionCreation.java:237) at org.apache.geode.internal.cache.xmlcache.CacheCreation.initializeRegions(CacheCreation.java:658) at org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:592) at org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:338) at org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4081) at org.apache.geode.internal.cache.GemFireCacheImpl.initializeDeclarativeCache(GemFireCacheImpl.java:1535) at org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1374) at org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:191) at org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:158) at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:142) at org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52) at org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:894) at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:809) at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:739) at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:256) This is cache.xml used: xmlns="http://geode.apache.org/schema/cache"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://geode.apache.org/schema/cache http://geode.apache.org/schema/cache/cache-1.0.xsd"; version="1.0" copy-on-read="true"> org.apache.geode.cache.util.StringPrefixPartitionResolver redundant-copies="1"> org.apache.geode.cache.util.StringPrefixPartitionResolver redundant-copies="1"> org.apache.geode.cache.util.StringPrefixPartitionResolver There is nothing explicitly said about this in documentation, and It is not clear why this is not allowed. Non-persistent parallel gateway sender is attached only to non-persistent region ‘part_a’ (and not to persistent region ‘_part_hidden’) . Why is this not allowed by Geode? Is there any way around this issue? Geode version: 1.12, 1.11 BRs, Jakov
