subject:"\[jira\] \[Commented\] \(GEODE\-8029\) java.lang.IllegalArgumentException\: Too large \(805306401 expected elements with load factor 0.75\)"

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151053#comment-17151053
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos merged pull request #5338:
URL: https://github.com/apache/geode/pull/5338


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-03 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151054#comment-17151054
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit acf14743915d5e8db7adc880e98287209f7299a5 in geode's branch 
refs/heads/support/1.13 from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=acf1474 ]

GEODE-8029: Allow OplogEntryIdSet to Overflow (#5329) (#5338)

Do not delete drf files during member startup as that should be only
done by the compactor thread. Instead, allow the OplogEntryIdSet to
grow over the default capacity and log a warning message instructing
the user to manually compact the disk-stores.

- Added unit tests.
- Replaced usages of 'junit.Assert' by 'assertj'.
- Modified DiskStoreImpl.deadRecordCount to return long instead of int.
- Added internal overflow implementation to the OplogEntryIdSet so it can
  grow above the default limit.

(cherry picked from commit fdc440131f0d562d97f2340d2e7ba5aacf935d62)

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151051#comment-17151051
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos merged pull request #5337:
URL: https://github.com/apache/geode/pull/5337


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-03 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151052#comment-17151052
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit 4be10d6a2892cdad7f42ae32f34e0863149f342c in geode's branch 
refs/heads/support/1.12 from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=4be10d6 ]

GEODE-8029: Allow OplogEntryIdSet to Overflow (#5329) (#5337)

Do not delete drf files during member startup as that should be only
done by the compactor thread. Instead, allow the OplogEntryIdSet to
grow over the default capacity and log a warning message instructing
the user to manually compact the disk-stores.

- Added unit tests.
- Replaced usages of 'junit.Assert' by 'assertj'.
- Modified DiskStoreImpl.deadRecordCount to return long instead of int.
- Added internal overflow implementation to the OplogEntryIdSet so it can
  grow above the default limit.

(cherry picked from commit fdc440131f0d562d97f2340d2e7ba5aacf935d62)

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150479#comment-17150479
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

agingade commented on a change in pull request #5329:
URL: https://github.com/apache/geode/pull/5329#discussion_r449178722



##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -3517,33 +3518,86 @@ public String toString() {
   }
 
   /**
-   * Set of OplogEntryIds (longs). Memory is optimized by using an int[] for 
ids in the unsigned int
-   * range.
+   * Set of OplogEntryIds (longs).
+   * Memory is optimized by using an int[] for ids in the unsigned int range.
+   * By default we can't have more than 805306401 ids for a load factor of 
0.75, the internal lists
+   * are used to overcome this limit, allowing the disk-store to recover 
successfully (the internal
+   * class is **only** used during recovery to read all deleted entries).
*/
   static class OplogEntryIdSet {
-private final IntOpenHashSet ints = new IntOpenHashSet((int) INVALID_ID);
-private final LongOpenHashSet longs = new LongOpenHashSet((int) 
INVALID_ID);
+private final List allInts;
+private final List allLongs;
+private final AtomicReference currentInts;
+private final AtomicReference currentLongs;
+
+// For testing purposes only.
+@VisibleForTesting
+OplogEntryIdSet(List allInts, List 
allLongs) {
+  this.allInts = allInts;
+  this.currentInts = new AtomicReference<>(this.allInts.get(0));
+
+  this.allLongs = allLongs;
+  this.currentLongs = new AtomicReference<>(this.allLongs.get(0));
+}
+
+public OplogEntryIdSet() {
+  IntOpenHashSet intHashSet = new IntOpenHashSet((int) INVALID_ID);
+  this.allInts = new ArrayList<>();
+  this.allInts.add(intHashSet);
+  this.currentInts = new AtomicReference<>(intHashSet);
+
+  LongOpenHashSet longHashSet = new LongOpenHashSet((int) INVALID_ID);
+  this.allLongs = new ArrayList<>();
+  this.allLongs.add(longHashSet);
+  this.currentLongs = new AtomicReference<>(longHashSet);
+}
 
 public void add(long id) {
   if (id == 0) {
 throw new IllegalArgumentException();
-  } else if (id > 0 && id <= 0xL) {
-this.ints.add((int) id);
-  } else {
-this.longs.add(id);
+  }
+
+  try {
+if (id > 0 && id <= 0xL) {
+  this.currentInts.get().add((int) id);
+} else {
+  this.currentLongs.get().add(id);
+}
+  } catch (IllegalArgumentException illegalArgumentException) {
+// See GEODE-8029.
+// Too many entries on the accumulated drf files, overflow and 
continue.
+logger.warn(
+"There are too many entries within the disk-store, please execute 
an offline compaction.",
+illegalArgumentException);
+
+// Overflow to the next [Int|Long]OpenHashSet and continue.
+if (id > 0 && id <= 0xL) {

Review comment:
   Thanks for the clarification.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit s

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150276#comment-17150276
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos opened a new pull request #5338:
URL: https://github.com/apache/geode/pull/5338


   Do not delete drf files during member startup as that should be only
   done by the compactor thread. Instead, allow the OplogEntryIdSet to
   grow over the default capacity and log a warning message instructing
   the user to manually compact the disk-stores.
   
   - Added unit tests.
   - Replaced usages of 'junit.Assert' by 'assertj'.
   - Modified DiskStoreImpl.deadRecordCount to return long instead of int.
   - Added internal overflow implementation to the OplogEntryIdSet so it can
 grow above the default limit.
   
   (cherry picked from commit fdc440131f0d562d97f2340d2e7ba5aacf935d62)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150268#comment-17150268
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos opened a new pull request #5337:
URL: https://github.com/apache/geode/pull/5337


   Do not delete drf files during member startup as that should be only
   done by the compactor thread. Instead, allow the OplogEntryIdSet to
   grow over the default capacity and log a warning message instructing
   the user to manually compact the disk-stores.
   
   - Added unit tests.
   - Replaced usages of 'junit.Assert' by 'assertj'.
   - Modified DiskStoreImpl.deadRecordCount to return long instead of int.
   - Added internal overflow implementation to the OplogEntryIdSet so it can
 grow above the default limit.
   
   (cherry picked from commit fdc440131f0d562d97f2340d2e7ba5aacf935d62)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150240#comment-17150240
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit fdc440131f0d562d97f2340d2e7ba5aacf935d62 in geode's branch 
refs/heads/develop from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=fdc4401 ]

GEODE-8029: Allow OplogEntryIdSet to Overflow (#5329)

Do not delete drf files during member startup as that should be only
done by the compactor thread. Instead, allow the OplogEntryIdSet to
grow over the default capacity and log a warning message instructing
the user to manually compact the disk-stores.

- Added unit tests.
- Replaced usages of 'junit.Assert' by 'assertj'.
- Modified DiskStoreImpl.deadRecordCount to return long instead of int.
- Added internal overflow implementation to the OplogEntryIdSet so it can
  grow above the default limit.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLaunche

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150239#comment-17150239
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos merged pull request #5329:
URL: https://github.com/apache/geode/pull/5329


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distribute

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150180#comment-17150180
 ] 

Juan Ramos commented on GEODE-8029:
---

{quote}
Juan, I deleted the content inside the diskstore and restarted the servers.
{quote}

Thanks for the update [~jagan23527001], we have a potential fix for the issue 
that will be merged into {{develop}} shortly.
Once that's done, I'll back port the changes to the {{support/1.13}} branch, 
which is what we'll use to ship the {{1.13}} release in the coming weeks.


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150172#comment-17150172
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5329:
URL: https://github.com/apache/geode/pull/5329#discussion_r448907611



##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -3517,33 +3518,86 @@ public String toString() {
   }
 
   /**
-   * Set of OplogEntryIds (longs). Memory is optimized by using an int[] for 
ids in the unsigned int
-   * range.
+   * Set of OplogEntryIds (longs).
+   * Memory is optimized by using an int[] for ids in the unsigned int range.
+   * By default we can't have more than 805306401 ids for a load factor of 
0.75, the internal lists
+   * are used to overcome this limit, allowing the disk-store to recover 
successfully (the internal
+   * class is **only** used during recovery to read all deleted entries).
*/
   static class OplogEntryIdSet {
-private final IntOpenHashSet ints = new IntOpenHashSet((int) INVALID_ID);
-private final LongOpenHashSet longs = new LongOpenHashSet((int) 
INVALID_ID);
+private final List allInts;
+private final List allLongs;
+private final AtomicReference currentInts;
+private final AtomicReference currentLongs;
+
+// For testing purposes only.
+@VisibleForTesting
+OplogEntryIdSet(List allInts, List 
allLongs) {
+  this.allInts = allInts;
+  this.currentInts = new AtomicReference<>(this.allInts.get(0));
+
+  this.allLongs = allLongs;
+  this.currentLongs = new AtomicReference<>(this.allLongs.get(0));
+}
+
+public OplogEntryIdSet() {
+  IntOpenHashSet intHashSet = new IntOpenHashSet((int) INVALID_ID);
+  this.allInts = new ArrayList<>();
+  this.allInts.add(intHashSet);
+  this.currentInts = new AtomicReference<>(intHashSet);
+
+  LongOpenHashSet longHashSet = new LongOpenHashSet((int) INVALID_ID);
+  this.allLongs = new ArrayList<>();
+  this.allLongs.add(longHashSet);
+  this.currentLongs = new AtomicReference<>(longHashSet);
+}
 
 public void add(long id) {
   if (id == 0) {
 throw new IllegalArgumentException();
-  } else if (id > 0 && id <= 0xL) {
-this.ints.add((int) id);
-  } else {
-this.longs.add(id);
+  }
+
+  try {
+if (id > 0 && id <= 0xL) {
+  this.currentInts.get().add((int) id);
+} else {
+  this.currentLongs.get().add(id);
+}
+  } catch (IllegalArgumentException illegalArgumentException) {
+// See GEODE-8029.
+// Too many entries on the accumulated drf files, overflow and 
continue.
+logger.warn(
+"There are too many entries within the disk-store, please execute 
an offline compaction.",
+illegalArgumentException);
+
+// Overflow to the next [Int|Long]OpenHashSet and continue.
+if (id > 0 && id <= 0xL) {

Review comment:
   @agingade: I think we're good on this regard, `ids` are unique and the 
actual size of the `OplogEntryIdSet` is _**only used**_ by `offline-validate` / 
`offline-compact` commands. That is, if we ever report a bigger number for 
`deadRecordCount` (set through `OplogEntryIdSet.size()`) because there are 
duplicated `ids` (shouldn't happen), the user would know that the `disk-store` 
should be compacted anyways.
   The real problem would happen if we ever report there are not compact-able 
entries when there actually are, which is not the case here.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server w

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150162#comment-17150162
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on pull request #5329:
URL: https://github.com/apache/geode/pull/5329#issuecomment-652921469


   @pivotal-eshu 
   > If we no longer need the OplogEntryIdSet after the recovery, shall we 
clear these sets once it is done?
   
   There are no references to the `OplogEntryIdSet` instance once we finish the 
recovery, so the class itself (along with all internal structures) will 
eventually be removed by the garbage collector.
   
   > With these many entryIds in drf files, it will also take longer time to 
recover. I think Darrel's idea for removing unnecessary drf files will help in 
this case as well.
   
   Agreed, I'll create another ticket for that.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150161#comment-17150161
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5329:
URL: https://github.com/apache/geode/pull/5329#discussion_r448898320



##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -3517,33 +3518,86 @@ public String toString() {
   }
 
   /**
-   * Set of OplogEntryIds (longs). Memory is optimized by using an int[] for 
ids in the unsigned int
-   * range.
+   * Set of OplogEntryIds (longs).
+   * Memory is optimized by using an int[] for ids in the unsigned int range.
+   * By default we can't have more than 805306401 ids for a load factor of 
0.75, the internal lists
+   * are used to overcome this limit, allowing the disk-store to recover 
successfully (the internal
+   * class is **only** used during recovery to read all deleted entries).
*/
   static class OplogEntryIdSet {
-private final IntOpenHashSet ints = new IntOpenHashSet((int) INVALID_ID);
-private final LongOpenHashSet longs = new LongOpenHashSet((int) 
INVALID_ID);
+private final List allInts;
+private final List allLongs;
+private final AtomicReference currentInts;
+private final AtomicReference currentLongs;
+
+// For testing purposes only.
+@VisibleForTesting
+OplogEntryIdSet(List allInts, List 
allLongs) {
+  this.allInts = allInts;
+  this.currentInts = new AtomicReference<>(this.allInts.get(0));
+
+  this.allLongs = allLongs;
+  this.currentLongs = new AtomicReference<>(this.allLongs.get(0));
+}
+
+public OplogEntryIdSet() {
+  IntOpenHashSet intHashSet = new IntOpenHashSet((int) INVALID_ID);
+  this.allInts = new ArrayList<>();
+  this.allInts.add(intHashSet);
+  this.currentInts = new AtomicReference<>(intHashSet);
+
+  LongOpenHashSet longHashSet = new LongOpenHashSet((int) INVALID_ID);
+  this.allLongs = new ArrayList<>();
+  this.allLongs.add(longHashSet);
+  this.currentLongs = new AtomicReference<>(longHashSet);
+}
 
 public void add(long id) {
   if (id == 0) {
 throw new IllegalArgumentException();
-  } else if (id > 0 && id <= 0xL) {
-this.ints.add((int) id);
-  } else {
-this.longs.add(id);
+  }
+
+  try {
+if (id > 0 && id <= 0xL) {
+  this.currentInts.get().add((int) id);
+} else {
+  this.currentLongs.get().add(id);
+}
+  } catch (IllegalArgumentException illegalArgumentException) {
+// See GEODE-8029.
+// Too many entries on the accumulated drf files, overflow and 
continue.
+logger.warn(
+"There are too many entries within the disk-store, please execute 
an offline compaction.",

Review comment:
   👍 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.un

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150160#comment-17150160
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5329:
URL: https://github.com/apache/geode/pull/5329#discussion_r448898069



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,17 +937,11 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
+// I don't think the drf needs to be opened. It is only used during
+// recovery.
+// At some point the compacter my identify that it can be deleted.

Review comment:
   Thanks @dschneider-pivotal!. I'll fix the type and create another ticket 
for the long term solution. 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.in

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149698#comment-17149698
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

dschneider-pivotal commented on a change in pull request #5329:
URL: https://github.com/apache/geode/pull/5329#discussion_r448615239



##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -3517,33 +3518,86 @@ public String toString() {
   }
 
   /**
-   * Set of OplogEntryIds (longs). Memory is optimized by using an int[] for 
ids in the unsigned int
-   * range.
+   * Set of OplogEntryIds (longs).
+   * Memory is optimized by using an int[] for ids in the unsigned int range.
+   * By default we can't have more than 805306401 ids for a load factor of 
0.75, the internal lists
+   * are used to overcome this limit, allowing the disk-store to recover 
successfully (the internal
+   * class is **only** used during recovery to read all deleted entries).
*/
   static class OplogEntryIdSet {
-private final IntOpenHashSet ints = new IntOpenHashSet((int) INVALID_ID);
-private final LongOpenHashSet longs = new LongOpenHashSet((int) 
INVALID_ID);
+private final List allInts;
+private final List allLongs;
+private final AtomicReference currentInts;
+private final AtomicReference currentLongs;
+
+// For testing purposes only.
+@VisibleForTesting
+OplogEntryIdSet(List allInts, List 
allLongs) {
+  this.allInts = allInts;
+  this.currentInts = new AtomicReference<>(this.allInts.get(0));
+
+  this.allLongs = allLongs;
+  this.currentLongs = new AtomicReference<>(this.allLongs.get(0));
+}
+
+public OplogEntryIdSet() {
+  IntOpenHashSet intHashSet = new IntOpenHashSet((int) INVALID_ID);
+  this.allInts = new ArrayList<>();
+  this.allInts.add(intHashSet);
+  this.currentInts = new AtomicReference<>(intHashSet);
+
+  LongOpenHashSet longHashSet = new LongOpenHashSet((int) INVALID_ID);
+  this.allLongs = new ArrayList<>();
+  this.allLongs.add(longHashSet);
+  this.currentLongs = new AtomicReference<>(longHashSet);
+}
 
 public void add(long id) {
   if (id == 0) {
 throw new IllegalArgumentException();
-  } else if (id > 0 && id <= 0xL) {
-this.ints.add((int) id);
-  } else {
-this.longs.add(id);
+  }
+
+  try {
+if (id > 0 && id <= 0xL) {
+  this.currentInts.get().add((int) id);
+} else {
+  this.currentLongs.get().add(id);
+}
+  } catch (IllegalArgumentException illegalArgumentException) {
+// See GEODE-8029.
+// Too many entries on the accumulated drf files, overflow and 
continue.
+logger.warn(
+"There are too many entries within the disk-store, please execute 
an offline compaction.",
+illegalArgumentException);
+
+// Overflow to the next [Int|Long]OpenHashSet and continue.
+if (id > 0 && id <= 0xL) {

Review comment:
   I don't think so. The ids are unique and should only be deleted once 
from a drf. If this did happen it would just cause the size to be larger than 
it should be. So you could review the code that uses the size of this class.
   You could prevent this from having each add first call contains which 
shouldn't slow down the add too much.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServ

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-01 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149669#comment-17149669
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Juan, I deleted the content inside the diskstore and restarted the servers.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149663#comment-17149663
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

agingade commented on a change in pull request #5329:
URL: https://github.com/apache/geode/pull/5329#discussion_r448560371



##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -3517,33 +3518,86 @@ public String toString() {
   }
 
   /**
-   * Set of OplogEntryIds (longs). Memory is optimized by using an int[] for 
ids in the unsigned int
-   * range.
+   * Set of OplogEntryIds (longs).
+   * Memory is optimized by using an int[] for ids in the unsigned int range.
+   * By default we can't have more than 805306401 ids for a load factor of 
0.75, the internal lists
+   * are used to overcome this limit, allowing the disk-store to recover 
successfully (the internal
+   * class is **only** used during recovery to read all deleted entries).
*/
   static class OplogEntryIdSet {
-private final IntOpenHashSet ints = new IntOpenHashSet((int) INVALID_ID);
-private final LongOpenHashSet longs = new LongOpenHashSet((int) 
INVALID_ID);
+private final List allInts;
+private final List allLongs;
+private final AtomicReference currentInts;
+private final AtomicReference currentLongs;
+
+// For testing purposes only.
+@VisibleForTesting
+OplogEntryIdSet(List allInts, List 
allLongs) {
+  this.allInts = allInts;
+  this.currentInts = new AtomicReference<>(this.allInts.get(0));
+
+  this.allLongs = allLongs;
+  this.currentLongs = new AtomicReference<>(this.allLongs.get(0));
+}
+
+public OplogEntryIdSet() {
+  IntOpenHashSet intHashSet = new IntOpenHashSet((int) INVALID_ID);
+  this.allInts = new ArrayList<>();
+  this.allInts.add(intHashSet);
+  this.currentInts = new AtomicReference<>(intHashSet);
+
+  LongOpenHashSet longHashSet = new LongOpenHashSet((int) INVALID_ID);
+  this.allLongs = new ArrayList<>();
+  this.allLongs.add(longHashSet);
+  this.currentLongs = new AtomicReference<>(longHashSet);
+}
 
 public void add(long id) {
   if (id == 0) {
 throw new IllegalArgumentException();
-  } else if (id > 0 && id <= 0xL) {
-this.ints.add((int) id);
-  } else {
-this.longs.add(id);
+  }
+
+  try {
+if (id > 0 && id <= 0xL) {
+  this.currentInts.get().add((int) id);
+} else {
+  this.currentLongs.get().add(id);
+}
+  } catch (IllegalArgumentException illegalArgumentException) {
+// See GEODE-8029.
+// Too many entries on the accumulated drf files, overflow and 
continue.
+logger.warn(
+"There are too many entries within the disk-store, please execute 
an offline compaction.",

Review comment:
   "Too many entries", this is referring to deleted entries, right? In that 
case how about changing it 
   to "large number of deleted entries"?

##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -3517,33 +3518,86 @@ public String toString() {
   }
 
   /**
-   * Set of OplogEntryIds (longs). Memory is optimized by using an int[] for 
ids in the unsigned int
-   * range.
+   * Set of OplogEntryIds (longs).
+   * Memory is optimized by using an int[] for ids in the unsigned int range.
+   * By default we can't have more than 805306401 ids for a load factor of 
0.75, the internal lists
+   * are used to overcome this limit, allowing the disk-store to recover 
successfully (the internal
+   * class is **only** used during recovery to read all deleted entries).
*/
   static class OplogEntryIdSet {
-private final IntOpenHashSet ints = new IntOpenHashSet((int) INVALID_ID);
-private final LongOpenHashSet longs = new LongOpenHashSet((int) 
INVALID_ID);
+private final List allInts;
+private final List allLongs;
+private final AtomicReference currentInts;
+private final AtomicReference currentLongs;
+
+// For testing purposes only.
+@VisibleForTesting
+OplogEntryIdSet(List allInts, List 
allLongs) {
+  this.allInts = allInts;
+  this.currentInts = new AtomicReference<>(this.allInts.get(0));
+
+  this.allLongs = allLongs;
+  this.currentLongs = new AtomicReference<>(this.allLongs.get(0));
+}
+
+public OplogEntryIdSet() {
+  IntOpenHashSet intHashSet = new IntOpenHashSet((int) INVALID_ID);
+  this.allInts = new ArrayList<>();
+  this.allInts.add(intHashSet);
+  this.currentInts = new AtomicReference<>(intHashSet);
+
+  LongOpenHashSet longHashSet = new LongOpenHashSet((int) INVALID_ID);
+  this.allLongs = new ArrayList<>();
+  this.allLongs.add(longHashSet);
+  this.currentLongs = new AtomicReference<>(longHashSet);

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-07-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149611#comment-17149611
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

dschneider-pivotal commented on a change in pull request #5329:
URL: https://github.com/apache/geode/pull/5329#discussion_r448518869



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,17 +937,11 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
+// I don't think the drf needs to be opened. It is only used during
+// recovery.
+// At some point the compacter my identify that it can be deleted.

Review comment:
   change "my" to "may"





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
>

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-30 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148489#comment-17148489
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos opened a new pull request #5329:
URL: https://github.com/apache/geode/pull/5329


   Do not delete drf files during member startup as that should be only
   done by the compactor thread. Instead, allow the OplogEntryIdSet to
   grow over the default capacity and log a warning message instructing
   the user to manually compact the disk-stores.
   
   - Added unit tests.
   - Replaced usages of 'junit.Assert' by 'assertj'.
   - Modified DiskStoreImpl.deadRecordCount to return long instead of int.
   - Added internal overflow implementation to the OplogEntryIdSet so it can
 grow above the default limit.
   
   Thank you for submitting a contribution to Apache Geode.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [X] Is there a JIRA ticket associated with this PR? Is it referenced in 
the commit message?
   
   - [X] Has your PR been rebased against the latest commit within the target 
branch (typically `develop`)?
   
   - [X] Is your initial contribution a single, squashed commit?
   
   - [X] Does `gradlew build` run cleanly?
   
   - [X] Have you written or updated unit tests to verify your changes?
   
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   
   ### Note:
   Please ensure that once the PR is submitted, check Concourse for build 
issues and
   submit an update to your PR as soon as possible. If you need help, please 
send an
   email to d...@geode.apache.org.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFact

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-26 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17146413#comment-17146413
 ] 

Juan Ramos commented on GEODE-8029:
---

Hello [~jagan23527001],
This issue is proving to be quite evasive to fully solve, the fix I've checked 
in is certainly not complete and the issue might still happen under certain 
conditions, so I'm trying to understand why/how the old {{drf}} files were not 
compacted/deleted for your use case.
I will certainly understand if you answer "no" to my question but, if you still 
have a copy of the {{disk-store}} when the issue happened, would you be able to 
share it with us so we can keep troubleshooting the root cause?.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.st

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-10 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132596#comment-17132596
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit 7c1ffd528ff72b920dd606604de2a744c1728b23 in geode's branch 
refs/heads/support/1.13 from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=7c1ffd5 ]

GEODE-8029: Delete orphaned drf files (2nd try) (#5099)

This reverts the revert of the original commit and also fixes the
originally introduced issues.



The OpLog initialization now delete unused drf files to prevent the
proliferation of unused records and files within the system, which
can could cause members to fail during startup while recovering
disk-stores (especially when they are isolated for gateway-senders).

- Added distributed tests.
- Delete orphaned drf files when deleting the corresponding
  crf during recovery.

(cherry picked from commit bc0090dc93643fd4d09c79a4b0c29d883172b546)


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.creat

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-10 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132575#comment-17132575
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit bdeff9d6144c47ea687cf3b7d25f12598228 in geode's branch 
refs/heads/support/1.12 from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=bdeff9d ]

GEODE-8029: Delete orphaned drf files (2nd try) (#5099)

This reverts the revert of the original commit and also fixes the
originally introduced issues.



The OpLog initialization now delete unused drf files to prevent the
proliferation of unused records and files within the system, which
can could cause members to fail during startup while recovering
disk-stores (especially when they are isolated for gateway-senders).

- Added distributed tests.
- Delete orphaned drf files when deleting the corresponding
  crf during recovery.

(cherry picked from commit bc0090dc93643fd4d09c79a4b0c29d883172b546)


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.creat

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-09 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17129520#comment-17129520
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos merged pull request #5099:
URL: https://github.com/apache/geode/pull/5099


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-08 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17128521#comment-17128521
 ] 

Juan Ramos commented on GEODE-8029:
---

Hello [~jagan23527001],

Geode ships minor releases on a quarterly basis (see 
[here|https://cwiki.apache.org/confluence/display/GEODE/Shipping+patch+releases]
 for further details) and the release branch for version {{1.13}} has been cut 
recently, so I don't think {{1.14}} will be released until, at least, September.
That said, I'll propose on the {{dev}} list to back port this fix, if there are 
enough votes it should be included in {{1.13}} (will be released soon) as well.
Cheers.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at o

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-08 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17128513#comment-17128513
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Hi Juan,

 

When 1.14.0 Geode version will be released ?

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Fix For: 1.14.0
>
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-08 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17128503#comment-17128503
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit bc0090dc93643fd4d09c79a4b0c29d883172b546 in geode's branch 
refs/heads/develop from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=bc0090d ]

GEODE-8029: Delete orphaned drf files (2nd try) (#5099)

This reverts the revert of the original commit and also fixes the
originally introduced issues.



The OpLog initialization now delete unused drf files to prevent the
proliferation of unused records and files within the system, which
can could cause members to fail during startup while recovering
disk-stores (especially when they are isolated for gateway-senders).

- Added distributed tests.
- Delete orphaned drf files when deleting the corresponding
  crf during recovery.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.cr

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-05 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126710#comment-17126710
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5099:
URL: https://github.com/apache/geode/pull/5099#discussion_r435872866



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,21 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.

Review comment:
   @agingade 
   Thanks, I've made the requested changes already and verified that no new 
regressions have been introduced due to the modifications, please go ahead and 
approve the `PR` (if there's nothing else you want me to change) so I can merge 
it 👍.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-04 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126225#comment-17126225
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

agingade commented on a change in pull request #5099:
URL: https://github.com/apache/geode/pull/5099#discussion_r435581008



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,21 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.

Review comment:
   I am fine with the approach you have taken. Wanted to bring it up that 
there could be distinct possibility.
   Looks good to me.  





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
>

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-04 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126224#comment-17126224
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

agingade commented on a change in pull request #5099:
URL: https://github.com/apache/geode/pull/5099#discussion_r435581008



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,21 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.

Review comment:
   I am fine with the approach you have taken. Wanted to bring it up, there 
could be distinct possibility.
   Looks good to me.  





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-04 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125916#comment-17125916
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5099:
URL: https://github.com/apache/geode/pull/5099#discussion_r435237982



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,21 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.

Review comment:
   > This change comes into picture after the recovery is completed - 
"initAfterRecovery".
   The real problem seems to be during recovery. If there is a periodic 
recovery this could be fine, but if there are large number of deleted records 
during the first recovery, the issue may still arise...We may need to have some 
checks during recovery (before reading the drfs) to avoid reading large deleted 
records. Please correct me if my understanding is wrong.
   
   Your understanding is correct, however, the change are to prevent the issue 
from happening in the first place. By deleting the unused `drf` files here and 
avoid storing them on disk when not needed, users won't hit the 
`IllegalStateException` at all.
   As a side note, in order to hit the problem in the first place, the user 
would need to have more than `805306401` delete operations within the `opLog` 
files in one single run, and compaction should have not run at all, which is 
highly unlikely. Adding to that, when we record a delete operation within the 
`drf` file, we 
[check](https://github.com/apache/geode/blob/develop/geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java#L4959)
 whether the new `drf` size would be higher than the max configured size for 
the `drf` file, and roll to a new one afterwards. Considering that the `drf` 
file is 10% of the `max-oplog-size`, and that we use `10 bytes` per 
`DEL_ENTRY`, to be able to hit the problem the max `drf` file should be higher 
than `8.5GB`, meaning that the `max-oplog-size` should be configured, at least, 
to be `85GB` (with no automatic compaction at all, which is something we don't 
recommend).
   
   If you think we should add the count while reading the files instead and, 
somehow, expand the load factor when needed or something similar, let me know 
and I'll give it a try. I'm worried, though, about the performance impact this 
might have while recovering files from disk... the recovery time will suffer 
during every single startup and, considering that there's only a handful of 
scenarios on which the actual issue can happen, I believe making sure we delete 
unused files is a better approach.
   
   > Also, looking at the other part of the code where "setHasDeletes" is 
called, it looks like it is called before the deleteDRF() - there could be a 
reason for doing this.
   
   Will change this, thanks for catching it.
   
   > And there is also call to "getOplogSet().removeDrf(this);" This may be 
needed here...
   
   This is already done within the `deleteDRF()` method.
   
   ---
   
   Please let me know what you think about point 1 (add an extra check while 
recovering to make sure we are below the load-factor threshold), so I can go 
ahead and make all the changes at once.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-04 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125904#comment-17125904
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5099:
URL: https://github.com/apache/geode/pull/5099#discussion_r435237982



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,21 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.

Review comment:
   > This change comes into picture after the recovery is completed - 
"initAfterRecovery".
   The real problem seems to be during recovery. If there is a periodic 
recovery this could be fine, but if there are large number of deleted records 
during the first recovery, the issue may still arise...We may need to have some 
checks during recovery (before reading the drfs) to avoid reading large deleted 
records. Please correct me if my understanding is wrong.
   
   Your understanding is correct, however, the change are to prevent the issue 
from happening in the first place. By deleting the unused `drf` files here and 
avoid storing them on disk when not needed, users won't hit the 
`IllegalStateException` at all.
   As a side note, in order to hit the problem in the first place, the user 
would need to have more than `805306401` delete operations within the `opLog` 
files in one single run, and compaction should have not run at all, which is 
highly unlikely.
   If you think we should add the count while reading the files instead and, 
somehow, expand the load factor when needed or something similar, let me know 
and I'll give it a try. I'm worried, though, about the performance impact this 
might have while recovering files from disk... the recovery time will suffer 
during every single startup and, considering that there's only a handful of 
scenarios on which the actual issue can happen, I believe making sure we delete 
unused files is a better approach.
   
   > Also, looking at the other part of the code where "setHasDeletes" is 
called, it looks like it is called before the deleteDRF() - there could be a 
reason for doing this.
   
   Will change this, thanks for catching it.
   
   > And there is also call to "getOplogSet().removeDrf(this);" This may be 
needed here...
   
   This is already done within the `deleteDRF()` method.
   
   ---
   
   Please let me know what you think about point 1 (add an extra check while 
recovering to make sure we are below the load-factor threshold), so I can go 
ahead and make all the changes at once.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (8053064

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-06-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125276#comment-17125276
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

agingade commented on a change in pull request #5099:
URL: https://github.com/apache/geode/pull/5099#discussion_r434821063



##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,21 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.

Review comment:
   This change comes into picture after the recovery is completed - 
"initAfterRecovery".
   The real problem seems to be during recovery. If there is a periodic 
recovery this could be fine, but if there are large number of deleted records 
during the first recovery, the issue may still arise...We may need to have some 
checks during recovery (before reading the drfs) to avoid reading large deleted 
records. Please correct me if my understanding is wrong. 
   
   Also, looking at the other part of the code where "setHasDeletes" is called, 
it looks like it is called before the deleteDRF() - there could be a reason for 
doing this.
   And there is also call to "getOplogSet().removeDrf(this);" This may be 
needed here...
   





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCr

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-08 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102647#comment-17102647
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit 17f9a237047068c1ea821cd1dee0afd38606c54a in geode's branch 
refs/heads/develop from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=17f9a23 ]

Revert "GEODE-8029: Delete orphaned drf files (#5037)" (#5073)

This reverts commit be8ac497eb1ece588e9a6c299d6aab4feb192ed3.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent b

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-08 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102608#comment-17102608
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos opened a new pull request #5073:
URL: https://github.com/apache/geode/pull/5073


   This reverts commit be8ac497eb1ece588e9a6c299d6aab4feb192ed3.
   
   Thank you for submitting a contribution to Apache Geode.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [X] Is there a JIRA ticket associated with this PR? Is it referenced in 
the commit message?
   
   - [X] Has your PR been rebased against the latest commit within the target 
branch (typically `develop`)?
   
   - [X] Is your initial contribution a single, squashed commit?
   
   - [X] Does `gradlew build` run cleanly?
   
   - [X] Have you written or updated unit tests to verify your changes?
   
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   
   ### Note:
   Please ensure that once the PR is submitted, check Concourse for build 
issues and
   submit an update to your PR as soon as possible. If you need help, please 
send an
   email to d...@geode.apache.org.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apach

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-08 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102556#comment-17102556
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos opened a new pull request #5072:
URL: https://github.com/apache/geode/pull/5072


   This reverts commit be8ac497eb1ece588e9a6c299d6aab4feb192ed3.
   
   Thank you for submitting a contribution to Apache Geode.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [X] Is there a JIRA ticket associated with this PR? Is it referenced in 
the commit message?
   
   - [X] Has your PR been rebased against the latest commit within the target 
branch (typically `develop`)?
   
   - [X] Is your initial contribution a single, squashed commit?
   
   - [X] Does `gradlew build` run cleanly?
   
   - [X] Have you written or updated unit tests to verify your changes?
   
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   
   ### Note:
   Please ensure that once the PR is submitted, check Concourse for build 
issues and
   submit an update to your PR as soon as possible. If you need help, please 
send an
   email to d...@geode.apache.org.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apach

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-08 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102551#comment-17102551
 ] 

Juan Ramos commented on GEODE-8029:
---

The fix introduced a regression, shown by our internal testing framework, for 
which several tests fail with the following stack trace:

{noformat}
Caused by: org.apache.geode.cache.DiskAccessException: For DiskStore: X: 
Failed to read file during recovery from /X.drf, caused by 
java.io.FileNotFoundException: /.drf (No such file or directory)
  at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1561)
  at 
org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:462)
  at 
org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:379)
  at 
org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2074)
  at 
org.apache.geode.internal.cache.DiskStoreImpl.initializeOwner(DiskStoreImpl.java:655)
  at 
org.apache.geode.internal.cache.DiskRegion.initializeOwner(DiskRegion.java:239)
  at 
org.apache.geode.internal.cache.DistributedRegion.initialize(DistributedRegion.java:1071)
  at 
org.apache.geode.internal.cache.BucketRegion.initialize(BucketRegion.java:259)
  at 
org.apache.geode.internal.cache.LocalRegion.createSubregion(LocalRegion.java:981)
  at 
org.apache.geode.internal.cache.PartitionedRegionDataStore.createBucketRegion(PartitionedRegionDataStore.java:785)
  at 
org.apache.geode.internal.cache.PartitionedRegionDataStore.grabFreeBucket(PartitionedRegionDataStore.java:460)
  at 
org.apache.geode.internal.cache.PartitionedRegionDataStore.grabFreeBucketRecursively(PartitionedRegionDataStore.java:319)
  at 
org.apache.geode.internal.cache.PartitionedRegionDataStore.grabBucket(PartitionedRegionDataStore.java:2896)
  at 
org.apache.geode.internal.cache.ProxyBucketRegion.recoverFromDisk(ProxyBucketRegion.java:450)
  at 
org.apache.geode.internal.cache.ProxyBucketRegion.recoverFromDiskRecursively(ProxyBucketRegion.java:406)
  at 
org.apache.geode.internal.cache.PRHARedundancyProvider$2.run2(PRHARedundancyProvider.java:1640)
  at 
org.apache.geode.internal.cache.partitioned.RecoveryRunnable.run(RecoveryRunnable.java:60)
  at 
org.apache.geode.internal.cache.PRHARedundancyProvider$2.run(PRHARedundancyProvider.java:1630)
  at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.FileNotFoundException: /.drf (No such file or directory)
  at java.base/java.io.FileInputStream.open0(Native Method)
  at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
  at java.base/java.io.FileInputStream.(FileInputStream.java:157)
  at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1477)
  ... 18 more
{noformat}

Re-opening the ticket to revert the original commit and keep working on a 
definite fix for the issue.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.jav

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-05 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099670#comment-17099670
 ] 

ASF subversion and git services commented on GEODE-8029:


Commit be8ac497eb1ece588e9a6c299d6aab4feb192ed3 in geode's branch 
refs/heads/develop from Juan José Ramos
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=be8ac49 ]

GEODE-8029: Delete orphaned drf files (#5037)

The OpLog initialization now delete unused drf files to prevent the
proliferation of unused records and files within the system, which
can could cause members to fail during startup while recovering
disk-stores (especially when they are isolated for gateway-senders).

- Added distributed tests.
- Delete orphaned drf files when deleting the corresponding
  crf during recovery.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createC

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098502#comment-17098502
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5037:
URL: https://github.com/apache/geode/pull/5037#discussion_r419131931



##
File path: 
geode-wan/src/distributedTest/java/org/apache/geode/internal/cache/wan/PersistentGatewaySenderWithIsolatedDiskStoreDistributedTest.java
##
@@ -0,0 +1,346 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more 
contributor license
+ * agreements. See the NOTICE file distributed with this work for additional 
information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache 
License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the 
License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software 
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 
KIND, either express
+ * or implied. See the License for the specific language governing permissions 
and limitations under
+ * the License.
+ */
+package org.apache.geode.internal.cache.wan;
+
+import static 
org.apache.geode.distributed.ConfigurationProperties.DISTRIBUTED_SYSTEM_ID;
+import static org.apache.geode.distributed.ConfigurationProperties.LOCATORS;
+import static org.apache.geode.distributed.ConfigurationProperties.MCAST_PORT;
+import static 
org.apache.geode.distributed.ConfigurationProperties.REMOTE_LOCATORS;
+import static 
org.apache.geode.distributed.ConfigurationProperties.START_LOCATOR;
+import static 
org.apache.geode.internal.AvailablePortHelper.getRandomAvailableTCPPortsForDUnitSite;
+import static org.apache.geode.test.awaitility.GeodeAwaitility.await;
+import static org.apache.geode.test.dunit.VM.getVM;
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.File;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Properties;
+import java.util.Set;
+import java.util.stream.IntStream;
+
+import junitparams.JUnitParamsRunner;
+import junitparams.Parameters;
+import junitparams.naming.TestCaseName;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+import org.apache.geode.cache.DiskStore;
+import org.apache.geode.cache.DiskStoreFactory;
+import org.apache.geode.cache.Region;
+import org.apache.geode.cache.RegionShortcut;
+import org.apache.geode.cache.wan.GatewayReceiverFactory;
+import org.apache.geode.cache.wan.GatewaySender;
+import org.apache.geode.cache.wan.GatewaySenderFactory;
+import org.apache.geode.distributed.internal.InternalDistributedSystem;
+import org.apache.geode.internal.cache.DiskStoreImpl;
+import org.apache.geode.internal.cache.RegionQueue;
+import org.apache.geode.internal.cache.persistence.OplogType;
+import org.apache.geode.test.dunit.VM;
+import org.apache.geode.test.dunit.rules.CacheRule;
+import org.apache.geode.test.dunit.rules.DistributedDiskDirRule;
+import org.apache.geode.test.dunit.rules.DistributedRule;
+import org.apache.geode.test.junit.rules.serializable.SerializableTestName;
+
+/**
+ * Tests to verify WAN functionality when the gateway-sender(s) have isolated, 
non-shared with
+ * other region(s), disk-store(s).
+ */
+@RunWith(JUnitParamsRunner.class)
+public class PersistentGatewaySenderWithIsolatedDiskStoreDistributedTest 
implements Serializable {
+  private static final String REGION_NAME = "TestRegion";
+  private static final String DISK_STORE_ID = "testDisk";
+  private static final String GATEWAY_SENDER_ID = "testSender";
+  private static final String TEST_CASE_NAME = "[{index}] 
{method}(RegionType:{0}, Parallel:{1})";
+  private int site1Port, site2Port;
+  private VM serverCluster1, serverCluster2;
+
+  @Rule
+  public CacheRule cacheRule = new CacheRule();
+
+  @Rule
+  public DistributedRule distributedRule = new DistributedRule();
+
+  @Rule
+  public SerializableTestName testName = new SerializableTestName();
+
+  @Rule
+  public DistributedDiskDirRule distributedDiskDirRule = new 
DistributedDiskDirRule();
+
+  private Properties createLocatorConfiguration(int distributedSystemId, int 
localLocatorPort,
+  int remoteLocatorPort) {
+Properties config = new Properties();
+config.setProperty(MCAST_PORT, "0");
+config.setProperty(DISTRIBUTED_SYSTEM_ID, 
String.valueOf(distributedSystemId));
+config.setProperty(LOCATORS, "localhost[" + localLocatorPort + ']');
+config.setProperty(REMOTE_LOCATORS, "localhost[" + remoteLocatorPort + 
']');
+config.setProperty(START_LOCATOR,
+"localhost[" + localLocatorPort + 
"],server=true,peer=true,hostname-for-clients=localhost");
+
+return con

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098500#comment-17098500
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos commented on a change in pull request #5037:
URL: https://github.com/apache/geode/pull/5037#discussion_r419131852



##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -4279,6 +4279,28 @@ public static void validate(String name, File[] dirs) 
throws Exception {
 }
   }
 
+  /**
+   * Validates the disk-store in offline mode, and returns the validated 
DiskStore instance.
+   * This method is an "almost exact copy" of {@link 
DiskStoreImpl#validate(String, File[])}.

Review comment:
   Sounds good.

##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,20 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.
+// The drf file needs to be deleted, specially when the disk-store is 
*only* used by

Review comment:
   👍 

##
File path: 
geode-wan/src/distributedTest/java/org/apache/geode/internal/cache/wan/PersistentGatewaySenderWithIsolatedDiskStoreDistributedTest.java
##
@@ -0,0 +1,346 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more 
contributor license
+ * agreements. See the NOTICE file distributed with this work for additional 
information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache 
License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the 
License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software 
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 
KIND, either express
+ * or implied. See the License for the specific language governing permissions 
and limitations under
+ * the License.
+ */
+package org.apache.geode.internal.cache.wan;
+
+import static 
org.apache.geode.distributed.ConfigurationProperties.DISTRIBUTED_SYSTEM_ID;
+import static org.apache.geode.distributed.ConfigurationProperties.LOCATORS;
+import static org.apache.geode.distributed.ConfigurationProperties.MCAST_PORT;
+import static 
org.apache.geode.distributed.ConfigurationProperties.REMOTE_LOCATORS;
+import static 
org.apache.geode.distributed.ConfigurationProperties.START_LOCATOR;
+import static 
org.apache.geode.internal.AvailablePortHelper.getRandomAvailableTCPPortsForDUnitSite;
+import static org.apache.geode.test.awaitility.GeodeAwaitility.await;
+import static org.apache.geode.test.dunit.VM.getVM;
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.File;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Properties;
+import java.util.Set;
+import java.util.stream.IntStream;
+
+import junitparams.JUnitParamsRunner;
+import junitparams.Parameters;
+import junitparams.naming.TestCaseName;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+import org.apache.geode.cache.DiskStore;
+import org.apache.geode.cache.DiskStoreFactory;
+import org.apache.geode.cache.Region;
+import org.apache.geode.cache.RegionShortcut;
+import org.apache.geode.cache.wan.GatewayReceiverFactory;
+import org.apache.geode.cache.wan.GatewaySender;
+import org.apache.geode.cache.wan.GatewaySenderFactory;
+import org.apache.geode.distributed.internal.InternalDistributedSystem;
+import org.apache.geode.internal.cache.DiskStoreImpl;
+import org.apache.geode.internal.cache.RegionQueue;
+import org.apache.geode.internal.cache.persistence.OplogType;
+import org.apache.geode.test.dunit.VM;
+import org.apache.geode.test.dunit.rules.CacheRule;
+import org.apache.geode.test.dunit.rules.DistributedDiskDirRule;
+import org.apache.geode.test.dunit.rules.DistributedRule;
+import org.apache.geode.test.junit.rules.serializable.SerializableTestName;
+
+/**
+ * Tests to verify WAN functionality when the gateway-sender(s) have isolated, 
non-shared with
+ * other region(s), disk-store(s).
+ */
+@RunWith(JUnitParamsRunner.class)
+public class PersistentGatewaySenderWithIsolatedDiskStoreDistributedTest 
implements Serializable {
+  private static final String REGION_NAME = "TestRegion";
+  private static final String DISK_STORE_ID = "testDisk";
+  private static final St

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097529#comment-17097529
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

DonalEvans commented on a change in pull request #5037:
URL: https://github.com/apache/geode/pull/5037#discussion_r418620200



##
File path: 
geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
##
@@ -4279,6 +4279,28 @@ public static void validate(String name, File[] dirs) 
throws Exception {
 }
   }
 
+  /**
+   * Validates the disk-store in offline mode, and returns the validated 
DiskStore instance.
+   * This method is an "almost exact copy" of {@link 
DiskStoreImpl#validate(String, File[])}.

Review comment:
   Instead of having two almost identical methods, might it be better to 
have one call the other to prevent code duplication? Or maybe mark this one as 
a test method for now?

##
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##
@@ -937,14 +937,20 @@ void initAfterRecovery(boolean offline) {
 // this.crf.raf.seek(this.crf.currSize);
   } else if (!offline) {
 // drf exists but crf has been deleted (because it was empty).
-// I don't think the drf needs to be opened. It is only used during
-// recovery.
-// At some point the compacter my identify that it can be deleted.
 this.crf.RAFClosed = true;
 deleteCRF();
+
+// See GEODE-8029.
+// The drf file needs to be deleted, specially when the disk-store is 
*only* used by

Review comment:
   This should be "especially".

##
File path: 
geode-wan/src/distributedTest/java/org/apache/geode/internal/cache/wan/PersistentGatewaySenderWithIsolatedDiskStoreDistributedTest.java
##
@@ -0,0 +1,346 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more 
contributor license
+ * agreements. See the NOTICE file distributed with this work for additional 
information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache 
License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the 
License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software 
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 
KIND, either express
+ * or implied. See the License for the specific language governing permissions 
and limitations under
+ * the License.
+ */
+package org.apache.geode.internal.cache.wan;
+
+import static 
org.apache.geode.distributed.ConfigurationProperties.DISTRIBUTED_SYSTEM_ID;
+import static org.apache.geode.distributed.ConfigurationProperties.LOCATORS;
+import static org.apache.geode.distributed.ConfigurationProperties.MCAST_PORT;
+import static 
org.apache.geode.distributed.ConfigurationProperties.REMOTE_LOCATORS;
+import static 
org.apache.geode.distributed.ConfigurationProperties.START_LOCATOR;
+import static 
org.apache.geode.internal.AvailablePortHelper.getRandomAvailableTCPPortsForDUnitSite;
+import static org.apache.geode.test.awaitility.GeodeAwaitility.await;
+import static org.apache.geode.test.dunit.VM.getVM;
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.File;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Properties;
+import java.util.Set;
+import java.util.stream.IntStream;
+
+import junitparams.JUnitParamsRunner;
+import junitparams.Parameters;
+import junitparams.naming.TestCaseName;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+import org.apache.geode.cache.DiskStore;
+import org.apache.geode.cache.DiskStoreFactory;
+import org.apache.geode.cache.Region;
+import org.apache.geode.cache.RegionShortcut;
+import org.apache.geode.cache.wan.GatewayReceiverFactory;
+import org.apache.geode.cache.wan.GatewaySender;
+import org.apache.geode.cache.wan.GatewaySenderFactory;
+import org.apache.geode.distributed.internal.InternalDistributedSystem;
+import org.apache.geode.internal.cache.DiskStoreImpl;
+import org.apache.geode.internal.cache.RegionQueue;
+import org.apache.geode.internal.cache.persistence.OplogType;
+import org.apache.geode.test.dunit.VM;
+import org.apache.geode.test.dunit.rules.CacheRule;
+import org.apache.geode.test.dunit.rules.DistributedDiskDirRule;
+import org.apache.geode.test.dunit.rules.DistributedRule;
+import org.apache.geode.test.junit.rules.serializable.SerializableTestName;
+
+/**
+ * Tests to verify WAN functionality when the gateway-sender(s) have isolated, 
non-shared with
+ * other region(s), disk-store(s).
+ */
+@RunWith(JUnitParamsRunner.class)
+public class PersistentGatewaySenderWithIsolatedDiskStor

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-05-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097290#comment-17097290
 ] 

ASF GitHub Bot commented on GEODE-8029:
---

jujoramos opened a new pull request #5037:
URL: https://github.com/apache/geode/pull/5037


   The OpLog initialization now delete unused drf files to prevent the
   proliferation of unused records and files within the system, which
   can could cause members to fail during startup while recovering
   disk-stores (especially when they are isolated for gateway-senders).
   
   - Added distributed tests.
   - Delete orphaned drf files when deleting the corresponding
 crf during recovery.
   
   Thank you for submitting a contribution to Apache Geode.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [X] Is there a JIRA ticket associated with this PR? Is it referenced in 
the commit message?
   
   - [X] Has your PR been rebased against the latest commit within the target 
branch (typically `develop`)?
   
   - [X] Is your initial contribution a single, squashed commit?
   
   - [X] Does `gradlew build` run cleanly?
   
   - [X] Have you written or updated unit tests to verify your changes?
   
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   
   ### Note:
   Please ensure that once the PR is submitted, check Concourse for build 
issues and
   submit an update to your PR as soon as possible. If you need help, please 
send an
   email to d...@geode.apache.org.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(Cac

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-29 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095524#comment-17095524
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Thanks Juan, we will compact the disk-store periodically whenever records are 
more than threshold (10 records). 

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-29 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095497#comment-17095497
 ] 

Juan Ramos commented on GEODE-8029:
---

Hello [~jagan23527001],

Just wanted to give you a quick update here, I can confirm that this is a bug 
within the product, reproducible only when a {{gateway-sender}} has a 
{{disk-store}} attached and that {{disk-store}} is not attached to any other 
regions (this is the recommended way of configuring things, BTW, so there's 
nothing wrong with what you did).
I'll start working on a fix for the problem but, as a workaround and to avoid 
hitting the {{IllegalArgumentException}}, I'd recommend to execute offline 
compaction on your {{disk-stores}} regularly. How often?, hard to tell as it 
depends on your workload and the amount of compactable records within the 
{{disk-store}}... you can execute the {{validate offline-disk-store}} command 
and verify the output to check the actual amount, if it's higher than a certain 
threshold (let's say 10 records), go ahead and compact it while the servers 
are online.
For more information about this, please have a look at [Running Compaction on 
Disk Store Log 
Files|https://geode.apache.org/docs/guide/112/managing/disk_storage/compacting_disk_stores.html].



> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCach

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094663#comment-17094663
 ] 

Juan Ramos commented on GEODE-8029:
---

Hello [~jagan23527001]
Great, glad to hear it worked!.
That said, old {{oplog}} files should be automatically deleted by Geode when 
you have {{auto-compaction}} enabled (which you do according to the logs), I'm 
still not sure how the server got into that situation, it's certainly the first 
time I see this issue. I'll continue investigating and will update the JIRA 
once I have something more concise to share.
Please feel free to reach out if you have further problems.
Cheers.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLaun

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094661#comment-17094661
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Thanks Juan, 

 

Cache Server came up with the workaround approach ( deleting the old files). is 
there any configuration in "alter" disk-store to automatically compact or 
delete the old files after certain period ? because the Disk-store was 
containing the old files from 2018 and all getting loaded into memory.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sen

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094628#comment-17094628
 ] 

Juan Ramos commented on GEODE-8029:
---

Hello [~jagan23527001],

The {{Geode SHell}} tool is just a regular java process so you can directly 
execute {{export JAVA_ARGS="-XmxZg"}} (being Z the max heap you want to use) 
before starting the tool. I'm not entirely sure {{compaction}} will succeed in 
this case, though, chances are high you will hit the same 
{{IllegalArgumentException}}, as Geode needs to load the oplogs into the 
internal hash table before trying to compact old entries... worth giving it a 
try, anyway, if it doesn't work you can go ahead and try the other workaround 
mentioned (delete the files and let the member get the date from the other 
running servers).

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094616#comment-17094616
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

gfsh>compact offline-disk-store --name=geodeStore 
--disk-dirs=/app/provServerHO2/data
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at it.unimi.dsi.fastutil.ints.IntOpenHashSet.rehash(IntOpenHashSet.java:600)
 at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
 at 
org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
 at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
 at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
 at 
org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
 at 
org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
 at 
org.apache.geode.internal.cache.DiskStoreImpl.offlineCompact(DiskStoreImpl.java:3934)
 at 
org.apache.geode.internal.cache.DiskStoreImpl.offlineCompact(DiskStoreImpl.java:4226)
 at 
org.apache.geode.management.internal.cli.util.DiskStoreCompacter.compact(DiskStoreCompacter.java:135)
 at 
org.apache.geode.management.internal.cli.util.DiskStoreCompacter.main(DiskStoreCompacter.java:71)

gfsh>

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apac

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094606#comment-17094606
 ] 

Juan Ramos commented on GEODE-8029:
---

Thanks [~jagan23527001],

Looks like the validation fails with the same exception, which is expected... 
my current theory is that you have a *huge* amount of deleted records within 
the {{disk-store}} that should have been compacted but, instead, are still 
there, preventing the member from starting up. If my theory is correct, you 
should be able to execute an {{offline compaction}} instead of the steps I've 
previously shared, the steps are below:

# For member {{provServerHO2}}, copy all files under 
{{/app/provServerHO2/data/}} to another directory, just as a backup.
# For member {{provServerHO2}}, execute {{compact offline-disk-store 
--name=geodeStore --disk-dirs=/app/provServerHO2/data}}.
# Try to start member {{provServerHO2}} again, it should come up just fine.
# At this point the cluster should be fully operational, so you can go ahead 
and execute your internal verifications to double check everything is correct.

If the above steps don't work (they should, I'm just adding another option as a 
workaround), you can go ahead and execute the steps I've shared previously, 
that will guarantee the member starts fresh and gets the data from the already 
running servers. The steps are below again, just for your reference:

# Make sure {{provServerHO1}} and {{provServerHO3}} are fully up and running, 
without any exceptions in the logs. If you notice any exceptions or weirdness 
within these members logs, don't continue with the rest of the steps.
# For member {{provServerHO2}}, copy all files under 
{{/app/provServerHO2/data/}} to another directory, just as a backup.
# For member {{provServerHO2}}, remove all files under 
{{/app/provServerHO2/data/}}.
# Try to start member {{provServerHO2}} again, during the startup procedure the 
member should be able to get the latest data from the other running members 
({{provServerHO1}} and {{provServerHO3}}).
# If the above steps finished correctly, execute the [{{gfsh 
rebalance}}|https://geode.apache.org/docs/guide/112/tools_modules/gfsh/command-pages/rebalance.html]
 command to make sure buckets are evenly distributed across the three members 
(this is an expensive operation, so you might want to go through [Rebalancing 
Partitioned Region 
Data|https://geode.apache.org/docs/guide/112/developing/partitioned_regions/rebalancing_pr_data.html]
 to fully understand the implications and requirements).
# At this point the cluster should be fully operational, so you can go ahead 
and execute your internal verifications to double check everything is correct.

Please let me know how it goes.


> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.in

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094601#comment-17094601
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

There is no reason for default DiskStore for region and geodeStore for gateway 
Sender, based on Geode document we have created the Diskstore for gateway 
Sender. 

 

Thanks Juan for your help. 

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094598#comment-17094598
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

gfsh>validate offline-disk-store --name=geodeStore 
--disk-dirs=/app/provServerHO2/data
/PdxTypes: entryCount=0
/ConfigCacheSenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.2_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.5_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.7_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/ConfigCacheSenderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.2_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.5_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.7_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWSLASenderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.2_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.5_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.7_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWLiteSLASenderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.2_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.5_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.7_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBaccGuiSenderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.2_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.5_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.7_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/BACCRDUSiteIDSenderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyDeviceCacheSenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyDeviceCacheSenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyDeviceCacheSenderHOC.2_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyDeviceCacheSenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyDeviceCacheSenderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.2_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.5_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.7_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/TopologyBACCCOSSendderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: entryCount=0
/PGWKafkaSenderHOC_PARALLEL_GATEWAY_SENDER_QUEUE entryCount=0 bucketCount=13
/PSKafkaSenderHOC_PARALLEL_GATEWAY_SENDER_QUEUE entryCount=0 bucketCount=13
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at it.uni

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094585#comment-17094585
 ] 

Juan Ramos commented on GEODE-8029:
---

Thanks [~jagan23527001], what about {{validate offline-disk-store 
--name=geodeStore --disk-dirs=/app/provServerHO2/data}}?, can you execute that 
and attach the results?.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094583#comment-17094583
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

gfsh>describe offline-disk-store --name=geodeStore 
--disk-dirs=/app/provServerHO2/data
Regions in the disk store:
 /BACCRDUSiteIDSenderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /BACCRDUSiteIDSenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /ConfigCacheSenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyBaccGuiSenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /PGWLiteSLASenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyBACCCOSSendderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: 
-lru=lifo-memory-size -lruAction=overflow-to-disk -lruLimit=150 
-concurrencyLevel=16 -initialCapacity=16 -loadFactor=0.75 -offHeap=false 
-compressor=none -statisticsEnabled=false
 /PGWSLASenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyDeviceCacheSenderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: 
-lru=lifo-memory-size -lruAction=overflow-to-disk -lruLimit=300 
-concurrencyLevel=16 -initialCapacity=16 -loadFactor=0.75 -offHeap=false 
-compressor=none -statisticsEnabled=false
 /PGWSLASenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyBaccGuiSenderHOC.8_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /PGWLiteSLASenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyBACCCOSSendderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: 
-lru=lifo-memory-size -lruAction=overflow-to-disk -lruLimit=150 
-concurrencyLevel=16 -initialCapacity=16 -loadFactor=0.75 -offHeap=false 
-compressor=none -statisticsEnabled=false
 /PGWLiteSLASenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyBACCCOSSendderHOC.4_SERIAL_GATEWAY_SENDER_QUEUE: 
-lru=lifo-memory-size -lruAction=overflow-to-disk -lruLimit=150 
-concurrencyLevel=16 -initialCapacity=16 -loadFactor=0.75 -offHeap=false 
-compressor=none -statisticsEnabled=false
 /PGWSLASenderHOC.0_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyBaccGuiSenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /ConfigCacheSenderHOC.3_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /BACCRDUSiteIDSenderHOC.6_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /PGWSLASenderHOC.9_SERIAL_GATEWAY_SENDER_QUEUE: -lru=lifo-memory-size 
-lruAction=overflow-to-disk -lruLimit=150 -concurrencyLevel=16 
-initialCapacity=16 -loadFactor=0.75 -offHeap=false -compressor=none 
-statisticsEnabled=false
 /TopologyDeviceCacheSenderHOC.1_SERIAL_GATEWAY_SENDER_QUEUE: 
-lru=lifo-memory-size -lruAction=overflow-to-d

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094400#comment-17094400
 ] 

Juan Ramos commented on GEODE-8029:
---

Hello again [~jagan23527001], another thing I've noticed from your 
configuration is that you're using the {{DEFAULT}} {{disk-store}} for all your 
regions, and the custom one ({{geodeStore}}) only for your {{PdxTypes}} and 
{{gateway-senders}}, is there any reason for that?.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-28 Thread Juan Ramos (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094347#comment-17094347
 ] 

Juan Ramos commented on GEODE-8029:
---

Hello [~jagan23527001], just wanted to let you know that I'm currently working 
on the issue, will get back to you as soon as I have an update.
That said, according to your description 2 out of 3 servers have been 
_*successfully*_ started, and I can also see from the logs that you have six 
{{PERSISTENT_REPLICATE}} regions (data replicated across all members) and two 
{{PARTITION_PERSISTENT}} regions, both with 2 redundant copies. Based on the 
above facts, we can assume that the two members that are up and running contain 
all the data, so it should be safe to delete the {{disk-store}} data from the 
failing member and start it from scratch to force it to get the data from the 
members that are already up running.
Would you be able to execute the following steps and update the ticket with the 
outcome?:
# Make sure {{provServerHO1}} and {{provServerHO3}} are fully up and running, 
without any exceptions in the logs. If you notice any exceptions or weirdness 
within these members logs, don't continue with the rest of the steps.
# For member {{provServerHO2}}, copy all files under 
{{/app/provServerHO2/data/}} to another directory, just as a backup.
# For member {{provServerHO2}}, remove all files under 
{{/app/provServerHO2/data/}}.
# Try to start member {{provServerHO2}} again, during the startup procedure the 
member should be able to get the latest data from the other running members 
({{provServerHO1}} and {{provServerHO3}}).
# If the above steps finished correctly, execute the [{{gfsh 
rebalance}}|https://geode.apache.org/docs/guide/112/tools_modules/gfsh/command-pages/rebalance.html]
 command to make sure buckets are evenly distributed across the three members 
(this is an expensive operation, so you might want to go through [Rebalancing 
Partitioned Region 
Data|https://geode.apache.org/docs/guide/112/developing/partitioned_regions/rebalancing_pr_data.html]
 to fully understand the implications and requirements).
# At this point the cluster should be fully operational, so you can go ahead 
and execute your internal verifications to double check everything is correct.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Assignee: Juan Ramos
>Priority: Major
>  Labels: GeodeCommons, caching-applications
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStore

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-27 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093937#comment-17093937
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

*If i move this file "/app/provServerHO2/data/BACKUPgeodeStore_5.drf" to 
another Directory and start the Cache server,  am getting below error.* 

 

Starting a Geode Server in /app/provServerHO2...
The Cache Server process terminated unexpectedly with exit status 1. Please 
refer to the log file in /app/provServerHO2 for full details.

Exception in thread "main" java.lang.Illega*lStateException: The following 
required files could not be found: *.drf files with these ids: [5].*

at 
org.apache.geode.internal.cache.DiskInitFile.verifyOplogs(DiskInitFile.java:832)

at 
org.apache.geode.internal.cache.DiskInitFile.verifyOplogs(DiskInitFile.java:805)

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Priority: Major
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geo

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-27 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093934#comment-17093934
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Server is getting closed when it loads the data from diskstore which is very 
old ( 2 years old drf files). Attached the log file [^server02.log]

 

 

[info 2020/04/27 20:57:57.485 UTC  tid=0x1] Recovering oplog#6 
/app/provServerHO2/data/BACKUPgeodeStore_6.drf for disk store geodeStore.

[info 2020/04/27 20:57:58.292 UTC  tid=0x1] Recovering oplog#5 
/app/provServerHO2/data/BACKUPgeodeStore_5.drf for disk store geodeStore.

[error 2020/04/27 21:01:58.716 UTC  tid=0x1] Cache initialization for 
GemFireCache[id = 847839957; isClosing = false; isShutDownAll = false; created 
= Mon Apr 27 20:54:11 UTC 2020; server = false; copyOnRead = false; lockLease = 
120; lockTimeout = 60] failed because: java.lang.IllegalArgumentException: Too 
large (805306401 expected elements with load factor 0.75)

[info 2020/04/27 21:01:58.743 UTC  tid=0x1] GemFireCache[id = 847839957; 
isClosing = true; isShutDownAll = false; created = Mon Apr 27 20:54:11 UTC 
2020; server = false; copyOnRead = false; lockLease = 120; lockTimeout = 60]: 
Now closing.

[info 2020/04/27 21:01:58.792 UTC  tid=0x1] Shutting down 
DistributionManager .79.149(provServerHO2:9798):1024.

[info 2020/04/27 21:01:58.896 UTC  tid=0x1] Now closing distribution for 
.79.149(provServerHO2:9798):1024

[info 2020/04/27 21:01:58.897 UTC  tid=0x1] Stopping membership services

[info 2020/04/27 21:01:58.901 UTC  
tid=0x1f] GMSHealthMonitor server thread exiting

[info 2020/04/27 21:01:58.915 UTC  tid=0x1] DistributionManager stopped 
in 123ms.

[info 2020/04/27 21:01:58.916 UTC  tid=0x1] Marking DistributionManager 
.79.149(provServerHO2:9798):1024 as closed.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Priority: Major
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png, server02.log
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-27 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093868#comment-17093868
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Hi Antony,

YEs, In One region  yes we  create and destroy the keys . we would have 
destroyed more thant 2M+ entries in that region. Others are constant.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Priority: Major
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-27 Thread Anthony Baker (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093857#comment-17093857
 ] 

Anthony Baker commented on GEODE-8029:
--

Are you creating and destroying a lot of keys?  This should not be a problem, 
just trying to understand the use case.

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Affects Versions: 1.9.0
>Reporter: Jagadeesh sivasankaran
>Priority: Major
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-27 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093809#comment-17093809
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Under the "/app/data" Diskstore we have lot backup files, is that getting 
loaded into hash and causing the issue. ?

 

!Screen Shot 2020-04-27 at 12.21.19 PM.png!

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Reporter: Jagadeesh sivasankaran
>Priority: Major
> Attachments: Screen Shot 2020-04-27 at 12.21.19 PM.png, Screen Shot 
> 2020-04-27 at 12.21.19 PM.png
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-27 Thread Jagadeesh sivasankaran (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093796#comment-17093796
 ] 

Jagadeesh sivasankaran commented on GEODE-8029:
---

Version - 1.9.0

JDK - openjdk version "1.8.0_242"

Total REgion - 10 Region

Region - Most of the  Regions have Entries in hundreds Except one region which 
have maximum 72000 Entries . Total  Entires in all region should not exceed 
more than 100K.

REgion, DiskStore, GatewaySender Configuration similar to all regions -

 

create disk-store --name=clusteraHoStore --dir=/storage/data/

create gateway-sender --id=WODetails5 --parallel=false 
--remote-distributed-system-id=1 --maximum-queue-memory=1500 
--enable-persistence=true --disk-store-name=clusteraHoStore 
--dispatcher-threads=10 --order-policy=PARTITION


create region --name=workOrderDetails2 --gateway-sender-id=WODetails3 
--type=REPLICATE_PERSISTENT_OVERFLOW --entry-idle-time-expiration=604800 
--enable-statistics=true

 

 

Let check with the My Manager, whether we can share log file.

 

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Reporter: Jagadeesh sivasankaran
>Priority: Major
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.create

[jira] [Commented] (GEODE-8029) java.lang.IllegalArgumentException: Too large (805306401 expected elements with load factor 0.75)

2020-04-27 Thread Anthony Baker (Jira)



[ 
https://issues.apache.org/jira/browse/GEODE-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093661#comment-17093661
 ] 

Anthony Baker commented on GEODE-8029:
--

Thanks for the bug report.  To help you better, can you provide the following:

 
 * What was the version of geode?
 * What was the region configuration?
 * How many entries were in the region?
 * Do you have a complete log file you can share?
 * What was the JDK version?

> java.lang.IllegalArgumentException: Too large (805306401 expected elements 
> with load factor 0.75)
> -
>
> Key: GEODE-8029
> URL: https://issues.apache.org/jira/browse/GEODE-8029
> Project: Geode
>  Issue Type: Bug
>  Components: configuration, core, gfsh
>Reporter: Jagadeesh sivasankaran
>Priority: Major
> Fix For: 1.9.0
>
>
> we have a cluster of three Locator Geode and three Cache Server running in 
> CentOS servers. Today (April 27) after patching our CENTOS servers , all 
> locator and 2 servers came up , But one Cache server was not starting . here 
> is the Exception details.  Please let me know how to resolve the beloe issue 
> and need any configuration changes to diskstore ? 
>  
>  
> Starting a Geode Server in /app/provServerHO2...
> The
>  Cache Server process terminated unexpectedly with exit status 1. Please 
> refer to the log file in /app/provServerHO2 for full details.
> Exception in thread "main" java.lang.IllegalArgumentException: Too large 
> (805306401 expected elements with load factor 0.75)
> at it.unimi.dsi.fastutil.HashCommon.arraySize(HashCommon.java:222)
> at it.unimi.dsi.fastutil.ints.IntOpenHashSet.add(IntOpenHashSet.java:308)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl$OplogEntryIdSet.add(DiskStoreImpl.java:3474)
> at org.apache.geode.internal.cache.Oplog.readDelEntry(Oplog.java:3007)
> at org.apache.geode.internal.cache.Oplog.recoverDrf(Oplog.java:1500)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverOplogs(PersistentOplogSet.java:445)
> at 
> org.apache.geode.internal.cache.PersistentOplogSet.recoverRegionsThatAreReady(PersistentOplogSet.java:369)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.recoverRegionsThatAreReady(DiskStoreImpl.java:2053)
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.initializeIfNeeded(DiskStoreImpl.java:2041)
> security-peer-auth-init=
> at 
> org.apache.geode.internal.cache.DiskStoreImpl.doInitialRecovery(DiskStoreImpl.java:2046)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.initializeDiskStore(DiskStoreFactoryImpl.java:184)
> at 
> org.apache.geode.internal.cache.DiskStoreFactoryImpl.create(DiskStoreFactoryImpl.java:150)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.createDiskStore(CacheCreation.java:794)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.initializePdxDiskStore(CacheCreation.java:785)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:509)
> at 
> org.apache.geode.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:337)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4272)
> at 
> org.apache.geode.internal.cache.ClusterConfigurationLoader.applyClusterXmlConfiguration(ClusterConfigurationLoader.java:197)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.applyJarAndXmlFromClusterConfig(GemFireCacheImpl.java:1240)
> at 
> org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1206)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:207)
> at 
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:164)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:139)
> at 
> org.apache.geode.distributed.internal.DefaultServerLauncherCacheProvider.createCache(DefaultServerLauncherCacheProvider.java:52)
> at 
> org.apache.geode.distributed.ServerLauncher.createCache(ServerLauncher.java:869)
> at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:786)
> at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
> at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

61 matches

Mail list logo