Jai Bheemsen Rao Dhanwada created CASSANDRA-14840: -----------------------------------------------------
Summary: Bootstrap of new node fails with OOM in a large cluster Key: CASSANDRA-14840 URL: https://issues.apache.org/jira/browse/CASSANDRA-14840 Project: Cassandra Issue Type: Bug Components: Streaming and Messaging Reporter: Jai Bheemsen Rao Dhanwada We are seeing new node addition fails with OOM during bootstrap in a cluster of more than 80 nodes and 3000 CF without any data in those CFs. Steps to reproduce: # Launch a 3 node cluster # Create 3000 CF in the cluster # Start adding nodes to the cluster one by one # After adding 75-80 nodes, the new node bootstrap fails with OOM. {code:java} ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2018-10-24 03:26:47,870 JVMStabilityInspector.java:78 - Exiting due to error while processing commit log during initialization. java.lang.OutOfMemoryError: Java heap space at java.util.regex.Pattern.matcher(Pattern.java:1093) ~[na:1.8.0_151] at java.util.Formatter.parse(Formatter.java:2547) ~[na:1.8.0_151] at java.util.Formatter.format(Formatter.java:2501) ~[na:1.8.0_151] at java.util.Formatter.format(Formatter.java:2455) ~[na:1.8.0_151] at java.lang.String.format(String.java:2940) ~[na:1.8.0_151] at org.apache.cassandra.db.commitlog.AbstractCommitLogService$1.run(AbstractCommitLogService.java:105) ~[apache-cassandra-2.1.16.jar:2.1.16] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]{code} Cassandra Version: 2.1.16 OS: CentOS7 num_tokens: 256 on each node. This behavior is blocking us from adding extra capacity when needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org