[ https://issues.apache.org/jira/browse/CASSANDRA-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183177#comment-17183177 ]
ZhaoYang edited comment on CASSANDRA-16071 at 8/24/20, 11:57 AM: ----------------------------------------------------------------- {code:java} long maxMemMb = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null ? (long) (1048576 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for memtable : Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION)); {code} [~mck] I think the default {{"1048576 * 0.15 mb" (153GB)}} may still cause OOM. How about we rename {{"maxMemMb"}} to {{"maxMemBytes"}} and rename {{"IndexMode#maxCompactionFlushMemoryInMb"}} to {{"maxCompactionFlushMemoryInBytes"}}. So it should be : {code:java} long maxMemBytes = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null ? (long) (1073741824 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for memtable : 1048576L * Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION)); {code} was (Author: jasonstack): {code:java} long maxMemMb = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null ? (long) (1048576 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for memtable : Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION)); {code} [~mck] I think the default {{"1048576 * 0.15 mb" (153GB)}} may still cause OOM. How about we rename {{"maxMemMb"}} to {{"maxMemBytes" and rename {{"IndexMode#maxCompactionFlushMemoryInMb"}} to {{"maxCompactionFlushMemoryInBytes"}}. So it should be : {code:java} long maxMemBytes = indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION) == null ? (long) (1073741824 * INDEX_MAX_FLUSH_DEFAULT_MULTIPLIER) // 1G default for memtable : 1048576L * Long.parseLong(indexOptions.get(INDEX_MAX_FLUSH_MEMORY_OPTION)); {code} > max_compaction_flush_memory_in_mb is interpreted as bytes > --------------------------------------------------------- > > Key: CASSANDRA-16071 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16071 > Project: Cassandra > Issue Type: Bug > Components: Feature/SASI > Reporter: Michael Semb Wever > Assignee: Michael Semb Wever > Priority: Normal > Fix For: 3.11.x, 4.0-beta > > > In CASSANDRA-12662, [~scottcarey] > [reported|https://issues.apache.org/jira/browse/CASSANDRA-12662?focusedCommentId=17070055&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17070055] > that the {{max_compaction_flush_memory_in_mb}} setting gets incorrectly > interpreted in bytes rather than megabytes as its name implies. > {quote} > 1. the setting 'max_compaction_flush_memory_in_mb' is a misnomer, it is > actually memory in BYTES. If you take it at face value, and set it to say, > '512' thinking that means 512MB, you will produce a million temp files > rather quickly in a large compaction, which will exhaust even large values of > max_map_count rapidly, and get the OOM: Map Error issue above and possibly > have a very difficult situation to get a cluster back into a place where > nodes aren't crashing while initilaizing or soon after. This issue is minor > if you know about it in advance and set the value IN BYTES. > {quote} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org