[
https://issues.apache.org/jira/browse/IGNITE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov updated IGNITE-21399:
------------------------------------
Labels: ise (was: )
> IgniteCache#getAll invoked on transactional cache can lead to a node failure
> with OutOfMemoryError
> --------------------------------------------------------------------------------------------------
>
> Key: IGNITE-21399
> URL: https://issues.apache.org/jira/browse/IGNITE-21399
> Project: Ignite
> Issue Type: Bug
> Reporter: Mikhail Petrov
> Assignee: Mikhail Petrov
> Priority: Major
> Labels: ise
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Exception:
> {code:java}
> 2024-01-18 09:28:07.374 [ERROR][sys-#4371][] Critical system error detected.
> Will be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
> failureCtx=FailureContext [type=CRITICAL_ERROR,
> err=java.lang.OutOfMemoryError]]
> java.lang.OutOfMemoryError: null
> at
> java.lang.AbstractStringBuilder.hugeCapacity(AbstractStringBuilder.java:214)
> ~[?:?]
> at
> java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:206)
> ~[?:?]
> at
> java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:173)
> ~[?:?]
> at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538) ~[?:?]
> at java.lang.StringBuilder.append(StringBuilder.java:178) ~[?:?]
> at java.lang.StringBuilder.append(StringBuilder.java:172) ~[?:?]
> at
> java.util.concurrent.ConcurrentHashMap.toString(ConcurrentHashMap.java:1329)
> ~[?:?]
> at java.lang.String.valueOf(String.java:2951) ~[?:?]
> at java.lang.StringBuilder.append(StringBuilder.java:172) ~[?:?]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils$18.toString(GridCacheUtils.java:646)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at java.lang.String.valueOf(String.java:2951) ~[?:?]
> at
> org.apache.ignite.internal.util.GridStringBuilder.a(GridStringBuilder.java:102)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.tostring.SBLimitedLength.a(SBLimitedLength.java:100)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:910)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl0(GridToStringBuilder.java:1121)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1055)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:622)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:561)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.toString(GridCompoundFuture.java:421)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.future.GridCompoundIdentityFuture.toString(GridCompoundIdentityFuture.java:47)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.CacheDistributedGetFutureAdapter.toString(CacheDistributedGetFutureAdapter.java:377)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedGetFuture.toString(GridPartitionedGetFuture.java:664)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at java.lang.String.valueOf(String.java:2951) ~[?:?]
> at java.lang.StringBuilder.append(StringBuilder.java:172) ~[?:?]
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.dumpLongRunningOperations0(GridCachePartitionExchangeManager.java:2290)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.dumpLongRunningOperations(GridCachePartitionExchangeManager.java:2439)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.lambda$null$0(IgniteTxManager.java:398)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7437)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> ~[ignite-core-14.1.2.jar:14.1.2]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) ~[?:?]
> 2024-01-18 09:28:07.411 [WARN
> ][sys-#4371][org.apache.ignite.internal.processors.cache.CacheDiagnosticManager]
> Page locks dump:
> {code}
> Java version: OpenJDK Runtime Environment 11.0.20+8-LTS
> IgniteCache#getAll is called on a transactional cache with a huge number of
> keys (> 1000000). Near node creates a future for this operation and sends
> requests for cache entries to remote nodes. All received entries are
> accumulated in the near node reducer (CacheDistributedGetFutureAdapter#rdc).
> If the mentioned future is not completed for a long time, long running
> operation detector will dump all pending futures in a log. The result
> includes mentioned reducer and string representation of all received cache
> entries.
> The `toString` implementation in the reducer just invokes
> ConcurrentHashMap#toString which can result in the OutOfMemoryError if huge
> amount of keys are requested. The OutOfMemoryError in this case does not mean
> that no free heap space is available (see
> https://github.com/openjdk/jdk/blob/d7ca08a5cc64c8d3941493d98423a49b5bc1b922/src/java.base/share/classes/java/lang/AbstractStringBuilder.java#L269)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
