[ https://issues.apache.org/jira/browse/IGNITE-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maxim Muzafarov updated IGNITE-9090: ------------------------------------ Fix Version/s: (was: 2.8) 2.9 > When client node make cache.QueryCursorImpl.getAll they have OOM and continue > working > ------------------------------------------------------------------------------------- > > Key: IGNITE-9090 > URL: https://issues.apache.org/jira/browse/IGNITE-9090 > Project: Ignite > Issue Type: Bug > Components: clients > Affects Versions: 2.4 > Environment: 2 server node, 1 client, 1 cache with 15 kk size > Reporter: ARomantsov > Priority: Critical > Fix For: 2.9 > > > {code:java} > [12:21:22,390][SEVERE][query-#69][GridCacheIoManager] Failed to process > message [senderId=30cab4ec-1da7-4e9f-a262-bdfa4d466865, messageType=class > o.a.i.i.processors.cache.query.GridCacheQueryResponse] > java.lang.OutOfMemoryError: GC overhead limit exceeded > at java.lang.Long.valueOf(Long.java:840) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:250) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:421) > at > org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:421) > at > org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:227) > at > org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94) > at > org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1777) > at > org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964) > at > org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716) > at > org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310) > at > org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99) > at > org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82) > at > org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.unmarshalCollection0(GridCacheQueryResponse.java:189) > at > org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.finishUnmarshal(GridCacheQueryResponse.java:162) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1530) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:576) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1613) > at > org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) > at > org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125) > at > org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752) > at > org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516) > at > org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125) > at > org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [12:21:28,573][INFO][ignite-update-notifier-timer][GridUpdateNotifier] Update > status is not available. > [12:21:23,759][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 22446 milliseconds. > [12:21:23,758][INFO][grid-timeout-worker-#39][IgniteKernal] > Metrics for local node (to disable set 'metricsLogFrequency' to 0) > ^-- Node [id=c1f087b1, uptime=00:01:25.431] > ^-- H/N/C [hosts=2, nodes=3, CPUs=32] > ^-- CPU [cur=100%, avg=79.09%, GC=8.93%] > ^-- PageMemory [pages=0] > ^-- Heap [used=216MB, free=8.57%, comm=236MB] > ^-- Non heap [used=62MB, free=-1%, comm=65MB] > ^-- Outbound messages queue [size=0] > ^-- Public thread pool [active=0, idle=0, qSize=0] > ^-- System thread pool [active=0, idle=0, qSize=0] > [12:21:32,609][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 8800 milliseconds. > [12:21:36,042][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 3383 milliseconds. > [12:21:41,883][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 5791 milliseconds. > [12:21:41,884][SEVERE][grid-timeout-worker-#39][GridTimeoutProcessor] Error > when executing timeout callback: CancelableTask > [id=92823e5d461-bae41fe7-1702-4a25-8681-916897029985, endTime=1532596959200, > period=60000, cancel=false, task=o.a.i.i.IgniteKernal$4@c0ec5ba] > java.lang.OutOfMemoryError: GC overhead limit exceeded > [12:21:43,225][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1291 milliseconds. > [12:21:44,553][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1279 milliseconds. > [12:21:46,979][SEVERE][grid-timeout-worker-#39][GridTimeoutProcessor] Runtime > error caught during grid runnable execution: GridWorker > [name=grid-timeout-worker, igniteInstanceName=null, finished=false, > hashCode=1065113566, interrupted=false, runner=grid-timeout-worker-#39] > java.lang.OutOfMemoryError: GC overhead limit exceeded > [12:21:48,392][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 3789 milliseconds. > [12:21:49,712][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1270 milliseconds. > [12:21:51,007][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1245 milliseconds. > [12:21:52,115][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1058 milliseconds. > [12:21:53,427][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1262 milliseconds. > [12:21:53,427][SEVERE][tcp-client-disco-sock-reader-#3][TcpDiscoverySpi] > Runtime error caught during grid runnable execution: IgniteSpiThread > [name=tcp-client-disco-sock-reader-#3] > java.lang.OutOfMemoryError: GC overhead limit exceeded > [12:21:54,808][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1331 milliseconds. > [12:21:55,594][SEVERE][ignite-license-checker-null][GridEntLicenseProcessor] > License grace/burst period - left 57 mins. > [12:22:18,203][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 1368 milliseconds. > [12:22:25,228][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 2370 milliseconds. > [12:22:28,428][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 3150 milliseconds. > [12:22:32,748][WARNING][jvm-pause-detector-worker][] Possible too long JVM > pause: 4270 milliseconds. > [12:22:37,250][SEVERE][tcp-client-disco-sock-writer-#2][TcpDiscoverySpi] > Failed to send message: null > java.io.IOException: Failed to get acknowledge for message: > TcpDiscoveryClientMetricsUpdateMessage [super=TcpDiscoveryAbstractMessage > [sndNodeId=null, id=cb823e5d461-c1f087b1-678e-4377-a3d1-4bb5dbdb4c98, > verifierNodeId=null, topVer=0, pendingIdx=0, failedNodes=null, isClient=true]] > at > org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketWriter.body(ClientImpl.java:1359) > at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)