Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hello! Yes, data is kept in off-heap. But if you do a large group by or order by without using index, the intermediate results may overwhelm your heap. I have no idea whether increasing heap to 6G will help or not. The recommendation is to test every query on development cluster and never experiment with group by/order by queries live in production. Regards, -- Ilya Kasnacheev вт, 19 мар. 2019 г. в 03:33, James Wang 王升平 (edvance CN) < james.w...@edvancesecurity.com>: > Hi Ilya Kasnacheev, > > > > On Ingite 2.7 > > No code is using affinityCall() > > Haven’t enabled cache statistics > > > > Could you please answer my questions in red below: > > > > Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps > as it is very sensitive to free heap memory. > > James>> Yes, it helps. > > As I understand all the data is kept in non-heap memory, But could you > advise when the heap memory will be used, like “order”, “group by” > operation ? > > Looks like you have Visor node (or may be web-console) in grid. Is OOM > happened only when Visor attached to grid? > > James>> Most like to occur if execute a complicate query. > > Another question is: if I set Xmx=6GB, is it possible to avoide > out-of-memory failure. I regard this very critical because it make node > halt. > > > > Best Regards, > > James Wang > > M/WeChat: +86 135 1215 1134 > > > > *From:* Ilya Kasnacheev > *Sent:* Tuesday, March 5, 2019 09:44 PM > *To:* user@ignite.apache.org > *Subject:* Re: java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > Hello! > > > > Is there a chance you were using affinityCall() and you're on Ignite 2.7? > > > > We have a bug which can cause spurious OOMs here: > https://issues.apache.org/jira/browse/IGNITE-10925 - disabling cache > statistics will help > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > сб, 2 мар. 2019 г. в 18:13, James Wang 王升平 (edvance CN) < > james.w...@edvancesecurity.com>: > > Hi Support, > > > > Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps > as it is very sensitive to free heap memory. > > James>> will increase heap memory and retry. > > As I understand all the data is kept in non-heap memory, But could you > advise when the heap memory will be used, like “order”, “group by” > operation ? > > > > Looks like you have Visor node (or may be web-console) in grid. Is OOM > happened only when Visor attached to grid? > > James>> Most like to occur if execute a complicate query. > > > > Another question is: if I set Xmx=6GB, is it possible to avoide > out-of-memory failure. I regard this very critical because it make node > halt. > > > > Thank you. > > > > Best Regards, > > James Wang > > M/WeChat: +86 135 1215 1134 > > > > *From:* Andrey Mashenkov > *Sent:* Friday, March 1, 2019 06:50 PM > *To:* user@ignite.apache.org > *Subject:* Re: java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > Hi, > > > > Most likely heap size is too low. > > Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps > as it is very sensitive to free heap memory. > > > > Looks like you have Visor node (or may be web-console) in grid. Is OOM > happened only when Visor attached to grid? > > > > On Fri, Mar 1, 2019 at 7:17 AM James Wang 王升平 (edvance CN) < > james.w...@edvancesecurity.com> wrote: > > OS: 4C +8GB > > Data Region = 4GB > > > > start command: > > nohup $IGNITE_HOME/bin/ignite.sh -Xmx1024m -XX:+UseG1GC > -XX:MaxDirectMemorySize=1G grtip-config.xml > ignite.log 2>&1 & > > How to adjust the the memory tunning. > > > > [21:29:22,777][SEVERE][mgmt-#33519%234-236-237-241%][GridJobWorker] > Runtime error caught during grid runnable execution: GridJobWorker > [createTime=1551360450243, startTime=1551360453071, > finishTime=1551360483944, taskNode=TcpDiscoveryNode > [id=a7839266-6396-4f7c-9ef7-c3a4b2355782, addrs=[0:0:0:0:0:0:0:1%lo, > 127.0.0.1, 192.168.1.236], sockAddrs=[/192.168.1.236:47500, > /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500], discPort=47500, order=56, > intOrder=31, lastExchangeTime=1551151457630, loc=false, > ver=2.7.0#20181201-sha1:256ae401, isClient=false], internal=true, > marsh=BinaryMarshaller [], ses=GridJobSessionImpl [ses=GridTaskSessionImpl > [taskName=o.a.i.i.v.service.VisorServiceTask, dep=GridDeployment > [ts=1551094729510, depMode=SHARED, > clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, > clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a
RE: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hi Ilya Kasnacheev,
On Ingite 2.7
No code is using affinityCall()
Haven’t enabled cache statistics
Could you please answer my questions in red below:
Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps as
it is very sensitive to free heap memory.
James>> Yes, it helps.
As I understand all the data is kept in non-heap memory, But could you advise
when the heap memory will be used, like “order”, “group by” operation ?
Looks like you have Visor node (or may be web-console) in grid. Is OOM happened
only when Visor attached to grid?
James>> Most like to occur if execute a complicate query.
Another question is: if I set Xmx=6GB, is it possible to avoide out-of-memory
failure. I regard this very critical because it make node halt.
Best Regards,
James Wang
M/WeChat: +86 135 1215 1134
From: Ilya Kasnacheev
mailto:ilya.kasnach...@gmail.com>>
Sent: Tuesday, March 5, 2019 09:44 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hello!
Is there a chance you were using affinityCall() and you're on Ignite 2.7?
We have a bug which can cause spurious OOMs here:
https://issues.apache.org/jira/browse/IGNITE-10925 - disabling cache statistics
will help
Regards,
--
Ilya Kasnacheev
сб, 2 мар. 2019 г. в 18:13, James Wang 王升平 (edvance CN)
mailto:james.w...@edvancesecurity.com>>:
Hi Support,
Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps as
it is very sensitive to free heap memory.
James>> will increase heap memory and retry.
As I understand all the data is kept in non-heap memory, But could you advise
when the heap memory will be used, like “order”, “group by” operation ?
Looks like you have Visor node (or may be web-console) in grid. Is OOM happened
only when Visor attached to grid?
James>> Most like to occur if execute a complicate query.
Another question is: if I set Xmx=6GB, is it possible to avoide out-of-memory
failure. I regard this very critical because it make node halt.
Thank you.
Best Regards,
James Wang
M/WeChat: +86 135 1215 1134
From: Andrey Mashenkov
mailto:andrey.mashen...@gmail.com>>
Sent: Friday, March 1, 2019 06:50 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hi,
Most likely heap size is too low.
Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps as
it is very sensitive to free heap memory.
Looks like you have Visor node (or may be web-console) in grid. Is OOM happened
only when Visor attached to grid?
On Fri, Mar 1, 2019 at 7:17 AM James Wang 王升平 (edvance CN)
mailto:james.w...@edvancesecurity.com>> wrote:
OS: 4C +8GB
Data Region = 4GB
start command:
nohup $IGNITE_HOME/bin/ignite.sh<http://ignite.sh> -Xmx1024m -XX:+UseG1GC
-XX:MaxDirectMemorySize=1G grtip-config.xml > ignite.log 2>&1 &
How to adjust the the memory tunning.
[21:29:22,777][SEVERE][mgmt-#33519%234-236-237-241%][GridJobWorker] Runtime
error caught during grid runnable execution: GridJobWorker
[createTime=1551360450243, startTime=1551360453071, finishTime=1551360483944,
taskNode=TcpDiscoveryNode [id=a7839266-6396-4f7c-9ef7-c3a4b2355782,
addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.1.236],
sockAddrs=[/192.168.1.236:47500<http://192.168.1.236:47500>,
/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500<http://127.0.0.1:47500>],
discPort=47500, order=56, intOrder=31, lastExchangeTime=1551151457630,
loc=false, ver=2.7.0#20181201-sha1:256ae401, isClient=false], internal=true,
marsh=BinaryMarshaller [], ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=o.a.i.i.v.service.VisorServiceTask, dep=GridDeployment
[ts=1551094729510, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6<mailto:clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6>,
clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0,
loc=true, sampleClsName=o.a.i.i.processors.cache.CachesRegistry,
pendingUndeploy=false, undeployed=false, usage=0],
taskClsName=o.a.i.i.v.service.VisorServiceTask,
sesId=8a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782,
startTime=1551360450899, endTime=9223372036854775807,
taskNodeId=a7839266-6396-4f7c-9ef7-c3a4b2355782,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6<mailto:clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6>,
closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
internal=true, topPred=ContainsNodeIdsPredicate [],
subjId=a7839266-6396-4f7c-9ef7-c3a4b2355782, mapFut=IgniteFuture
[orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=327805292]], execName=null],
jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782],
jobCtx=GridJobContextImpl
[jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782, timeoutObj=null,
attrs={}], dep=GridDeployment [ts=1551094729510, depMode=SHAR
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hello!
Is there a chance you were using affinityCall() and you're on Ignite 2.7?
We have a bug which can cause spurious OOMs here:
https://issues.apache.org/jira/browse/IGNITE-10925 - disabling cache
statistics will help
Regards,
--
Ilya Kasnacheev
сб, 2 мар. 2019 г. в 18:13, James Wang 王升平 (edvance CN) <
james.w...@edvancesecurity.com>:
> Hi Support,
>
>
>
> Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps
> as it is very sensitive to free heap memory.
>
> James>> will increase heap memory and retry.
>
> As I understand all the data is kept in non-heap memory, But could you
> advise when the heap memory will be used, like “order”, “group by”
> operation ?
>
>
>
> Looks like you have Visor node (or may be web-console) in grid. Is OOM
> happened only when Visor attached to grid?
>
> James>> Most like to occur if execute a complicate query.
>
>
>
> Another question is: if I set Xmx=6GB, is it possible to avoide
> out-of-memory failure. I regard this very critical because it make node
> halt.
>
>
>
> Thank you.
>
>
>
> Best Regards,
>
> James Wang
>
> M/WeChat: +86 135 1215 1134
>
>
>
> *From:* Andrey Mashenkov
> *Sent:* Friday, March 1, 2019 06:50 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
>
>
>
> Hi,
>
>
>
> Most likely heap size is too low.
>
> Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps
> as it is very sensitive to free heap memory.
>
>
>
> Looks like you have Visor node (or may be web-console) in grid. Is OOM
> happened only when Visor attached to grid?
>
>
>
> On Fri, Mar 1, 2019 at 7:17 AM James Wang 王升平 (edvance CN) <
> james.w...@edvancesecurity.com> wrote:
>
> OS: 4C +8GB
>
> Data Region = 4GB
>
>
>
> start command:
>
> nohup $IGNITE_HOME/bin/ignite.sh -Xmx1024m -XX:+UseG1GC
> -XX:MaxDirectMemorySize=1G grtip-config.xml > ignite.log 2>&1 &
>
> How to adjust the the memory tunning.
>
>
>
> [21:29:22,777][SEVERE][mgmt-#33519%234-236-237-241%][GridJobWorker]
> Runtime error caught during grid runnable execution: GridJobWorker
> [createTime=1551360450243, startTime=1551360453071,
> finishTime=1551360483944, taskNode=TcpDiscoveryNode
> [id=a7839266-6396-4f7c-9ef7-c3a4b2355782, addrs=[0:0:0:0:0:0:0:1%lo,
> 127.0.0.1, 192.168.1.236], sockAddrs=[/192.168.1.236:47500,
> /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500], discPort=47500, order=56,
> intOrder=31, lastExchangeTime=1551151457630, loc=false,
> ver=2.7.0#20181201-sha1:256ae401, isClient=false], internal=true,
> marsh=BinaryMarshaller [], ses=GridJobSessionImpl [ses=GridTaskSessionImpl
> [taskName=o.a.i.i.v.service.VisorServiceTask, dep=GridDeployment
> [ts=1551094729510, depMode=SHARED,
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
> clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0,
> loc=true, sampleClsName=o.a.i.i.processors.cache.CachesRegistry,
> pendingUndeploy=false, undeployed=false, usage=0],
> taskClsName=o.a.i.i.v.service.VisorServiceTask,
> sesId=8a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782,
> startTime=1551360450899, endTime=9223372036854775807,
> taskNodeId=a7839266-6396-4f7c-9ef7-c3a4b2355782,
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false,
> cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
> internal=true, topPred=ContainsNodeIdsPredicate [],
> subjId=a7839266-6396-4f7c-9ef7-c3a4b2355782, mapFut=IgniteFuture
> [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
> hash=327805292]], execName=null],
> jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782],
> jobCtx=GridJobContextImpl
> [jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782, timeoutObj=null,
> attrs={}], dep=GridDeployment [ts=1551094729510, depMode=SHARED,
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
> clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0,
> loc=true, sampleClsName=o.a.i.i.processors.cache.CachesRegistry,
> pendingUndeploy=false, undeployed=false, usage=0], finishing=true,
> masterLeaveGuard=false, timedOut=false, sysCancelled=false,
> sysStopping=false, isStarted=true, job=VisorServiceJob [], held=0,
> partsReservation=null, reqTopVer=null, execName=null]
>
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> [21:29:22,781][SEVERE][nio-acceptor-tcp-comm-#28%234-236-237-241%][] JVM
> will be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: GC overhead limit
> exceeded]]
>
> [21:29:22,814][SEVERE][grid-timeout-worker-#23%234-236-
RE: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hi Support,
Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps as
it is very sensitive to free heap memory.
James>> will increase heap memory and retry.
As I understand all the data is kept in non-heap memory, But could you advise
when the heap memory will be used, like “order”, “group by” operation ?
Looks like you have Visor node (or may be web-console) in grid. Is OOM happened
only when Visor attached to grid?
James>> Most like to occur if execute a complicate query.
Another question is: if I set Xmx=6GB, is it possible to avoide out-of-memory
failure. I regard this very critical because it make node halt.
Thank you.
Best Regards,
James Wang
M/WeChat: +86 135 1215 1134
From: Andrey Mashenkov
mailto:andrey.mashen...@gmail.com>>
Sent: Friday, March 1, 2019 06:50 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hi,
Most likely heap size is too low.
Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps as
it is very sensitive to free heap memory.
Looks like you have Visor node (or may be web-console) in grid. Is OOM happened
only when Visor attached to grid?
On Fri, Mar 1, 2019 at 7:17 AM James Wang 王升平 (edvance CN)
mailto:james.w...@edvancesecurity.com>> wrote:
OS: 4C +8GB
Data Region = 4GB
start command:
nohup $IGNITE_HOME/bin/ignite.sh<http://ignite.sh> -Xmx1024m -XX:+UseG1GC
-XX:MaxDirectMemorySize=1G grtip-config.xml > ignite.log 2>&1 &
How to adjust the the memory tunning.
[21:29:22,777][SEVERE][mgmt-#33519%234-236-237-241%][GridJobWorker] Runtime
error caught during grid runnable execution: GridJobWorker
[createTime=1551360450243, startTime=1551360453071, finishTime=1551360483944,
taskNode=TcpDiscoveryNode [id=a7839266-6396-4f7c-9ef7-c3a4b2355782,
addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.1.236],
sockAddrs=[/192.168.1.236:47500<http://192.168.1.236:47500>,
/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500<http://127.0.0.1:47500>],
discPort=47500, order=56, intOrder=31, lastExchangeTime=1551151457630,
loc=false, ver=2.7.0#20181201-sha1:256ae401, isClient=false], internal=true,
marsh=BinaryMarshaller [], ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=o.a.i.i.v.service.VisorServiceTask, dep=GridDeployment
[ts=1551094729510, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6<mailto:clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6>,
clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0,
loc=true, sampleClsName=o.a.i.i.processors.cache.CachesRegistry,
pendingUndeploy=false, undeployed=false, usage=0],
taskClsName=o.a.i.i.v.service.VisorServiceTask,
sesId=8a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782,
startTime=1551360450899, endTime=9223372036854775807,
taskNodeId=a7839266-6396-4f7c-9ef7-c3a4b2355782,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6<mailto:clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6>,
closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
internal=true, topPred=ContainsNodeIdsPredicate [],
subjId=a7839266-6396-4f7c-9ef7-c3a4b2355782, mapFut=IgniteFuture
[orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=327805292]], execName=null],
jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782],
jobCtx=GridJobContextImpl
[jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782, timeoutObj=null,
attrs={}], dep=GridDeployment [ts=1551094729510, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6<mailto:clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6>,
clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0,
loc=true, sampleClsName=o.a.i.i.processors.cache.CachesRegistry,
pendingUndeploy=false, undeployed=false, usage=0], finishing=true,
masterLeaveGuard=false, timedOut=false, sysCancelled=false, sysStopping=false,
isStarted=true, job=VisorServiceJob [], held=0, partsReservation=null,
reqTopVer=null, execName=null]
java.lang.OutOfMemoryError: GC overhead limit exceeded
[21:29:22,781][SEVERE][nio-acceptor-tcp-comm-#28%234-236-237-241%][] JVM will
be halted immediately due to the failure: [failureCtx=FailureContext
[type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: GC overhead limit
exceeded]]
[21:29:22,814][SEVERE][grid-timeout-worker-#23%234-236-237-241%][GridTimeoutProcessor]
Error when executing timeout callback: CancelableTask
[id=94ef2742961-0742c515-5b96-4aad-b07a-9f1ec60af5f3, endTime=1551360514795,
period=3000, cancel=false, task=MetricsUpdater [prevGcTime=38806,
prevCpuTime=2195727,
super=o.a.i.i.managers.discovery.GridDiscoveryManager$MetricsUpdater@73766331<mailto:super=o.a.i.i.managers.discovery.GridDiscoveryManager$MetricsUpdater@73766331>]]
java.lang.OutOfMemoryError: GC overhead limit exceeded
James Wang / edvance
Mobile/WeChat: +86 135 1215 1134
This message contains information that
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Hi,
Most likely heap size is too low.
Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps
as it is very sensitive to free heap memory.
Looks like you have Visor node (or may be web-console) in grid. Is OOM
happened only when Visor attached to grid?
On Fri, Mar 1, 2019 at 7:17 AM James Wang 王升平 (edvance CN) <
james.w...@edvancesecurity.com> wrote:
> OS: 4C +8GB
> Data Region = 4GB
>
> start command:
> nohup $IGNITE_HOME/bin/ignite.sh -Xmx1024m -XX:+UseG1GC
> -XX:MaxDirectMemorySize=1G grtip-config.xml > ignite.log 2>&1 &
>
> How to adjust the the memory tunning.
>
> [21:29:22,777][SEVERE][mgmt-#33519%234-236-237-241%][GridJobWorker]
> Runtime error caught during grid runnable execution: GridJobWorker
> [createTime=1551360450243, startTime=1551360453071,
> finishTime=1551360483944, taskNode=TcpDiscoveryNode
> [id=a7839266-6396-4f7c-9ef7-c3a4b2355782, addrs=[0:0:0:0:0:0:0:1%lo,
> 127.0.0.1, 192.168.1.236], sockAddrs=[/192.168.1.236:47500,
> /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500], discPort=47500, order=56,
> intOrder=31, lastExchangeTime=1551151457630, loc=false,
> ver=2.7.0#20181201-sha1:256ae401, isClient=false], internal=true,
> marsh=BinaryMarshaller [], ses=GridJobSessionImpl [ses=GridTaskSessionImpl
> [taskName=o.a.i.i.v.service.VisorServiceTask, dep=GridDeployment
> [ts=1551094729510, depMode=SHARED,
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
> clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0,
> loc=true, sampleClsName=o.a.i.i.processors.cache.CachesRegistry,
> pendingUndeploy=false, undeployed=false, usage=0],
> taskClsName=o.a.i.i.v.service.VisorServiceTask,
> sesId=8a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782,
> startTime=1551360450899, endTime=9223372036854775807,
> taskNodeId=a7839266-6396-4f7c-9ef7-c3a4b2355782,
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false,
> cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
> internal=true, topPred=ContainsNodeIdsPredicate [],
> subjId=a7839266-6396-4f7c-9ef7-c3a4b2355782, mapFut=IgniteFuture
> [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
> hash=327805292]], execName=null],
> jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782],
> jobCtx=GridJobContextImpl
> [jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782, timeoutObj=null,
> attrs={}], dep=GridDeployment [ts=1551094729510, depMode=SHARED,
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
> clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0,
> loc=true, sampleClsName=o.a.i.i.processors.cache.CachesRegistry,
> pendingUndeploy=false, undeployed=false, usage=0], finishing=true,
> masterLeaveGuard=false, timedOut=false, sysCancelled=false,
> sysStopping=false, isStarted=true, job=VisorServiceJob [], held=0,
> partsReservation=null, reqTopVer=null, execName=null]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [21:29:22,781][SEVERE][nio-acceptor-tcp-comm-#28%234-236-237-241%][] JVM
> will be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: GC overhead limit
> exceeded]]
> [21:29:22,814][SEVERE][grid-timeout-worker-#23%234-236-237-241%][GridTimeoutProcessor]
> Error when executing timeout callback: CancelableTask
> [id=94ef2742961-0742c515-5b96-4aad-b07a-9f1ec60af5f3,
> endTime=1551360514795, period=3000, cancel=false, task=MetricsUpdater
> [prevGcTime=38806, prevCpuTime=2195727,
> super=o.a.i.i.managers.discovery.GridDiscoveryManager$MetricsUpdater@73766331
> ]]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
>
> James Wang / edvance
> Mobile/WeChat: +86 135 1215 1134
>
> This message contains information that is deemed confidential and
> privileged. Unless you are the addressee (or authorized to receive for the
> addressee), you may not use, copy or disclose to anyone the message or any
> information contained in the message. If you have received the message in
> error, please advise the sender by reply e-mail and delete the message.
>
--
Best regards,
Andrey V. Mashenkov
java.lang.OutOfMemoryError: GC overhead limit exceeded
OS: 4C +8GB
Data Region = 4GB
start command:
nohup $IGNITE_HOME/bin/ignite.sh -Xmx1024m -XX:+UseG1GC
-XX:MaxDirectMemorySize=1G grtip-config.xml > ignite.log 2>&1 &
How to adjust the the memory tunning.
[21:29:22,777][SEVERE][mgmt-#33519%234-236-237-241%][GridJobWorker] Runtime
error caught during grid runnable execution: GridJobWorker
[createTime=1551360450243, startTime=1551360453071, finishTime=1551360483944,
taskNode=TcpDiscoveryNode [id=a7839266-6396-4f7c-9ef7-c3a4b2355782,
addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.1.236],
sockAddrs=[/192.168.1.236:47500, /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500],
discPort=47500, order=56, intOrder=31, lastExchangeTime=1551151457630,
loc=false, ver=2.7.0#20181201-sha1:256ae401, isClient=false], internal=true,
marsh=BinaryMarshaller [], ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=o.a.i.i.v.service.VisorServiceTask, dep=GridDeployment
[ts=1551094729510, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0, loc=true,
sampleClsName=o.a.i.i.processors.cache.CachesRegistry, pendingUndeploy=false,
undeployed=false, usage=0], taskClsName=o.a.i.i.v.service.VisorServiceTask,
sesId=8a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782,
startTime=1551360450899, endTime=9223372036854775807,
taskNodeId=a7839266-6396-4f7c-9ef7-c3a4b2355782,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false, cpSpi=null,
failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=true,
topPred=ContainsNodeIdsPredicate [],
subjId=a7839266-6396-4f7c-9ef7-c3a4b2355782, mapFut=IgniteFuture
[orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=327805292]], execName=null],
jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782],
jobCtx=GridJobContextImpl
[jobId=9a35b792961-a7839266-6396-4f7c-9ef7-c3a4b2355782, timeoutObj=null,
attrs={}], dep=GridDeployment [ts=1551094729510, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
clsLdrId=3aef2742961-5a63d018-0eb4-493e-91a4-be6d41caff85, userVer=0, loc=true,
sampleClsName=o.a.i.i.processors.cache.CachesRegistry, pendingUndeploy=false,
undeployed=false, usage=0], finishing=true, masterLeaveGuard=false,
timedOut=false, sysCancelled=false, sysStopping=false, isStarted=true,
job=VisorServiceJob [], held=0, partsReservation=null, reqTopVer=null,
execName=null]
java.lang.OutOfMemoryError: GC overhead limit exceeded
[21:29:22,781][SEVERE][nio-acceptor-tcp-comm-#28%234-236-237-241%][] JVM will
be halted immediately due to the failure: [failureCtx=FailureContext
[type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: GC overhead limit
exceeded]]
[21:29:22,814][SEVERE][grid-timeout-worker-#23%234-236-237-241%][GridTimeoutProcessor]
Error when executing timeout callback: CancelableTask
[id=94ef2742961-0742c515-5b96-4aad-b07a-9f1ec60af5f3, endTime=1551360514795,
period=3000, cancel=false, task=MetricsUpdater [prevGcTime=38806,
prevCpuTime=2195727,
super=o.a.i.i.managers.discovery.GridDiscoveryManager$MetricsUpdater@73766331]]
java.lang.OutOfMemoryError: GC overhead limit exceeded
James Wang / edvance
Mobile/WeChat: +86 135 1215 1134
This message contains information that is deemed confidential and privileged.
Unless you are the addressee (or authorized to receive for the addressee), you
may not use, copy or disclose to anyone the message or any information
contained in the message. If you have received the message in error, please
advise the sender by reply e-mail and delete the message.
