We are getting these warnings quite often

```
./ignite-3fb86334.0.log:[09:48:02,663][WARNING][grid-timeout-worker-#71][IgniteKernal]
Possible thread pool starvation detected (no task completed in last 30000ms,
is query thread pool size large enough?)
./ignite-3fb86334.5.log:[03:50:26,683][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54439
./ignite-3fb86334.5.log:[03:50:56,687][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54440
./ignite-3fb86334.5.log:[03:51:26,690][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54441
./ignite-3fb86334.5.log:[03:51:26,692][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54442
./ignite-3fb86334.5.log:[03:51:56,695][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54443
./ignite-3fb86334.5.log:[03:52:24,302][WARNING][grid-timeout-worker-#71][diagnostic]
First 10 long running cache futures [total=1]
54444
./ignite-3fb86334.5.log:[03:52:24,302][WARNING][grid-timeout-worker-#71][diagnostic]
>>> Future [startTime=03:49:57.774, curTime=03:52:24.237,
fut=GridDhtAtomicUpdateFuture [updateCntr=49148, supe     
r=GridDhtAtomicAbstractUpdateFuture [futId=158766398, resCnt=0,
addedReader=false, dhtRes=TransformMapView
{43935ea9-e7cc-43d7-8895-3b16fc865f0a=[res=false, size=49148,
nearSize=0]}]]]
54445
./ignite-3fb86334.5.log:[03:52:26,702][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54446
./ignite-3fb86334.5.log:[04:02:26,797][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54447
./ignite-3fb86334.5.log:[04:17:24,415][WARNING][grid-timeout-worker-#71][diagnostic]
First 10 long running cache futures [total=1]
54448
./ignite-3fb86334.5.log:[04:17:24,415][WARNING][grid-timeout-worker-#71][diagnostic]
>>> Future [startTime=04:16:21.004, curTime=04:17:24.412,
fut=GridDhtAtomicUpdateFuture [updateCntr=49100, supe     
r=GridDhtAtomicAbstractUpdateFuture [futId=159106604, resCnt=0,
addedReader=false, dhtRes=TransformMapView
{43935ea9-e7cc-43d7-8895-3b16fc865f0a=[res=false, size=49100,
nearSize=0]}]]]
54449
./ignite-3fb86334.5.log:[04:46:57,195][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54450
./ignite-3fb86334.5.log:[04:55:05,029][WARNING][sys-stripe-1-#2][GridContinuousProcessor]
Failed to wait for ack message. [node=92ef74a3-d6c6-48e2-a786-67dd5cbe2132,
routine=0501eb73-1217-446b-8a9      6-4574a2483cc6]
54451
./ignite-3fb86334.5.log:[05:01:47,172][WARNING][checkpoint-runner-#222][GridCacheDatabaseSharedManager]
1 checkpoint pages were not written yet due to unsuccessful page write lock
acquisition and       will be retried
54452
./ignite-3fb86334.5.log:[05:01:57,451][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54453
./ignite-3fb86334.5.log:[05:15:29,983][WARNING][jvm-pause-detector-worker][IgniteKernal]
Possible too long JVM pause: 792 milliseconds.
54454
./ignite-3fb86334.5.log:[05:16:57,678][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54455
./ignite-3fb86334.5.log:[05:17:57,694][WARNING][grid-timeout-worker-#71][G]
>>> Possible starvation in striped pool.
54456
./ignite-3fb86334.5.log:[05:25:45,857][WARNING][sys-stripe-0-#1][GridContinuousProcessor]
Failed to wait for ack message. [node=92ef74a3-d6c6-48e2-a786-67dd5cbe2132,
routine=0501eb73-1217-446b-8a9      6-4574a2483cc6]
```


The setup ->

Two client servers which run 500 threads each simultaneously without locks.
Eagerly query Ignite DB.
Two DB servers with 30GB RAM (20GB off heap) and 2.3GHz - 64 core CPUs.

Important points:
-> Maximum we can get around 300 connections at a time to DB servers.
-> All queries are optimised.
-> All the caches are in Replication mode with FULL write sync. we use SQL
querying too.
-> In both DB servers, all thread pools sizes are default ones the value
which is of number of CPU cores.

We have done almost all kinds of performance optmizations mentioned in
Ignite docs but still facing these warnings. If we need to withstand a lot
of load like the above scenario, what we should we be doing?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Reply via email to