At this moments java hungs. Only one thread is work and it run mostly in OS
core, with follow trace:
[pid 1953] 0.050157 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE,
1) = 0 <0.000022>
[pid 1953] 0.000059 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202329,
797618000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050093>
[pid 1953] 0.050152 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000021>
[pid 1953] 0.000067 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202329,
847838000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050090>
[pid 1953] 0.050150 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000022>
[pid 1953] 0.000067 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202329,
898054000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050086>
[pid 1953] 0.050144 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000022>
[pid 1953] 0.000060 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202329,
948258000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050085>
[pid 1953] 0.050144 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000021>
[pid 1953] 0.000067 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202329,
998469000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050067>
[pid 1953] 0.050127 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000021>
[pid 1953] 0.000067 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
48664000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050102>
[pid 1953] 0.050161 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000021>
[pid 1953] 0.000059 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
98884000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050102>
[pid 1953] 0.050160 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000022>
[pid 1953] 0.000067 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
149111000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050097>
[pid 1953] 0.050157 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000022>
[pid 1953] 0.000059 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
199327000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050093>
[pid 1953] 0.050153 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000022>
[pid 1953] 0.000067 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
249547000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050095>
[pid 1953] 0.050155 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000022>
[pid 1953] 0.000059 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
299761000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050094>
[pid 1953] 0.050154 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000021>
[pid 1953] 0.000067 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
349981000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050092>
[pid 1953] 0.050168 futex(0x7fbe141ea428, FUTEX_WAKE_PRIVATE, 1)
= 0 <0.000023>
[pid 1953] 0.000066 futex(0x7fbc24023794,
FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, {1300202330,
400216000}, ffffffff) = -1 ETIMEDOUT (Connection timed out) <0.050090>
And this happens when mmap disck access is on, and in my case when VIRTUAL
space of java process is greate then 16G. It that case all system work
badly, utilities launch very slow (but not any swap activity), when kill
java process all system functionality back. What is that i don know,
perhaps this is OS depend i use Ubuntu 10.0.4(LTS)
Linux slv007 2.6.32-24-generic #43-Ubuntu SMP Thu Sep 16 14:58:24 UTC 2010
x86_64 GNU/Linux
2011/3/17 Narendra Sharma <narendra.sha...@gmail.com>
> Depending on your memtable thresholds the heap may be too small for the
> deployment. At the same time I don't see any other log statements around
> that long pause that you have shown in the log snippet. It looks little odd
> to me. All the ParNew collected almost same amount of heap and did not take
> lot of time.
>
> Check if it is due to some JVM bug.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6477891
>
> -Naren
>
>
> On Thu, Mar 17, 2011 at 9:47 AM, ruslan usifov <ruslan.usi...@gmail.com>wrote:
>
>>
>>
>> 2011/3/17 Narendra Sharma <narendra.sha...@gmail.com>
>>
>>> What heap size are you running with? and Which version of Cassandra?
>>>
>>> 4G with cassandra 0.7.4
>>
>
>
