On 25 February 2015 at 12:11, Daniel Iwan <iwan.dan...@gmail.com> wrote:
> Hi > > I've checked all logs and there is nothing regarding memory issues. > Since then I've had several Riak crashes but looks like other processes are > failing as well > > Feb 2 22:05:28 node2 kernel: [20052.901884] beam.smp[1830]: segfault at > 80000523111 ip 0000080000523111 sp 00007f03ba821be8 error 14 in > 000013.log[7f01b6890000+1400000] > Feb 4 10:16:01 node2 kernel: [150082.149742] MegaCli[29635]: segfault at > 80000549b14 ip 0000080000549b14 sp 00007fff3c5f7840 error 14 in > libtinfo.so.5.9[7f3ee87ba000+22000] > Feb 4 16:35:27 node2 kernel: [172812.410113] MegaCli[23019]: segfault at > 800005473d6 ip 00000800005473d6 sp 00007fffc8f49f20 error 14 in > libtinfo.so.5.9[7fa9e90c3000+22000] > Feb 6 14:50:38 node2 kernel: [339062.483587] sh[24637]: segfault at > 8000040460d ip 000008000040460d sp 00007fff35251160 error 14 in > libc-2.15.so[7fd4cc395000+1b5000] > Feb 7 11:59:32 node2 kernel: [415077.342034] df[6393]: segfault at > 800004029a0 ip 00000800004029a0 sp 00007fff758406d0 error 14 in > libc-2.15.so[7f5433243000+1b5000] > > Feb 8 10:04:31 node2 kernel: [494451.877635] df[22107]: segfault at > 80000404b00 ip 0000080000404b00 sp 00007ffffeee1b08 error 14 in > libc-2.15.so[7fbd82ce2000+1b5000] > Feb 9 16:30:20 node2 kernel: [603829.476142] ls[2873]: segfault at > 8000040d04e ip 000008000040d04e sp 00007fffaff38c60 error 14 in > libnss_files-2.15.so[7f257c9c4000+c000] > Feb 9 18:26:13 node2 kernel: [ 6503.710549] beam.smp[2140]: segfault at > 80000523a00 ip 0000080000523a00 sp 00007f3955ff2d80 error 14 in > 000006.log[7f377f27b000+1400000] > Feb 10 17:34:46 node2 kernel: [36949.199740] beam.smp[1877]: segfault at > 800005650b2 ip 00000800005650b2 sp 00007faba120fa70 error 14 in > 000009.log[7fa99827c000+1400000] > Feb 11 20:37:15 node2 kernel: [134145.969112] beam.smp[7276]: segfault at > 8000052287e ip 000008000052287e sp 00007ff8625c9be0 error 14 in > 000012.log[7ff66f703000+1400000] > Feb 13 08:58:57 node2 kernel: [ 6414.659327] beam.smp[1877]: segfault at > 80000569cfc ip 0000080000569cfc sp 00007f55aa48bab0 error 14 in > 000012.log[7f537dc0e000+1400000] > Feb 15 03:20:30 node2 kernel: [133707.360153] MegaCli[7442]: segfault at > 800005473d6 ip 00000800005473d6 sp 00007fff39ba1570 error 14 in > libtinfo.so.5.9[7f2728f79000+22000] > > Feb 15 10:02:23 node2 kernel: [157782.787481] beam.smp[2023]: segfault at > 800005239d0 ip 00000800005239d0 sp 00007f47e3fe6d68 error 14 in > 000061.log[7f463e32f000+1400000] > Feb 16 17:30:18 node2 kernel: [270880.717532] console-kit-dae[1548]: > segfault at800004123e8 ip 00000800004123e8 sp 00007fffc6d8c0c0 error 14 > Feb 16 19:31:45 node2 kernel: [278156.348900] beam.smp[16617]: segfault at > 800005650b2 ip 00000800005650b2 sp 00007f3dbe74ba70 error 14 in > 000019.log[7f3b89f65000+1400000] > Feb 16 21:45:34 node2 kernel: [286172.695110] sh[12432]: segfault at > 8000040460d ip 000008000040460d sp 00007fffe6e2b3b0 error 14 in > libc-2.15.so[7f9b57c77000+1b5000] > Feb 17 07:27:23 node2 kernel: [ 457.418215] beam.smp[1824]: segfault at > 80000523111 ip 0000080000523111 sp 00007f36e9574be8 error 14 in > 000021.log[7f34c24f3000+1400000] > > Feb 25 10:46:04 node2 kernel: [702478.037041] beam.smp[8832]: segfault at > 80000522980 ip 0000080000522980 sp 00007fe8bbffede8 error 14 in > 000006.log[7fe713e2c000+1400000] > > Riak is always touching anti-entropy files, like in the last example: > > > /var/lib/riak/anti_entropy/22835963083295358096932575511191922182123945984/000006.log > > Could it be an SSD failing? > > Daniel > > Hi Daniel, Random segfaults in all sorts of different programs and libraries are a strong indicator of hardware failure, most likely memory failure. You might want to check the memory modules of your server using memtest86 ( http://www.memtest86.com). For a an online tool explaining the different segfault error codes, please have a look at http://rgeissert.blogspot.com/p/segmentation-fault-error.html Regards, Magnus
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
