Hello Ivan, I will investigate the kernel topic.
When referring to buffers size, do you mean the Sedna setting? bufs-num N ? The present configuration is: -bufs-num 3200 -data-file-init-size 5000 -upd-crt .1 We moved Sedna to ubuntu 11.10, which after doing some research may have kernel related issues. We thought the issue could be a Sedna related after problem appeared when migrating to Sedna 3.5.135. The process is limited to 5 concurrent connections. Thanks you very much for the insight, Malcolm -----Original Message----- From: Ivan Shcheklein [mailto:[email protected]] Sent: Tuesday, January 10, 2012 12:40 PM To: Malcolm Davis Cc: [email protected] Subject: Re: [Sedna-discussion] SEDNA diagnostics Hi Malcolm, Most likely it's not Sedna related (though you have a lot of pressure on memory, try to decrease buffers size or number of sessions): "kernel: [60749.214946] BUG: unable to handle kernel paging request at ffff88001085b180" - this is definitely problem of this kernel on Amazon hardware. I've tried to google: "amazon BUG unable to handle kernel paging request" and it returns a lot of references to the similar problems: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/884320 http://serverfault.com/questions/249171/how-can-i-close-a-port-that-appears- to-be-orphaned-by-xvfb ... Usually, kernel update helps. Ivan Shcheklein, Sedna Team Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.214946] BUG: unable to handle kernel paging request at ffff88001085b180 On Mon, Jan 9, 2012 at 9:08 AM, Malcolm Davis <[email protected]> wrote: Is there a serious of steps, log files, etc. that would be helpful to provide to diagnose issues? (Sorry, I'm sure I have asked the diagnose question before). I am having an issue with a specific client system. SEDNA seems to have died. SEDNA is not responding to any request. (If I rebuild the system from scratch using the same data and procedures, the system dies in about the same spot in processing) The last line of the SEDNA event.log LOG 08/01/2012 17:59:10 (TRN xrdb pid=2693 sid=47 trid=-1) [tr_common_funcs.cpp:on_transaction_end:414]: Transaction has been COMMITED LOG 08/01/2012 17:59:10 (TRN xrdb pid=2693 sid=47 trid=43) [tr_functions.cpp:on_user_statement_begin:102]: User query: --- update insert (the insert query has be redacted. The line is valid, and I pushed the last requests through other systems.) free -t -m total used free shared buffers cached Mem: 592 578 13 0 8 455 -/+ buffers/cache: 115 476 Swap: 0 0 0 Total: 592 578 13 kern.log files before and after the problem occurs : Jan 8 01:07:04 ip-10-244-50-141 kernel: [ 21.248045] eth0: no IPv6 routers present Jan 8 01:07:08 ip-10-244-50-141 kernel: [ 24.387476] init: plymouth-upstart-bridge main process (527) killed by TERM signal Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.214946] BUG: unable to handle kernel paging request at ffff88001085b180 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.214966] IP: [<ffffffff81006c25>] xen_set_pte+0x25/0xe0 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.214979] PGD 1c04067 PUD 1c08067 PMD 2662e067 PTE 801000001085b065 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.214991] Oops: 0003 [#1] SMP Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.214998] CPU 0 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215001] Modules linked in: acpiphp Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215009] Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215013] Pid: 2687, comm: se_trn Not tainted 3.0.0-12-virtual #20-Ubuntu Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215022] RIP: e030:[<ffffffff81006c25>] [<ffffffff81006c25>] xen_set_pte+0x25/0xe0 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215030] RSP: e02b:ffff88002005bcb8 EFLAGS: 00010297 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215035] RAX: 0000000000000000 RBX: ffff88001085b180 RCX: 80000003cef13027 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215041] RDX: 0000000000000000 RSI: 80000003cef13027 RDI: ffff88001085b180 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215047] RBP: ffff88002005bcd8 R08: ffffea000039d3f8 R09: 00003ffffffff000 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215052] R10: 0000000000000000 R11: 0000000000000246 R12: 80000003cef13027 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215058] R13: 80000003cef13027 R14: 00007f25f4630000 R15: 0000000000000000 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215066] FS: 00007f261c5c3740(0000) GS:ffff88002657b000(0000) knlGS:0000000000000000 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215074] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215079] CR2: ffff88001085b180 CR3: 00000000216ac000 CR4: 0000000000002620 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215085] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215091] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000000 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215098] Process se_trn (pid: 2687, threadinfo ffff88002005a000, task ffff8800024c4500) Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215104] Stack: Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215107] 0000000000000000 00003ffffffff000 ffff88000a9fb000 ffffea00007182f8 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215119] ffff88002005bce8 ffffffff81006cf4 ffff88002005bd78 ffffffff8112aa2b Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215129] 000000000000000a 0000000000000000 0000020000000000 ffff88001085b180 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215140] Call Trace: Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215147] [<ffffffff81006cf4>] xen_set_pte_at+0x14/0x20 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215156] [<ffffffff8112aa2b>] __do_fault+0x22b/0x510 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215162] [<ffffffff8112df1a>] handle_pte_fault+0xfa/0x210 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215169] [<ffffffff81005cce>] ? xen_pmd_val+0xe/0x10 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215175] [<ffffffff81004759>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215224] [<ffffffff8112e3e8>] handle_mm_fault+0x1f8/0x350 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215232] [<ffffffff81073dbb>] ? set_current_blocked+0x5b/0x70 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215241] [<ffffffff816070ce>] do_page_fault+0x14e/0x530 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215249] [<ffffffff81603dd5>] page_fault+0x25/0x30 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215253] Code: 84 00 00 00 00 00 55 48 89 e5 48 83 ec 20 48 89 5d f0 4c 89 65 f8 66 66 66 66 90 48 89 fb 49 89 f4 e8 60 ba 02 00 83 f8 01 74 13 <4c> 89 23 48 8b 5d f0 4c 8b 65 f8 c9 c3 66 0f 1f 44 00 00 ff 14 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215340] RIP [<ffffffff81006c25>] xen_set_pte+0x25/0xe0 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215347] RSP <ffff88002005bcb8> Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215351] CR2: ffff88001085b180 Jan 8 17:59:13 ip-10-244-50-141 kernel: [60749.215359] ---[ end trace bac898b9a084c063 ]--- Thanks, Malcolm ---------------------------------------------------------------------------- -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox _______________________________________________ Sedna-discussion mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sedna-discussion ------------------------------------------------------------------------------ Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox _______________________________________________ Sedna-discussion mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sedna-discussion
