Re: [Xen-devel] Faulting linear address??
On Mon, 2017-09-11 at 02:00 +0900, Minjun Hong wrote: > I made the new Xen4.5 binary with 'debug=y' option that I modified > and install it. > Then, there was a kernel panic caused by the debugging code triggered > by 'debug=y' during booting process(of dom0): > Once again, can you show us here what you are changing? > (XEN) [ Xen-4.5.0 x86_64 debug=y Not tainted ] > (XEN) CPU: 7 > (XEN) RIP: e008:[] vcpu_migrate+0x1bd/0x374 > (XEN) RFLAGS: 00010096 CONTEXT: hypervisor > [...] > (XEN) Xen call trace: > (XEN) [] vcpu_migrate+0x1bd/0x374 > (XEN) [] vcpu_force_reschedule+0x9e/0xa7 > (XEN) [] do_vcpu_op+0x2e7/0x69d > (XEN) [] syscall_enter+0xeb/0x145 > (XEN) > (XEN) Pagetable walk from 82d081422020: > (XEN) L4[0x105] = 86092063 > (XEN) L3[0x142] = 8608f063 > (XEN) L2[0x00a] = > (XEN) > (XEN) > (XEN) Panic on CPU 7: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=] > (XEN) Faulting linear address: 82d081422020 > (XEN) > > Because I received a solution from my professor, I think it is a hard > work to change Xen version. > This makes it a bit harder for us to give effective advices, but if you really can't move forward, then fine, we still can at least try. > Anyway, even if I turned on the 'debug=y' option, I could not get > accurate information like with 'debug=n'; I get only linear > address(82d081422020). > Well, the difference is that now, if Xen is compiled with frame pointers, we are (much more) sure that the stack trace is accurate, i.e., about where the problem is actually happening, and how you got there. > So, I want to use a dis-assembly utility like 'addr2line' or > 'objdump', which binaries can I use as input to the utility? > I'm using Ubuntu and previously I used '/boot/xen-syms-4.5.0' as > input to the utilities. > Yes, if that is the binary of the hypervisor you compiled (with debug=y), that's what you should use. You should also have it, in the source tree, as xen/xen-syms. > But I could get wrong information, which told me a code line that is > never related this problem. > The address you pass to addr2line is not the 'Faulting linear address'. It must be the address of the instruction that was being executing when the exception occurred. IOW, you shall use 82d08012ba6c, from here: (XEN)[] vcpu_migrate+0x1bd/0x374 Which, in fact, is the address present in the program counter register (RIP): (XEN) RIP:e008:[] vcpu_migrate+0x1bd/0x374 > I know that a beginner in the Xen developer community like me might > be annoying you, but I ask you one more help. > You being a beginner is not a problem. :-) The biggest problem I have right now, while trying to help you, is that I can't see what you are doing, and how you're changing the code. Regards, Dario -- <> (Raistlin Majere) - Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) signature.asc Description: This is a digitally signed message part ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Faulting linear address??
This is late reply but, thank you for your kind reply, Dario Faggioli. I made the new Xen4.5 binary with 'debug=y' option that I modified and install it. Then, there was a kernel panic caused by the debugging code triggered by 'debug=y' during booting process(of dom0): (XEN) [ Xen-4.5.0 x86_64 debug=y Not tainted ] (XEN) CPU:7 (XEN) RIP:e008:[] vcpu_migrate+0x1bd/0x374 (XEN) RFLAGS: 00010096 CONTEXT: hypervisor (XEN) rax: 82d080413320 rbx: 830461c3e068 rcx: 82d080428e60 (XEN) rdx: 00201da0 rsi: 830086d2f000 rdi: 82d080280bc0 (XEN) rbp: 83045e77fe48 rsp: 83045e77fdd8 r8: 0007 (XEN) r9: deadbeef r10: 82d08024d870 r11: 0246 (XEN) r12: 830461c3e068 r13: 0007 r14: 82d080428e60 (XEN) r15: 830461c3e068 cr0: 80050033 cr4: 003526f0 (XEN) cr3: 000459c0c000 cr2: 82d081422020 (XEN) ds: 002b es: 002b fs: gs: ss: e010 cs: e008 (XEN) Xen stack trace from rsp=83045e77fdd8: (XEN)82d080428e60 82d080428e60 83045e77fdf8 82d080428e60 (XEN)83045e77fe00 830086d2f000 00201da00086 0246 (XEN)82d08012e0c6 830086d2f000 830461c3e068 82d080428e60 (XEN)82d080413320 83045e78a000 83045e77fe78 82d08012be23 (XEN)0007 830086d2f000 (XEN)83045e77fef8 82d080107052 0008 (XEN)00a0fb00 01000c02 (XEN)83045e77fe32 82d080196da9 0f01 830086d2f000 (XEN)880456bcc4c0 0007 (XEN)7cfba18800c7 82d080234d9b 8100130a 0018 (XEN) 0002 01000c02 880450b43e3c (XEN)880450b43e88 880456bcc4c0 0246 0004 (XEN) 0004 0018 8100130a (XEN) 0007 0007 00010100 (XEN)8100130a e033 0246 880450b43e70 (XEN)e02b (XEN) 0007 830086d2f000 0033e1815200 (XEN) (XEN) Xen call trace: (XEN)[] vcpu_migrate+0x1bd/0x374 (XEN)[] vcpu_force_reschedule+0x9e/0xa7 (XEN)[] do_vcpu_op+0x2e7/0x69d (XEN)[] syscall_enter+0xeb/0x145 (XEN) (XEN) Pagetable walk from 82d081422020: (XEN) L4[0x105] = 86092063 (XEN) L3[0x142] = 8608f063 (XEN) L2[0x00a] = (XEN) (XEN) (XEN) Panic on CPU 7: (XEN) FATAL PAGE FAULT (XEN) [error_code=] (XEN) Faulting linear address: 82d081422020 (XEN) Because I received a solution from my professor, I think it is a hard work to change Xen version. Anyway, even if I turned on the 'debug=y' option, I could not get accurate information like with 'debug=n'; I get only linear address(82d081422020). So, I want to use a dis-assembly utility like 'addr2line' or 'objdump', which binaries can I use as input to the utility? I'm using Ubuntu and previously I used '/boot/xen-syms-4.5.0' as input to the utilities. But I could get wrong information, which told me a code line that is never related this problem. I know that a beginner in the Xen developer community like me might be annoying you, but I ask you one more help. Thanks for your help again. On Fri, Sep 8, 2017 at 4:40 PM, Dario Faggioli wrote: > On Fri, 2017-09-08 at 14:33 +0900, Minjun Hong wrote: > > 1) I worked on the scheduler(credit scheduler) and I had a kernel > > panic by my modification. > > > To do what? > > Can you show here what you are changing, e.g., by putting together a > quick patch? > > It does not have to be properly formatted and follow all the rules of a > proper patch submission... it can just be a diff against original code, > to understand what you are changing. > > > 2) I tried to get any information for debugging so that, I used > > serial console and could gain the serial logs like following: > > > > (XEN) [ Xen-4.5.0 x86_64 debug=n Not tainted ] > > > First of all, when debugging, you should use a debug hypervisor, i.e., > a build of Xen, done with 'debug=y', or in general, with debug enabled. > > Also, Xen-4.5.0. Can you move to a more recent version? > > > (XEN) CPU:2 > > (XEN) RIP:e008:[] csched_schedule+0x373/0x1180 > > (XEN) RFLAGS: 00010086 CONTEXT: hypervisor > > [..] > > (XEN) Xen call trace: > > (XEN)[] > > csched_schedule+0x373/0x1180 > > (XEN)[] schedule+0xf3/0x590 > > (XEN)[] reprogram_timer+0x75/0xe0 > > (XEN)[] timer_softirq_action+0x13e/0x210 > > (XEN)[] __do
Re: [Xen-devel] Faulting linear address??
On Fri, 2017-09-08 at 14:33 +0900, Minjun Hong wrote: > 1) I worked on the scheduler(credit scheduler) and I had a kernel > panic by my modification. > To do what? Can you show here what you are changing, e.g., by putting together a quick patch? It does not have to be properly formatted and follow all the rules of a proper patch submission... it can just be a diff against original code, to understand what you are changing. > 2) I tried to get any information for debugging so that, I used > serial console and could gain the serial logs like following: > > (XEN) [ Xen-4.5.0 x86_64 debug=n Not tainted ] > First of all, when debugging, you should use a debug hypervisor, i.e., a build of Xen, done with 'debug=y', or in general, with debug enabled. Also, Xen-4.5.0. Can you move to a more recent version? > (XEN) CPU: 2 > (XEN) RIP: e008:[] csched_schedule+0x373/0x1180 > (XEN) RFLAGS: 00010086 CONTEXT: hypervisor > [..] > (XEN) Xen call trace: > (XEN) [] > csched_schedule+0x373/0x1180 > (XEN) [] schedule+0xf3/0x590 > (XEN) [] reprogram_timer+0x75/0xe0 > (XEN) [] timer_softirq_action+0x13e/0x210 > (XEN) [] __do_softirq+0x7c/0xd0 > (XEN) [] idle_loop+0x3a/0x70 > Again, use a debug hypervisor, compiled with frame pointers. > (XEN) Pagetable walk from 830088002c98: > (XEN) L4[0x106] = 86075063 > (XEN) L3[0x002] = 86071063 > (XEN) L2[0x040] = > (XEN) > (XEN) > (XEN) Panic on CPU 2: > (XEN) FATAL PAGE FAULT > (XEN) [error_code=] > (XEN) Faulting linear address: 830088002c98 > (XEN) > I want to know where I should start debugging from. > However, although I'm using serial console, I could get not enough > clues only from the kernel log: > 1) I could figure out what line and file caused the panic by its call > trace, but it is too rough so it does not help me. > That's exactly from where you usually start: looking at what's at the instruction that cause the system to explode, in your case, at address 0x82d080120973. You can figure that out by disassembling the Xen hypervisor binary, with `objdump', and looking up that address. Or you can use addr2line, to have an indication of the same thing, but in the C sources. I'm not sure what you mean with "it is too rough". At least the address of the instruction that caused the system to fail in the Xen binary, is usually pretty accurate (then, of course, you have to look at surrounding instructions, check how you got there, etc.). Again, make sure you use a debug hypervisor. > 2) What linear address brings about this situation; 'Faulting linear > address', but it is just an address and not recognizable something > that human cannot read. > > I think, literally, the 'Faulting linear address' is key point > because I heard that it represents bad address that I should never > access. > It is, but the only way of understanding why you hit such an access violation, is understand what the code is doing when it happens. Regards, Dario -- <> (Raistlin Majere) - Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) signature.asc Description: This is a digitally signed message part ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
