Re: debugging user app crashes

2024-01-13 Thread Nathan Hartman
On Thu, Jan 11, 2024 at 10:35 AM Gregory Nutt  wrote:

> Everything you need to know is in that referenced wiki page:
>
> https://cwiki.apache.org/confluence/display/NUTTX/Debugging+ELF+Loadable+Modules
>
> I don't know anything about QEMU or RISC-V.  But assuming that NSH is
> your init application...
>
>   * NSH is not in memory when you start the OS; it resides in your init
> file.  The OS will automatically start NSH as the init application
> (if properly configured).
>   * Since there is no gdbserver or ptrace, there is not elegant way to
> break into the program.  Some kludgey ways to do that are described
> in the referenced wiki page.
>   * Then just follow the instructions in the wiki page:
>   o Start the debugger
>   o Load the symbol table of the NSH application.  You probably
> don't need as load offset as described in the wiki page.
>   o Set a breakpoint at nsh_parse, and
>   o Continue
>
> I have nothing to add to that.



Note to self: Check whether above-linked wiki page has been migrated to
Documentation in the repository and, if not, migrate it...

I likely won't be able to work on that until next week, so if someone else
wants to do it, feel free. :-)

Cheers
Nathan


Re: debugging user app crashes

2024-01-11 Thread Gregory Nutt
Everything you need to know is in that referenced wiki page: 
https://cwiki.apache.org/confluence/display/NUTTX/Debugging+ELF+Loadable+Modules


I don't know anything about QEMU or RISC-V.  But assuming that NSH is 
your init application...


 * NSH is not in memory when you start the OS; it resides in your init
   file.  The OS will automatically start NSH as the init application
   (if properly configured).
 * Since there is no gdbserver or ptrace, there is not elegant way to
   break into the program.  Some kludgey ways to do that are described
   in the referenced wiki page.
 * Then just follow the instructions in the wiki page:
 o Start the debugger
 o Load the symbol table of the NSH application.  You probably
   don't need as load offset as described in the wiki page.
 o Set a breakpoint at nsh_parse, and
 o Continue

I have nothing to add to that.

On 1/10/2024 7:36 PM, yfliu2008 wrote:

Spudaneco,



Can you share more details about how to break in "nsh_parse()" with 
"rv-virt/knsh64"?


Regards,
yf





Original

  


From:"spudaneco"



Re:Re: debugging user app crashes

2024-01-10 Thread spudaneco
You should be able to set breakpoints properly, but only after the correct MMU 
mapping is in place.
 Original message From: yfliu2008  
Date: 1/9/24  8:06 PM  (GMT-06:00) To: dev  Subject: 
Re:Re: debugging user app crashes Yes, in kernel mode each user space program 
has similar but isolated virtual address spaces. So it seems that we can't set 
break points to them like FLAT or PROTECTED build.Not sure if we need some sort 
of debug agent in KERNEL build so that it can help debugging user space 
tasks. When use of kernel build araises, this need will increase I 
guess.Regards,yf   Original     
   From:"Alan C. Assis"< acas...@gmail.com ;Date:2024/1/10 
9:47To:"dev"< dev@nuttx.apache.org ;Subject:Re: debugging user app 
crashesYes! For kernel mode each program has its own main() implementation, 
notthe definition.BR,AlanOn Tue, Jan 9, 2024 at 10:34 PM Gregory Nutt  
wrote: On 1/9/2024 7:28 PM, Alan C. Assis wrote:  Hi Yf, 
 Just look at your System.map and you will see that all those main() 
became  _main.   It isn't possible to have a unique 
main() symbol, otherwise we will have a  duplicated symbol, also 
even if the compiler accepted that, how could NuttX  know you want 
to call the main from Hello instead of the main from Alarm?   
BR,   Alan This is not true in the kernel build mode.  All 
applications run at exactly the same virtual address and the entry point is 
main() for all of them.  In the kernel build, all applications are loadable 
modules. None of the symbols in the loadable modules will appear in 
System.map since each application module is separately compiled and linked.

Re: debugging user app crashes

2024-01-09 Thread Gregory Nutt



On 1/9/2024 8:06 PM, yfliu2008 wrote:

Yes, in kernel mode each user space program has similar but isolated virtual 
address spaces. So it seems that we can't set break points to them like FLAT or 
PROTECTED build.


Not sure if we need some sort of debug agent in KERNEL build so that it can help 
debugging user space tasks. When use of kernel build araises, this need 
will increase I guess.


Regards,
yf
People have periodically suggested implementing ptrace for NuttX. For 
example, https://github.com/apache/nuttx/issues/2028


Re: debugging user app crashes

2024-01-09 Thread Alan C. Assis
Yes! For kernel mode each program has its own main() implementation, not
the definition.
BR,
Alan

On Tue, Jan 9, 2024 at 10:34 PM Gregory Nutt  wrote:

>
> On 1/9/2024 7:28 PM, Alan C. Assis wrote:
> > Hi Yf,
> > Just look at your System.map and you will see that all those main()
> became
> > _main.
> >
> > It isn't possible to have a unique main() symbol, otherwise we will have
> a
> > duplicated symbol, also even if the compiler accepted that, how could
> NuttX
> > know you want to call the main from Hello instead of the main from Alarm?
> >
> > BR,
> >
> > Alan
> This is not true in the kernel build mode.  All applications run at
> exactly the same virtual address and the entry point is main() for all
> of them.  In the kernel build, all applications are loadable modules.
> None of the symbols in the loadable modules will appear in System.map
> since each application module is separately compiled and linked.


Re: debugging user app crashes

2024-01-09 Thread Gregory Nutt


On 1/9/2024 7:28 PM, Alan C. Assis wrote:

Hi Yf,
Just look at your System.map and you will see that all those main() became
_main.

It isn't possible to have a unique main() symbol, otherwise we will have a
duplicated symbol, also even if the compiler accepted that, how could NuttX
know you want to call the main from Hello instead of the main from Alarm?

BR,

Alan
This is not true in the kernel build mode.  All applications run at 
exactly the same virtual address and the entry point is main() for all 
of them.  In the kernel build, all applications are loadable modules.  
None of the symbols in the loadable modules will appear in System.map 
since each application module is separately compiled and linked.

Re: debugging user app crashes

2024-01-09 Thread Gregory Nutt

This is old and does not specifically address kernel mode, but I think most of 
the debug tips do apply in kernel mode too. The load address is a constant for 
kernel ELF modules so at least that part doesn't apply.  But the rest probably 
does.

On 1/9/2024 7:20 PM, yfliu2008 wrote:

Alan,


Thanks a lot for your warm replies.


These apps are built with kernel mode export libraries, their entrances are all 
"main" as they each run in their user address spaces.


So when gdb connected with QEMU, command "break qemu_rv_start" can work but "break 
main" simply got function not defined even though I did "load ../apps/bin/init". It seems to me that 
current gdb session is bound to the kernel ELF, not to the apps ELF.




Regards,
yf





Original

  


From:"Alan C. Assis"< acas...@gmail.com ;

Date:2024/1/10 8:11

To:"dev"< dev@nuttx.apache.org ;

Subject:Re: debugging user app crashes


Hi Yf Liu,
I don't know if this minidump server works on 64-bit and with QEMU, but
about the other question how to setup breakpoint in the NSH user space
program, just set the break point to the nsh_main if you want to debug the
NSH itself or set it to program_main to put the breakpoint to some user
program called from NSH. For example: setting it to hello_main will cause
the GDB to stop at hello_main when you type "nsh hello" and press enter.

BR,

Alan

On Mon, Jan 8, 2024 at 9:13 PM yfliu2008  wrote:

 Dear community,




 I have NSH command crashes within qemu-system-riscv64 using a testing
 rv-virt/knsh configuration, I wonder what tools can I use to find the crash
 stack?




 I have feed console crash.log to minidumpserver.py as per this guide:
 https://nuttx.apache.org/docs/12.3.0/guides/minidumpserver.htm,; but
 it seems the tool might not support 64bit yet?




 I've tried gdb connection with qemu-system-riscv64 as per this
 guide: https://nuttx.apache.org/docs/12.3.0/guides/gdbwithpython.html,
 but I don't know how to set a break point in the user space NSH program.




 Can anyone give me a pointer?




 Regards,

 yf


Re: Re: debugging user app crashes

2024-01-09 Thread Alan C. Assis
Hi Yf,
Just look at your System.map and you will see that all those main() became
_main.

It isn't possible to have a unique main() symbol, otherwise we will have a
duplicated symbol, also even if the compiler accepted that, how could NuttX
know you want to call the main from Hello instead of the main from Alarm?

BR,

Alan

On Tue, Jan 9, 2024 at 10:20 PM yfliu2008  wrote:

> Alan,
>
>
> Thanks a lot for your warm replies.
>
>
> These apps are built with kernel mode export libraries, their entrances
> are all "main" as they each run in their user address spaces.
>
>
> So when gdb connected with QEMU, command "break qemu_rv_start"
> can work but "break main" simply got function not defined even though I did
> "load ../apps/bin/init". It seems to me that current gdb session is bound
> to the kernel ELF, not to the apps ELF.
>
>
>
>
> Regards,
> yf
>
>
>
>
>
> Original
>
>
>
> From:"Alan C. Assis"< acas...@gmail.com ;
>
> Date:2024/1/10 8:11
>
> To:"dev"< dev@nuttx.apache.org ;
>
> Subject:Re: debugging user app crashes
>
>
> Hi Yf Liu,
> I don't know if this minidump server works on 64-bit and with QEMU, but
> about the other question how to setup breakpoint in the NSH user space
> program, just set the break point to the nsh_main if you want to debug the
> NSH itself or set it to program_main to put the breakpoint to some user
> program called from NSH. For example: setting it to hello_main will cause
> the GDB to stop at hello_main when you type "nsh hello" and press
> enter.
>
> BR,
>
> Alan
>
> On Mon, Jan 8, 2024 at 9:13 PM yfliu2008  wrote:
>
>  Dear community,
> 
> 
> 
> 
>  I have NSH command crashes within qemu-system-riscv64 using a testing
>  rv-virt/knsh configuration, I wonder what tools can I use to find the
> crash
>  stack?
> 
> 
> 
> 
>  I have feed console crash.log to minidumpserver.py as per this
> guide:
>  https://nuttx.apache.org/docs/12.3.0/guides/minidumpserver.htm,;
> but
>  it seems the tool might not support 64bit yet?
> 
> 
> 
> 
>  I've tried gdb connection with qemu-system-riscv64 as per this
>  guide: https://nuttx.apache.org/docs/12.3.0/guides/gdbwithpython.html
> ,
>  but I don't know how to set a break point in the user space NSH
> program.
> 
> 
> 
> 
>  Can anyone give me a pointer?
> 
> 
> 
> 
>  Regards,
> 
>  yf


Re: debugging user app crashes

2024-01-09 Thread Alan C. Assis
Hi Yf Liu,
I don't know if this minidump server works on 64-bit and with QEMU, but
about the other question how to setup breakpoint in the NSH user space
program, just set the break point to the nsh_main if you want to debug the
NSH itself or set it to program_main to put the breakpoint to some user
program called from NSH. For example: setting it to hello_main will cause
the GDB to stop at hello_main when you type "nsh> hello" and press enter.

BR,

Alan

On Mon, Jan 8, 2024 at 9:13 PM yfliu2008  wrote:

> Dear community,
>
>
>
>
> I have NSH command crashes within qemu-system-riscv64 using a testing
> rv-virt/knsh configuration, I wonder what tools can I use to find the crash
> stack?
>
>
>
>
> I have feed console crash.log to minidumpserver.py as per this guide:
> https://nuttx.apache.org/docs/12.3.0/guides/minidumpserver.htm,; but
> it seems the tool might not support 64bit yet?
>
>
>
>
> I've tried gdb connection with qemu-system-riscv64 as per this
> guide: https://nuttx.apache.org/docs/12.3.0/guides/gdbwithpython.html,
> but I don't know how to set a break point in the user space NSH program.
>
>
>
>
> Can anyone give me a pointer?
>
>
>
>
> Regards,
>
> yf