This is an automated email from the ASF dual-hosted git repository. xiaoxiang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/nuttx.git
commit b025c6285db64c6bb3b9b065b96e8831a4fb8aec Author: Tiago Medicci Serrano <[email protected]> AuthorDate: Thu Sep 19 14:59:34 2024 -0300 Documentation: Document stack and backtrace dump for Espressif SoCs Stack and backtrace dump for Espressif's SoCs (ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6 and ESP32-H2) is now documented in a new section for each chip entry page. --- Documentation/platforms/risc-v/esp32c3/index.rst | 90 ++++++++++++++- Documentation/platforms/risc-v/esp32c6/index.rst | 87 +++++++++++++- Documentation/platforms/risc-v/esp32h2/index.rst | 87 +++++++++++++- Documentation/platforms/xtensa/esp32/index.rst | 127 ++++++++++++++++++++- Documentation/platforms/xtensa/esp32s2/index.rst | 124 +++++++++++++++++++- Documentation/platforms/xtensa/esp32s3/index.rst | 137 ++++++++++++++++++++++- 6 files changed, 646 insertions(+), 6 deletions(-) diff --git a/Documentation/platforms/risc-v/esp32c3/index.rst b/Documentation/platforms/risc-v/esp32c3/index.rst index e8c68758b8..3987341481 100644 --- a/Documentation/platforms/risc-v/esp32c3/index.rst +++ b/Documentation/platforms/risc-v/esp32c3/index.rst @@ -120,8 +120,13 @@ Where ``<port>`` is typically ``/dev/ttyUSB0`` or similar and ``./`` is the path to the folder containing the externally-built 2nd stage bootloader for the ESP32-C3 as explained above. +Debugging +========= + +This section describes debugging techniques for the ESP32-C3. + Debugging with ``openocd`` and ``gdb`` -====================================== +-------------------------------------- Espressif uses a specific version of OpenOCD to support ESP32-C3: `openocd-esp32 <https://github.com/espressif/>`_. @@ -189,6 +194,89 @@ whereas the content of the ``gdbinit`` file is:: Please refer to :doc:`/quickstart/debugging` for more information about debugging techniques. +Stack Dump and Backtrace Dump +----------------------------- + +NuttX has a feature to dump the stack of a task and to dump the backtrace of it (and of all +the other tasks). This feature is useful to debug the system when it is not behaving as expected, +especially when it is crashing. + +In order to enable this feature, the following options must be enabled in the NuttX configuration: +``CONFIG_SCHED_BACKTRACE``, ``CONFIG_DEBUG_SYMBOLS`` and, optionally, ``CONFIG_ALLSYMS``. + +.. note:: + The first two options enable the backtrace dump. The third option enables the backtrace dump + with the associated symbols, but increases the size of the generated NuttX binary. + +Espressif also provides a tool to translate the backtrace dump into a human-readable format. +This tool is called ``btdecode.sh`` and is available at ``tools/espressif/btdecode.sh`` of NuttX +repository. + +.. note:: + This tool is not necessary if ``CONFIG_ALLSYMS`` is enabled. In this case, the backtrace dump + contains the function names. + +Example - Crash Dump +^^^^^^^^^^^^^^^^^^^^ + +A typical crash dump, caused by an illegal load with ``CONFIG_SCHED_BACKTRACE`` and +``CONFIG_DEBUG_SYMBOLS`` enabled, is shown below:: + + riscv_exception: EXCEPTION: Store/AMO access fault. MCAUSE: 00000007, EPC: 42012df2, MT0 + riscv_exception: PANIC!!! Exception = 00000007 + _assert: Current Version: NuttX 10.4.0 2ae3246e40-dirty Sep 19 2024 14:34:41 risc-v + _assert: Assertion failed panic: at file: :0 task: backtrace process: backtrace 0x42012dac + up_dump_register: EPC: 42012df2 + up_dump_register: A0: 0000005a A1: 3fc88a54 A2: 00000001 A3: 00000088 + up_dump_register: A4: 00007fff A5: 00000001 A6: 00000000 A7: 00000000 + up_dump_register: T0: 00000000 T1: 00000000 T2: ffffffff T3: 00000000 + up_dump_register: T4: 00000000 T5: 00000000 T6: 00000000 + up_dump_register: S0: 3fc87b16 S1: 3fc87b00 S2: 00000000 S3: 00000000 + up_dump_register: S4: 00000000 S5: 00000000 S6: 00000000 S7: 00000000 + up_dump_register: S8: 00000000 S9: 00000000 S10: 00000000 S11: 00000000 + up_dump_register: SP: 3fc88ab0 FP: 3fc87b16 TP: 00000000 RA: 42012df2 + dump_stack: User Stack: + dump_stack: base: 0x3fc87b20 + dump_stack: size: 00004048 + dump_stack: sp: 0x3fc88ab0 + stack_dump: 0x3fc88a90: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00001800 + stack_dump: 0x3fc88ab0: 00000000 3fc87718 42012dac 42006dd0 00000000 00000000 3fc87b00 00000002 + stack_dump: 0x3fc88ad0: 00000000 00000000 00000000 42004d4c 00000000 00000000 00000000 00000000 + stack_dump: 0x3fc88af0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + sched_dumpstack: backtrace| 2: 0x42012df2 + dump_tasks: PID GROUP PRI POLICY TYPE NPX STATE EVENT SIGMASK STACKBASE STACKSIZE COMMAND + dump_tasks: ---- --- --- -------- ------- --- ------- ---------- ---------------- 0x3fc845e0 1536 irq + dump_task: 0 0 0 FIFO Kthread - Ready 0000000000000000 0x3fc85d18 2032 Idle_Task + dump_task: 1 1 100 RR Task - Waiting Semaphore 0000000000000000 0x3fc86c30 2000 nsh_main + dump_task: 2 2 255 RR Task - Running 0000000000000000 0x3fc87b20 4048 backtrace task + sched_dumpstack: backtrace| 0: 0x42008420 + sched_dumpstack: backtrace| 1: 0x420089a8 + sched_dumpstack: backtrace| 2: 0x42012df2 + +The lines starting with ``sched_dumpstack`` show the backtrace of the tasks. By checking it, it is +possible to track the root cause of the crash. Saving this output to a file and using the ``btdecode.sh``:: + + ./tools/btdecode.sh esp32c3 /tmp/backtrace.txt + Backtrace for task 2: + 0x42012df2: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + + Backtrace dump for all tasks: + + Backtrace for task 2: + 0x42012df2: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + + Backtrace for task 1: + 0x420089a8: sys_call2 at syscall.h:227 + (inlined by) up_switch_context at riscv_switchcontext.c:95 + + Backtrace for task 0: + 0x42008420: up_idle at esp_idle.c:74 + +The above output shows the backtrace of the tasks. By checking it, it is possible to track the +functions that were being executed when the crash occurred. + Peripheral Support ================== diff --git a/Documentation/platforms/risc-v/esp32c6/index.rst b/Documentation/platforms/risc-v/esp32c6/index.rst index 6a51d3cf8e..b418eb0f18 100644 --- a/Documentation/platforms/risc-v/esp32c6/index.rst +++ b/Documentation/platforms/risc-v/esp32c6/index.rst @@ -119,8 +119,13 @@ Where ``<port>`` is typically ``/dev/ttyUSB0`` or similar and ``./`` is the path to the folder containing the externally-built 2nd stage bootloader for the ESP32-C6 as explained above. +Debugging +========= + +This section describes debugging techniques for the ESP32-C6. + Debugging with ``openocd`` and ``gdb`` -====================================== +-------------------------------------- Espressif uses a specific version of OpenOCD to support ESP32-C6: `openocd-esp32 <https://github.com/espressif/>`_. @@ -179,6 +184,86 @@ whereas the content of the ``gdbinit`` file is:: Please refer to :doc:`/quickstart/debugging` for more information about debugging techniques. +Stack Dump and Backtrace Dump +----------------------------- + +NuttX has a feature to dump the stack of a task and to dump the backtrace of it (and of all +the other tasks). This feature is useful to debug the system when it is not behaving as expected, +especially when it is crashing. + +In order to enable this feature, the following options must be enabled in the NuttX configuration: +``CONFIG_SCHED_BACKTRACE``, ``CONFIG_DEBUG_SYMBOLS`` and, optionally, ``CONFIG_ALLSYMS``. + +.. note:: + The first two options enable the backtrace dump. The third option enables the backtrace dump + with the associated symbols, but increases the size of the generated NuttX binary. + +Espressif also provides a tool to translate the backtrace dump into a human-readable format. +This tool is called ``btdecode.sh`` and is available at ``tools/espressif/btdecode.sh`` of NuttX +repository. + +.. note:: + This tool is not necessary if ``CONFIG_ALLSYMS`` is enabled. In this case, the backtrace dump + contains the function names. + +Example - Crash Dump +^^^^^^^^^^^^^^^^^^^^ + +A typical crash dump, caused by an illegal load with ``CONFIG_SCHED_BACKTRACE`` and +``CONFIG_DEBUG_SYMBOLS`` enabled, is shown below:: + + riscv_exception: EXCEPTION: Store/AMO access fault. MCAUSE: 00000007, EPC: 420168ac, MT0 + riscv_exception: PANIC!!! Exception = 00000007 + _assert: Current Version: NuttX 10.4.0 2ae3246e40-dirty Sep 19 2024 14:47:41 risc-v + _assert: Assertion failed panic: at file: :0 task: backtrace process: backtrace 0x42016866 + up_dump_register: EPC: 420168ac + up_dump_register: A0: 0000005a A1: 40809fc4 A2: 00000001 A3: 00000088 + up_dump_register: A4: 00007fff A5: 00000001 A6: 00000000 A7: 00000000 + up_dump_register: T0: 00000000 T1: 00000000 T2: ffffffff T3: 00000000 + up_dump_register: T4: 00000000 T5: 00000000 T6: 00000000 + up_dump_register: S0: 4080908e S1: 40809078 S2: 00000000 S3: 00000000 + up_dump_register: S4: 00000000 S5: 00000000 S6: 00000000 S7: 00000000 + up_dump_register: S8: 00000000 S9: 00000000 S10: 00000000 S11: 00000000 + up_dump_register: SP: 4080a020 FP: 4080908e TP: 00000000 RA: 420168ac + dump_stack: User Stack: + dump_stack: base: 0x40809098 + dump_stack: size: 00004040 + dump_stack: sp: 0x4080a020 + stack_dump: 0x4080a000: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00001880 + stack_dump: 0x4080a020: 00000000 40808c90 42016866 42006e06 00000000 00000000 40809078 00000002 + stack_dump: 0x4080a040: 00000000 00000000 00000000 42004d72 00000000 00000000 00000000 00000000 + stack_dump: 0x4080a060: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + sched_dumpstack: backtrace| 2: 0x420168ac + dump_tasks: PID GROUP PRI POLICY TYPE NPX STATE EVENT SIGMASK STACKBASE STACKSIZE COMMAND + dump_tasks: ---- --- --- -------- ------- --- ------- ---------- ---------------- 0x40805a90 2048 irq + dump_task: 0 0 0 FIFO Kthread - Ready 0000000000000000 0x40807290 2032 Idle_Task + dump_task: 1 1 100 RR Task - Waiting Semaphore 0000000000000000 0x408081a8 1992 nsh_main + dump_task: 2 2 255 RR Task - Running 0000000000000000 0x40809098 4040 backtrace task + sched_dumpstack: backtrace| 0: 0x42008420 + sched_dumpstack: backtrace| 1: 0x420089a2 + sched_dumpstack: backtrace| 2: 0x420168ac + +The lines starting with ``sched_dumpstack`` show the backtrace of the tasks. By checking it, it is +possible to track the root cause of the crash. Saving this output to a file and using the ``btdecode.sh``:: + + ./tools/btdecode.sh esp32c6 /tmp/backtrace.txt + Backtrace for task 2: + 0x420168ac: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + + Backtrace dump for all tasks: + + Backtrace for task 2: + 0x420168ac: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + + Backtrace for task 1: + 0x420089a2: sys_call2 at syscall.h:227 + (inlined by) up_switch_context at riscv_switchcontext.c:95 + + Backtrace for task 0: + 0x42008420: up_idle at esp_idle.c:74 + Peripheral Support ================== diff --git a/Documentation/platforms/risc-v/esp32h2/index.rst b/Documentation/platforms/risc-v/esp32h2/index.rst index 38bff47fb1..137b38e679 100644 --- a/Documentation/platforms/risc-v/esp32h2/index.rst +++ b/Documentation/platforms/risc-v/esp32h2/index.rst @@ -119,8 +119,13 @@ Where ``<port>`` is typically ``/dev/ttyUSB0`` or similar and ``./`` is the path to the folder containing the externally-built 2nd stage bootloader for the ESP32-H2 as explained above. +Debugging +========= + +This section describes debugging techniques for the ESP32-H2. + Debugging with ``openocd`` and ``gdb`` -====================================== +-------------------------------------- Espressif uses a specific version of OpenOCD to support ESP32-H2: `openocd-esp32 <https://github.com/espressif/>`_. @@ -179,6 +184,86 @@ whereas the content of the ``gdbinit`` file is:: Please refer to :doc:`/quickstart/debugging` for more information about debugging techniques. +Stack Dump and Backtrace Dump +----------------------------- + +NuttX has a feature to dump the stack of a task and to dump the backtrace of it (and of all +the other tasks). This feature is useful to debug the system when it is not behaving as expected, +especially when it is crashing. + +In order to enable this feature, the following options must be enabled in the NuttX configuration: +``CONFIG_SCHED_BACKTRACE``, ``CONFIG_DEBUG_SYMBOLS`` and, optionally, ``CONFIG_ALLSYMS``. + +.. note:: + The first two options enable the backtrace dump. The third option enables the backtrace dump + with the associated symbols, but increases the size of the generated NuttX binary. + +Espressif also provides a tool to translate the backtrace dump into a human-readable format. +This tool is called ``btdecode.sh`` and is available at ``tools/espressif/btdecode.sh`` of NuttX +repository. + +.. note:: + This tool is not necessary if ``CONFIG_ALLSYMS`` is enabled. In this case, the backtrace dump + contains the function names. + +Example - Crash Dump +^^^^^^^^^^^^^^^^^^^^ + +A typical crash dump, caused by an illegal load with ``CONFIG_SCHED_BACKTRACE`` and +``CONFIG_DEBUG_SYMBOLS`` enabled, is shown below:: + + riscv_exception: EXCEPTION: Store/AMO access fault. MCAUSE: 00000007, EPC: 42012df0, MT0 + riscv_exception: PANIC!!! Exception = 00000007 + _assert: Current Version: NuttX 10.4.0 2ae3246e40-dirty Sep 19 2024 14:53:33 risc-v + _assert: Assertion failed panic: at file: :0 task: backtrace process: backtrace 0x42012daa + up_dump_register: EPC: 42012df0 + up_dump_register: A0: 0000005a A1: 408095e4 A2: 00000001 A3: 00000088 + up_dump_register: A4: 00007fff A5: 00000001 A6: 00000000 A7: 00000000 + up_dump_register: T0: 00000000 T1: 00000000 T2: ffffffff T3: 00000000 + up_dump_register: T4: 00000000 T5: 00000000 T6: 00000000 + up_dump_register: S0: 408086ae S1: 40808698 S2: 00000000 S3: 00000000 + up_dump_register: S4: 00000000 S5: 00000000 S6: 00000000 S7: 00000000 + up_dump_register: S8: 00000000 S9: 00000000 S10: 00000000 S11: 00000000 + up_dump_register: SP: 40809640 FP: 408086ae TP: 00000000 RA: 42012df0 + dump_stack: User Stack: + dump_stack: base: 0x408086b8 + dump_stack: size: 00004040 + dump_stack: sp: 0x40809640 + stack_dump: 0x40809620: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00001880 + stack_dump: 0x40809640: 00000000 408082b0 42012daa 42006e1e 00000000 00000000 40808698 00000002 + stack_dump: 0x40809660: 00000000 00000000 00000000 42004d8a 00000000 00000000 00000000 00000000 + stack_dump: 0x40809680: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + sched_dumpstack: backtrace| 2: 0x42012df0 + dump_tasks: PID GROUP PRI POLICY TYPE NPX STATE EVENT SIGMASK STACKBASE STACKSIZE COMMAND + dump_tasks: ---- --- --- -------- ------- --- ------- ---------- ---------------- 0x40805120 2048 irq + dump_task: 0 0 0 FIFO Kthread - Ready 0000000000000000 0x408068b0 2032 Idle_Task + dump_task: 1 1 100 RR Task - Waiting Semaphore 0000000000000000 0x408077c8 1992 nsh_main + dump_task: 2 2 255 RR Task - Running 0000000000000000 0x408086b8 4040 backtrace task + sched_dumpstack: backtrace| 0: 0x42008420 + sched_dumpstack: backtrace| 1: 0x420089a2 + sched_dumpstack: backtrace| 2: 0x42012df0 + +The lines starting with ``sched_dumpstack`` show the backtrace of the tasks. By checking it, it is +possible to track the root cause of the crash. Saving this output to a file and using the ``btdecode.sh``:: + + ./tools/btdecode.sh esp32h2 /tmp/backtrace.txt + Backtrace for task 2: + 0x42012df0: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + + Backtrace dump for all tasks: + + Backtrace for task 2: + 0x42012df0: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + + Backtrace for task 1: + 0x420089a2: sys_call2 at syscall.h:227 + (inlined by) up_switch_context at riscv_switchcontext.c:95 + + Backtrace for task 0: + 0x42008420: up_idle at esp_idle.c:74 + Peripheral Support ================== diff --git a/Documentation/platforms/xtensa/esp32/index.rst b/Documentation/platforms/xtensa/esp32/index.rst index 115159d646..32f698efff 100644 --- a/Documentation/platforms/xtensa/esp32/index.rst +++ b/Documentation/platforms/xtensa/esp32/index.rst @@ -142,8 +142,13 @@ externally-built 2nd stage bootloader and the partition table (if applicable): w ``make bootloader``, these files are placed into ``nuttx`` folder. ``ESPTOOL_BAUD`` is able to change the flash baud rate if desired. +Debugging +========= + +This section describes debugging techniques for the ESP32. + Debugging with ``openocd`` and ``gdb`` -====================================== +-------------------------------------- Espressif uses a specific version of OpenOCD to support ESP32: `openocd-esp32 <https://github.com/espressif/>`_. @@ -194,6 +199,126 @@ whereas the content of the ``gdbinit`` file is:: Please refer to :doc:`/quickstart/debugging` for more information about debugging techniques. +Stack Dump and Backtrace Dump +----------------------------- + +NuttX has a feature to dump the stack of a task and to dump the backtrace of it (and of all +the other tasks). This feature is useful to debug the system when it is not behaving as expected, +especially when it is crashing. + +In order to enable this feature, the following options must be enabled in the NuttX configuration: +``CONFIG_SCHED_BACKTRACE``, ``CONFIG_DEBUG_SYMBOLS`` and, optionally, ``CONFIG_ALLSYMS``. + +.. note:: + The first two options enable the backtrace dump. The third option enables the backtrace dump + with the associated symbols, but increases the size of the generated NuttX binary. + +Espressif also provides a tool to translate the backtrace dump into a human-readable format. +This tool is called ``btdecode.sh`` and is available at ``tools/espressif/btdecode.sh`` of NuttX +repository. + +.. note:: + This tool is not necessary if ``CONFIG_ALLSYMS`` is enabled. In this case, the backtrace dump + contains the function names. + +Example - Crash Dump +^^^^^^^^^^^^^^^^^^^^ + +A typical crash dump, caused by an illegal load with ``CONFIG_SCHED_BACKTRACE`` and +``CONFIG_DEBUG_SYMBOLS`` enabled, is shown below:: + + xtensa_user_panic: User Exception: EXCCAUSE=001d task: backtrace + _assert: Current Version: NuttX 10.4.0 2ae3246e40-dirty Sep 19 2024 12:59:10 xtensa + _assert: Assertion failed user panic: at file: :0 task: backtrace process: backtrace 0x400f0724 + up_dump_register: PC: 400f0754 PS: 00060530 + up_dump_register: A0: 800e2fcc A1: 3ffe1400 A2: 00000000 A3: 3ffe0470 + up_dump_register: A4: 3ffe0486 A5: 3ffaf4b0 A6: 00000000 A7: 00000000 + up_dump_register: A8: 800f0751 A9: 3ffe13d0 A10: 0000005a A11: 3ffafcb0 + up_dump_register: A12: 00000059 A13: 3ffaf600 A14: 00000002 A15: 3ffafaa4 + up_dump_register: SAR: 00000018 CAUSE: 0000001d VADDR: 00000000 + up_dump_register: LBEG: 4000c28c LEND: 4000c296 LCNT: 00000000 + dump_stack: User Stack: + dump_stack: base: 0x3ffe0490 + dump_stack: size: 00004048 + dump_stack: sp: 0x3ffe1400 + stack_dump: 0x3ffe13e0: 00000059 3ffaf600 00000002 3ffafaa4 800e1eb4 3ffe1420 400f0724 00000002 + stack_dump: 0x3ffe1400: 3ffe0486 3ffaf4b0 00000000 00000000 00000000 3ffe1440 00000000 400f0724 + stack_dump: 0x3ffe1420: 3ffe0470 3ffafae8 00000000 3ffb0d2c 00000000 3ffe1460 00000000 00000000 + stack_dump: 0x3ffe1440: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + stack_dump: 0x3ffe1460: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + sched_dumpstack: backtrace| 2: 0x400ef738 0x40085152 0x40084d05 0x40084c7d 0x40080c84 0x400f0754 0x400e2fcc 0x400e1eb4 + sched_dumpstack: backtrace| 2: 0x40000000 0x400e2fcc 0x400e1eb4 0x40000000 + dump_tasks: PID GROUP PRI POLICY TYPE NPX STATE EVENT SIGMASK STACKBASE STACKSIZE COMMAND + dump_task: 0 0 0 FIFO Kthread - Ready 0000000000000000 0x3ffb0010 3056 Idle_Task + dump_task: 1 1 100 RR Task - Waiting Semaphore 0000000000000000 0x3ffaec10 3024 nsh_main + dump_task: 2 2 255 RR Task - Running 0000000000000000 0x3ffe0490 4048 backtrace task + sched_dumpstack: backtrace| 0: 0x400e12bb 0x400826eb + sched_dumpstack: backtrace| 1: 0x400edc59 0x400edb5b 0x400edb94 0x400e6c36 0x400e643c 0x400e6714 0x400e5830 0x400e56b8 + sched_dumpstack: backtrace| 1: 0x400e5689 0x400e2fcc 0x400e1eb4 0x40000000 + sched_dumpstack: backtrace| 2: 0x400ef738 0x40084ed4 0x400ed9ea 0x40085184 0x40084d05 0x40084c7d 0x40080c84 0x400f0754 + sched_dumpstack: backtrace| 2: 0x400e2fcc 0x400e1eb4 0x40000000 0x400e2fcc 0x400e1eb4 0x40000000 + +The lines starting with ``sched_dumpstack`` show the backtrace of the tasks. By checking it, it is +possible to track the root cause of the crash. Saving this output to a file and using the ``btdecode.sh``:: + + ./tools/btdecode.sh esp32 /tmp/backtrace.txt + Backtrace for task 2: + 0x400ef738: sched_dumpstack at sched_dumpstack.c:69 + 0x40085152: _assert at assert.c:691 + 0x40084d05: xtensa_user_panic at xtensa_assert.c:188 (discriminator 1) + 0x40084c7d: xtensa_user at ??:? + 0x40080c84: _xtensa_user_handler at xtensa_user_handler.S:194 + 0x400f0754: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + 0x400e2fcc: nxtask_startup at task_startup.c:70 + 0x400e1eb4: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x400e2fcc: nxtask_startup at task_startup.c:70 + 0x400e1eb4: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + + Backtrace dump for all tasks: + + Backtrace for task 2: + 0x400ef738: sched_dumpstack at sched_dumpstack.c:69 + 0x40084ed4: dump_backtrace at assert.c:418 + 0x400ed9ea: nxsched_foreach at sched_foreach.c:69 (discriminator 2) + 0x40085184: _assert at assert.c:726 + 0x40084d05: xtensa_user_panic at xtensa_assert.c:188 (discriminator 1) + 0x40084c7d: xtensa_user at ??:? + 0x40080c84: _xtensa_user_handler at xtensa_user_handler.S:194 + 0x400f0754: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + 0x400e2fcc: nxtask_startup at task_startup.c:70 + 0x400e1eb4: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x400e2fcc: nxtask_startup at task_startup.c:70 + 0x400e1eb4: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + + Backtrace for task 1: + 0x400edc59: nxsem_wait at sem_wait.c:217 + 0x400edb5b: nxsched_waitpid at sched_waitpid.c:165 + 0x400edb94: waitpid at sched_waitpid.c:618 + 0x400e6c36: nsh_builtin at nsh_builtin.c:163 + 0x400e643c: nsh_execute at nsh_parse.c:652 + (inlined by) nsh_parse_command at nsh_parse.c:2840 + 0x400e6714: nsh_parse at nsh_parse.c:2930 + 0x400e5830: nsh_session at nsh_session.c:246 + 0x400e56b8: nsh_consolemain at nsh_consolemain.c:79 + 0x400e5689: nsh_main at nsh_main.c:80 + 0x400e2fcc: nxtask_startup at task_startup.c:70 + 0x400e1eb4: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + + Backtrace for task 0: + 0x400e12bb: nx_start at nx_start.c:772 (discriminator 1) + 0x400826eb: __esp32_start at esp32_start.c:294 + (inlined by) __start at esp32_start.c:358 + +The above output shows the backtrace of the tasks. By checking it, it is possible to track the +functions that were being executed when the crash occurred. + Peripheral Support ================== diff --git a/Documentation/platforms/xtensa/esp32s2/index.rst b/Documentation/platforms/xtensa/esp32s2/index.rst index e88bbc0e09..57a0c5af1d 100644 --- a/Documentation/platforms/xtensa/esp32s2/index.rst +++ b/Documentation/platforms/xtensa/esp32s2/index.rst @@ -135,8 +135,13 @@ externally-built 2nd stage bootloader and the partition table (if applicable): w ``make bootloader``, these files are placed into ``nuttx`` folder. ``ESPTOOL_BAUD`` is able to change the flash baud rate if desired. +Debugging +========= + +This section describes debugging techniques for the ESP32-S2. + Debugging with ``openocd`` and ``gdb`` -====================================== +-------------------------------------- Espressif uses a specific version of OpenOCD to support ESP32-S2: `openocd-esp32 <https://github.com/espressif/>`_. @@ -186,6 +191,123 @@ whereas the content of the ``gdbinit`` file is:: Please refer to :doc:`/quickstart/debugging` for more information about debugging techniques. +Stack Dump and Backtrace Dump +----------------------------- + +NuttX has a feature to dump the stack of a task and to dump the backtrace of it (and of all +the other tasks). This feature is useful to debug the system when it is not behaving as expected, +especially when it is crashing. + +In order to enable this feature, the following options must be enabled in the NuttX configuration: +``CONFIG_SCHED_BACKTRACE``, ``CONFIG_DEBUG_SYMBOLS`` and, optionally, ``CONFIG_ALLSYMS``. + +.. note:: + The first two options enable the backtrace dump. The third option enables the backtrace dump + with the associated symbols, but increases the size of the generated NuttX binary. + +Espressif also provides a tool to translate the backtrace dump into a human-readable format. +This tool is called ``btdecode.sh`` and is available at ``tools/espressif/btdecode.sh`` of NuttX +repository. + +.. note:: + This tool is not necessary if ``CONFIG_ALLSYMS`` is enabled. In this case, the backtrace dump + contains the function names. + +Example - Crash Dump +^^^^^^^^^^^^^^^^^^^^ + +A typical crash dump, caused by an illegal load with ``CONFIG_SCHED_BACKTRACE`` and +``CONFIG_DEBUG_SYMBOLS`` enabled, is shown below:: + + xtensa_user_panic: User Exception: EXCCAUSE=001d task: backtrace + _assert: Current Version: NuttX 10.4.0 2ae3246e40-dirty Sep 19 2024 14:15:50 xtensa + _assert: Assertion failed user panic: at file: :0 task: backtrace process: backtrace 0x400a0aa8 + up_dump_register: PC: 400a0ad8 PS: 00060730 + up_dump_register: A0: 8009312c A1: 3ffbaf90 A2: 00000000 A3: 3ffba008 + up_dump_register: A4: 3ffba01e A5: 3ffb95c0 A6: 00000000 A7: 00000000 + up_dump_register: A8: 800a0ad5 A9: 3ffbaf60 A10: 0000005a A11: 3ffb9dc0 + up_dump_register: A12: 00000059 A13: 3ffb9710 A14: 00000002 A15: 3ffb9bb4 + up_dump_register: SAR: 00000018 CAUSE: 0000001d VADDR: 00000000 + dump_stack: User Stack: + dump_stack: base: 0x3ffba028 + dump_stack: size: 00004040 + dump_stack: sp: 0x3ffbaf90 + stack_dump: 0x3ffbaf70: 00000059 3ffb9710 00000002 3ffb9bb4 80091fbc 3ffbafb0 400a0aa8 00000002 + stack_dump: 0x3ffbaf90: 3ffba01e 3ffb95c0 00000000 00000000 00000000 3ffbafd0 00000000 400a0aa8 + stack_dump: 0x3ffbafb0: 3ffba008 3ffb9bf8 00000000 3ffb637c 00000000 3ffbaff0 00000000 00000000 + stack_dump: 0x3ffbafd0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + stack_dump: 0x3ffbaff0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + sched_dumpstack: backtrace| 2: 0x4009f930 0x4009d94a 0x40094f7d 0x400945cd 0x400228e1 0x400a0ad8 0x4009312c 0x40091fbc + sched_dumpstack: backtrace| 2: 0x40000000 0x4009312c 0x40091fbc 0x40000000 + dump_tasks: PID GROUP PRI POLICY TYPE NPX STATE EVENT SIGMASK STACKBASE STACKSIZE COMMAND + dump_task: 0 0 0 FIFO Kthread - Ready 0000000000000000 0x3ffb7720 3056 Idle_Task + dump_task: 1 1 100 RR Task - Waiting Semaphore 0000000000000000 0x3ffb8d20 3024 nsh_main + dump_task: 2 2 255 RR Task - Running 0000000000000000 0x3ffba028 4040 backtrace task + sched_dumpstack: backtrace| 0: 0x40091327 + sched_dumpstack: backtrace| 1: 0x4009dde1 0x4009dce3 0x4009dd1c 0x400969fa 0x40096200 0x400964d8 0x400955f4 0x4009547c + sched_dumpstack: backtrace| 1: 0x4009544d 0x4009312c 0x40091fbc 0x40000000 + sched_dumpstack: backtrace| 2: 0x4009f930 0x4009d6cc 0x4009db72 0x4009d97c 0x40094f7d 0x400945cd 0x400228e1 0x400a0ad8 + sched_dumpstack: backtrace| 2: 0x4009312c 0x40091fbc 0x40000000 0x4009312c 0x40091fbc 0x40000000 + +The lines starting with ``sched_dumpstack`` show the backtrace of the tasks. By checking it, it is +possible to track the root cause of the crash. Saving this output to a file and using the ``btdecode.sh``:: + + ./tools/btdecode.sh esp32s2 /tmp/backtrace.txt + Backtrace for task 2: + 0x4009f930: sched_dumpstack at sched_dumpstack.c:69 + 0x4009d94a: _assert at assert.c:691 + 0x40094f7d: xtensa_user_panic at xtensa_assert.c:188 (discriminator 1) + 0x400945cd: xtensa_user at ??:? + 0x400228e1: _xtensa_user_handler at xtensa_user_handler.S:194 + 0x400a0ad8: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + 0x4009312c: nxtask_startup at task_startup.c:70 + 0x40091fbc: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x4009312c: nxtask_startup at task_startup.c:70 + 0x40091fbc: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + + Backtrace dump for all tasks: + + Backtrace for task 2: + 0x4009f930: sched_dumpstack at sched_dumpstack.c:69 + 0x4009d6cc: dump_backtrace at assert.c:418 + 0x4009db72: nxsched_foreach at sched_foreach.c:69 (discriminator 2) + 0x4009d97c: _assert at assert.c:726 + 0x40094f7d: xtensa_user_panic at xtensa_assert.c:188 (discriminator 1) + 0x400945cd: xtensa_user at ??:? + 0x400228e1: _xtensa_user_handler at xtensa_user_handler.S:194 + 0x400a0ad8: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + 0x4009312c: nxtask_startup at task_startup.c:70 + 0x40091fbc: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x4009312c: nxtask_startup at task_startup.c:70 + 0x40091fbc: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + + Backtrace for task 1: + 0x4009dde1: nxsem_wait at sem_wait.c:217 + 0x4009dce3: nxsched_waitpid at sched_waitpid.c:165 + 0x4009dd1c: waitpid at sched_waitpid.c:618 + 0x400969fa: nsh_builtin at nsh_builtin.c:163 + 0x40096200: nsh_execute at nsh_parse.c:652 + (inlined by) nsh_parse_command at nsh_parse.c:2840 + 0x400964d8: nsh_parse at nsh_parse.c:2930 + 0x400955f4: nsh_session at nsh_session.c:246 + 0x4009547c: nsh_consolemain at nsh_consolemain.c:79 + 0x4009544d: nsh_main at nsh_main.c:80 + 0x4009312c: nxtask_startup at task_startup.c:70 + 0x40091fbc: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + + Backtrace for task 0: + 0x40091327: nx_start at nx_start.c:772 (discriminator 1) + +The above output shows the backtrace of the tasks. By checking it, it is possible to track the +functions that were being executed when the crash occurred. + Peripheral Support ================== diff --git a/Documentation/platforms/xtensa/esp32s3/index.rst b/Documentation/platforms/xtensa/esp32s3/index.rst index 7c5a11a9e3..fef95268ff 100644 --- a/Documentation/platforms/xtensa/esp32s3/index.rst +++ b/Documentation/platforms/xtensa/esp32s3/index.rst @@ -142,8 +142,13 @@ externally-built 2nd stage bootloader and the partition table (if applicable): w ``make bootloader``, these files are placed into ``nuttx`` folder. ``ESPTOOL_BAUD`` is able to change the flash baud rate if desired. +Debugging +========= + +This section describes debugging techniques for the ESP32-S3. + Debugging with ``openocd`` and ``gdb`` -====================================== +-------------------------------------- Espressif uses a specific version of OpenOCD to support ESP32-S3: `openocd-esp32 <https://github.com/espressif/>`_. @@ -182,6 +187,136 @@ whereas the content of the ``gdbinit`` file is:: Please refer to :doc:`/quickstart/debugging` for more information about debugging techniques. +Stack Dump and Backtrace Dump +----------------------------- + +NuttX has a feature to dump the stack of a task and to dump the backtrace of it (and of all +the other tasks). This feature is useful to debug the system when it is not behaving as expected, +especially when it is crashing. + +In order to enable this feature, the following options must be enabled in the NuttX configuration: +``CONFIG_SCHED_BACKTRACE``, ``CONFIG_DEBUG_SYMBOLS`` and, optionally, ``CONFIG_ALLSYMS``. + +.. note:: + The first two options enable the backtrace dump. The third option enables the backtrace dump + with the associated symbols, but increases the size of the generated NuttX binary. + +Espressif also provides a tool to translate the backtrace dump into a human-readable format. +This tool is called ``btdecode.sh`` and is available at ``tools/espressif/btdecode.sh`` of NuttX +repository. + +.. note:: + This tool is not necessary if ``CONFIG_ALLSYMS`` is enabled. In this case, the backtrace dump + contains the function names. + +Example - Crash Dump +^^^^^^^^^^^^^^^^^^^^ + +A typical crash dump, caused by an illegal load with ``CONFIG_SCHED_BACKTRACE`` and +``CONFIG_DEBUG_SYMBOLS`` enabled, is shown below:: + + xtensa_user_panic: User Exception: EXCCAUSE=001d task: backtrace + _assert: Current Version: NuttX 10.4.0 2ae3246e40-dirty Sep 19 2024 14:19:27 xtensa + _assert: Assertion failed user panic: at file: :0 task: backtrace process: backtrace 0x42020c90 + up_dump_register: PC: 42020cc0 PS: 00060930 + up_dump_register: A0: 82012d10 A1: 3fc8e2e0 A2: 00000000 A3: 3fc8d350 + up_dump_register: A4: 3fc8d366 A5: 3fc8c900 A6: 00000000 A7: 00000000 + up_dump_register: A8: 82020cbd A9: 3fc8e2b0 A10: 0000005a A11: 3fc8d108 + up_dump_register: A12: 00000059 A13: 3fc8ca50 A14: 00000002 A15: 3fc8cefc + up_dump_register: SAR: 00000018 CAUSE: 0000001d VADDR: 00000000 + up_dump_register: LBEG: 40056f08 LEND: 40056f12 LCNT: 00000000 + dump_stack: User Stack: + dump_stack: base: 0x3fc8d370 + dump_stack: size: 00004048 + dump_stack: sp: 0x3fc8e2e0 + stack_dump: 0x3fc8e2c0: 00000059 3fc8ca50 00000002 3fc8cefc 82011ba0 3fc8e300 42020c90 00000002 + stack_dump: 0x3fc8e2e0: 3fc8d366 3fc8c900 00000000 00000000 00000000 3fc8e320 00000000 42020c90 + stack_dump: 0x3fc8e300: 3fc8d350 3fc8cf40 00000000 3fc8912c 00000000 3fc8e340 00000000 00000000 + stack_dump: 0x3fc8e320: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + stack_dump: 0x3fc8e340: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 + sched_dumpstack: backtrace| 2: 0x4201fc6c 0x403773a0 0x40376f69 0x40376ee1 0x40374ca9 0x42020cc0 0x42012d10 0x42011ba0 + sched_dumpstack: backtrace| 2: 0x40000000 0x40000000 0x42012d10 0x42011ba0 0x40000000 0x40000000 + dump_tasks: PID GROUP PRI POLICY TYPE NPX STATE EVENT SIGMASK STACKBASE STACKSIZE COMMAND + dump_tasks: ---- --- --- -------- ------- --- ------- ---------- ---------------- 0x3fc8b220 2048 irq + dump_task: 0 0 0 FIFO Kthread - Ready 0000000000000000 0x3fc8a630 3056 Idle_Task + dump_task: 1 1 100 RR Task - Waiting Semaphore 0000000000000000 0x3fc8c468 1992 nsh_main + dump_task: 2 2 255 RR Task - Running 0000000000000000 0x3fc8d370 4048 backtrace task + sched_dumpstack: backtrace| 0: 0x42010f37 0x40374dda 0x40374e9a 0x40045c04 0x40043ab9 0x40034c48 0x40000000 + sched_dumpstack: backtrace| 1: 0x4201e131 0x4201e033 0x4201e06c 0x42017056 0x4201685c 0x42016b34 0x42015c50 0x42015ad8 + sched_dumpstack: backtrace| 1: 0x42015aa9 0x42012d10 0x42011ba0 0x40000000 0x40000000 + sched_dumpstack: backtrace| 2: 0x4201fc6c 0x40377098 0x4201df02 0x403773ed 0x40376f69 0x40376ee1 0x40374ca9 0x42020cc0 + sched_dumpstack: backtrace| 2: 0x42012d10 0x42011ba0 0x40000000 0x40000000 0x42012d10 0x42011ba0 0x40000000 0x40000000 + +The lines starting with ``sched_dumpstack`` show the backtrace of the tasks. By checking it, it is +possible to track the root cause of the crash. Saving this output to a file and using the ``btdecode.sh``:: + + ./tools/btdecode.sh esp32s3 /tmp/backtrace.txt + Backtrace for task 2: + 0x4201fc6c: sched_dumpstack at sched_dumpstack.c:69 + 0x403773a0: _assert at assert.c:691 + 0x40376f69: xtensa_user_panic at xtensa_assert.c:188 (discriminator 1) + 0x40376ee1: xtensa_user at ??:? + 0x40374ca9: _xtensa_user_handler at xtensa_user_handler.S:194 + 0x42020cc0: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + 0x42012d10: nxtask_startup at task_startup.c:70 + 0x42011ba0: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x40000000: ?? ??:0 + 0x42012d10: nxtask_startup at task_startup.c:70 + 0x42011ba0: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x40000000: ?? ??:0 + + Backtrace dump for all tasks: + + Backtrace for task 2: + 0x4201fc6c: sched_dumpstack at sched_dumpstack.c:69 + 0x40377098: dump_backtrace at assert.c:418 + 0x4201df02: nxsched_foreach at sched_foreach.c:69 (discriminator 2) + 0x403773ed: _assert at assert.c:726 + 0x40376f69: xtensa_user_panic at xtensa_assert.c:188 (discriminator 1) + 0x40376ee1: xtensa_user at ??:? + 0x40374ca9: _xtensa_user_handler at xtensa_user_handler.S:194 + 0x42020cc0: assert_on_task at backtrace_main.c:158 + (inlined by) backtrace_main at backtrace_main.c:194 + 0x42012d10: nxtask_startup at task_startup.c:70 + 0x42011ba0: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x40000000: ?? ??:0 + 0x42012d10: nxtask_startup at task_startup.c:70 + 0x42011ba0: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x40000000: ?? ??:0 + + Backtrace for task 1: + 0x4201e131: nxsem_wait at sem_wait.c:217 + 0x4201e033: nxsched_waitpid at sched_waitpid.c:165 + 0x4201e06c: waitpid at sched_waitpid.c:618 + 0x42017056: nsh_builtin at nsh_builtin.c:163 + 0x4201685c: nsh_execute at nsh_parse.c:652 + (inlined by) nsh_parse_command at nsh_parse.c:2840 + 0x42016b34: nsh_parse at nsh_parse.c:2930 + 0x42015c50: nsh_session at nsh_session.c:246 + 0x42015ad8: nsh_consolemain at nsh_consolemain.c:79 + 0x42015aa9: nsh_main at nsh_main.c:80 + 0x42012d10: nxtask_startup at task_startup.c:70 + 0x42011ba0: nxtask_start at task_start.c:75 + 0x40000000: ?? ??:0 + 0x40000000: ?? ??:0 + + Backtrace for task 0: + 0x42010f37: nx_start at nx_start.c:772 (discriminator 1) + 0x40374dda: __esp32s3_start at esp32s3_start.c:439 (discriminator 1) + 0x40374e9a: __start at ??:? + 0x40045c04: ?? ??:0 + 0x40043ab9: ?? ??:0 + 0x40034c48: ?? ??:0 + 0x40000000: ?? ??:0 + +The above output shows the backtrace of the tasks. By checking it, it is possible to track the +functions that were being executed when the crash occurred. + Peripheral Support ==================
