nttedt-miyawaki opened a new issue, #20881: URL: https://github.com/apache/nuttx/issues/20881
### Description / Steps to reproduce the issue ## Description `imxrt_dma_send()` in `arch/arm/src/imxrt/imxrt_serial.c` sets up the second TX DMA segment whenever `dev->dmatx.nbuffer` is non-NULL. When the TX ring buffer wraps with `head == 0`, `uart_xmitchars_dma()` sets `nbuffer` but `nlength` is 0, so a TCD with `CITER = 0` is linked after the first one. On an i.MX RT1176 board we captured the resulting eDMA errors: all were `ES[NCE]` on LPUART TX channels with `CR[HALT]` set at capture time. In the same intervals, SPI, LPI2C and LPUART peripherals served by the same eDMA reported errors. ## Code (apache/nuttx master, a46119d437) `drivers/serial/serial_dma.c`, `uart_xmitchars_dma()`: https://github.com/apache/nuttx/blob/a46119d43726b69bb91805af45ec94c5024c95d2/drivers/serial/serial_dma.c#L117-L123 ```c else { xfer->buffer = &dev->xmit.buffer[dev->xmit.tail]; xfer->length = dev->xmit.size - dev->xmit.tail; xfer->nbuffer = dev->xmit.buffer; xfer->nlength = dev->xmit.head; } ``` When `head == 0` and `tail > 0`, this branch sets `nbuffer` to the start of the buffer and `nlength` to 0. `arch/arm/src/imxrt/imxrt_serial.c`, `imxrt_dma_send()`: https://github.com/apache/nuttx/blob/a46119d43726b69bb91805af45ec94c5024c95d2/arch/arm/src/imxrt/imxrt_serial.c#L3603-L3618 ```c #if CONFIG_IMXRT_EDMA_NTCD > 1 /* Is this a split transfer? */ if (dev->dmatx.nbuffer) { config.iter = priv->dev.dmatx.nlength; config.saddr = (uint32_t) priv->dev.dmatx.nbuffer; ... imxrt_dmach_xfrsetup(priv->txdma, &config); } #endif ``` The second TCD is set up with `iter = nlength`, i.e. 0 in the case above. This check was added with the LPUART TX DMA support (fa58381e58). Commit 6b0f85b768 ("arch/arm/imxrt: LPUART DMA fixes") added the `#if CONFIG_IMXRT_EDMA_NTCD > 1` guard and kept the check. The i.MX RT1170 Reference Manual (Rev. 5) defines `ES[NCE]` as including "TCDn_CITER[CITER] = 0", and describes `CR[HOE]` as: "any error causes the eDMA engine to write 1 to the HALT field. Subsequently, the eDMA engine ignores all service requests until you write 0 to the HALT field." `CONFIG_IMXRT_EDMA_HOE` defaults to `y`: https://github.com/apache/nuttx/blob/a46119d43726b69bb91805af45ec94c5024c95d2/arch/arm/src/imxrt/Kconfig#L3111-L3113 `imxrt_dmaterminate()` writes 0 to the TCD `CSR` and `DLASTSGA` and does not change `CITER`: https://github.com/apache/nuttx/blob/a46119d43726b69bb91805af45ec94c5024c95d2/arch/arm/src/imxrt/imxrt_edma.c#L441-L476 RM 6.4.2: "If the error source is not removed before the next activation of the problem channel, the error is detected and recorded again." ## Where we observed it - Board: PX4 fmu-v6xrt (i.MX RT1176), a PX4-based build. In its NuttX, `imxrt_dma_send()`, `uart_xmitchars_dma()` and `imxrt_dmaterminate()` are identical to those in PX4/NuttX `b8a6c3268a`. The split-transfer check and `uart_xmitchars_dma()` are the same as in apache/nuttx master above; master additionally has the `NTCD > 1` guard. - `CONFIG_IMXRT_EDMA=y`, `CONFIG_IMXRT_EDMA_NTCD=64`, `CONFIG_IMXRT_EDMA_HOE=y`, `CONFIG_IMXRT_EDMA_ELINK=y`; LPSPI1/2/3, LPI2C1/2/3 and LPUART3/4/5/6/8/10 use eDMA. - Bench, nothing connected to the UARTs that showed the error. Two LPUARTs (LPUART4 and LPUART8) were transmitting continuously at 921600 baud (about 1.9 KB/s and 21 KB/s). ## Capture method We added code to `imxrt_error_interrupt()` that copies `CR`, `ES`, `ERR`, `ERQ`, `INT`, `HRS`, the failing channel's TCD and its DMAMUX setting, with a millisecond timestamp, into an 8-entry RAM ring buffer before the error is cleared, and an nsh command that prints it. It was read once a minute. We can share the patch. ## Results (3 h 22 min) - 930 eDMA errors. 888 were read; 42 were overwritten in the ring buffer before being read. - All 888: `ES[NCE]` set; channel = an LPUART TX channel (LPUART4, LPUART8, LPUART3 or LPUART5); TCD `CITER = BITER = 0`; TCD `SADDR` = first byte of that port's TX ring buffer (`g_lpuart4txbuffer`, `g_lpuart8txbuffer`, ...); `CR = 0x432` (HALT set). - `CR` read on demand (outside the interrupt) showed `0x412` (HALT clear). - The errors came in pairs on the same channel: the first with `CITER = 0`, `CSR = 0x000a`; the second with `CSR = 0x0000`, or with a non-zero `CITER`. Of 444 pairs, 425 had both entries in the same millisecond and 19 had the second entry 1-269 ms after the first. - In 14 of the 19 delayed pairs (gaps of 12-269 ms), the board's IMUs (LPSPI), barometers and magnetometer (LPI2C) and/or an I/O coprocessor link (LPUART6) reported errors. In each of these 14, the interruption of the IMU sample stream started 1-6 ms before the first eDMA error of the pair and lasted 1-6 ms longer than the gap between the two errors (for example, 268 ms gap / 270.2 ms interruption, 45 ms gap / 48.0 ms interruption). - For the other 5 delayed pairs (gaps of 1-9 ms), the IMU sample stream had interruptions of 10.5-12.0 ms and no peripheral errors were reported. - In the pairs with both entries in the same millisecond, the IMU sample stream had no interruption over 15 ms and no peripheral errors were reported. - One more interruption of 224.8 ms with peripheral errors occurred at a time for which no delayed pair was read; ring-buffer entries were lost in the same minute. ## Change tested ```diff - if (dev->dmatx.nbuffer) + if (dev->dmatx.nlength) ``` With this change, on the same board and settings: 0 eDMA errors in the first 397 s after boot, `CR = 0x412`, `ES = 0`, `ERR = 0`, and no peripheral errors. Before the change the rate was 930 / 12143 s = 0.077 errors/s, i.e. about 30 in 397 s. We have not yet run the changed firmware for longer than 397 s. In the wrap-around branch the first segment length is `size - tail`, which is at least 1, and `imxrt_dma_txcallback()` adds `length + nlength`, which is unchanged when the second segment is skipped. ## Unexplained observation The firmware writes the eDMA `CR` only in `arm_dma_initialize()` (checked by disassembly of the image). The RT1060 and RT1170 reference manuals say HALT is cleared by software writing 0. We nevertheless read `CR` with HALT clear after each error, and further DMA transfers and further eDMA errors occurred. We found no related item in the RT1170 errata (IMXRT1170ACE). ### On which OS does this issue occur? [OS: Linux] ### What is the version of your OS? Ubuntu 24.04 ### NuttX Version PX4's fork of NuttX, based on upstream Apache NuttX 10.3.0 ### Issue Architecture [Arch: arm] ### Issue Area [Area: Drivers] ### Host information _No response_ ### Verification - [x] I have verified before submitting the report. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
