Re: [PATCH RFC] hw/arm/virt: Avoid unexpected warning from Linux guest on host with Fujitsu CPUs
On Thu, Jun 6, 2024 at 7:57 PM Peter Maydell wrote: > > On Thu, 6 Jun 2024 at 11:48, Zhenyu Zhang wrote: > > > > Multiple warning messages and corresponding backtraces are observed when > > Linux > > guest is booted on the host with Fujitsu CPUs. One of them is shown as > > below. > > > > [0.032443] [ cut here ] > > [0.032446] uart-pl011 900.pl011: ARCH_DMA_MINALIGN smaller than > > CTR_EL0.CWG (128 < 256) > > [0.032454] WARNING: CPU: 0 PID: 1 at arch/arm64/mm/dma-mapping.c:54 > > arch_setup_dma_ops+0xbc/0xcc > > [0.032470] Modules linked in: > > [0.032475] CPU: 0 PID: 1 Comm: swapper/0 Not tainted > > 5.14.0-452.el9.aarch64 #1 > > [0.032481] Hardware name: linux,dummy-virt (DT) > > [0.032484] pstate: 6045 (nZCv daif +PAN -UAO -TCO -DIT -SSBS > > BTYPE=--) > > [0.032490] pc : arch_setup_dma_ops+0xbc/0xcc > > [0.032496] lr : arch_setup_dma_ops+0xbc/0xcc > > [0.032501] sp : 80008003b860 > > [0.032503] x29: 80008003b860 x28: x27: > > aae4b949049c > > [0.032510] x26: x25: x24: > > > > [0.032517] x23: 0100 x22: x21: > > > > [0.032523] x20: 0001 x19: 2f06c02ea400 x18: > > > > [0.032529] x17: 208a5f76 x16: 6589dbcb x15: > > aae4ba071c89 > > [0.032535] x14: x13: aae4ba071c84 x12: > > 455f525443206e61 > > [0.032541] x11: 68742072656c6c61 x10: 0029 x9 : > > aae4b7d21da4 > > [0.032547] x8 : 0029 x7 : 4c414e494d5f414d x6 : > > 0029 > > [0.032553] x5 : 000f x4 : aae4b9617a00 x3 : > > 0001 > > [0.032558] x2 : x1 : x0 : > > 2f06c029be40 > > [0.032564] Call trace: > > [0.032566] arch_setup_dma_ops+0xbc/0xcc > > [0.032572] of_dma_configure_id+0x138/0x300 > > [0.032591] amba_dma_configure+0x34/0xc0 > > [0.032600] really_probe+0x78/0x3dc > > [0.032614] __driver_probe_device+0x108/0x160 > > [0.032619] driver_probe_device+0x44/0x114 > > [0.032624] __device_attach_driver+0xb8/0x14c > > [0.032629] bus_for_each_drv+0x88/0xe4 > > [0.032634] __device_attach+0xb0/0x1e0 > > [0.032638] device_initial_probe+0x18/0x20 > > [0.032643] bus_probe_device+0xa8/0xb0 > > [0.032648] device_add+0x4b4/0x6c0 > > [0.032652] amba_device_try_add.part.0+0x48/0x360 > > [0.032657] amba_device_add+0x104/0x144 > > [0.032662] of_amba_device_create.isra.0+0x100/0x1c4 > > [0.032666] of_platform_bus_create+0x294/0x35c > > [0.032669] of_platform_populate+0x5c/0x150 > > [0.032672] of_platform_default_populate_init+0xd0/0xec > > [0.032697] do_one_initcall+0x4c/0x2e0 > > [0.032701] do_initcalls+0x100/0x13c > > [0.032707] kernel_init_freeable+0x1c8/0x21c > > [0.032712] kernel_init+0x28/0x140 > > [0.032731] ret_from_fork+0x10/0x20 > > [0.032735] ---[ end trace ]--- > > > > In Linux, a check is applied to every device which is exposed through > > device-tree > > node. The warning message is raised when the device isn't DMA coherent and > > the > > cache line size is larger than ARCH_DMA_MINALIGN (128 bytes). The cache > > line is > > sorted from CTR_EL0[CWG], which corresponds to 256 bytes on the guest CPUs. > > The DMA coherent capability is claimed through 'dma-coherent' in their > > device-tree nodes. > > For QEMU emulated all our DMA is always coherent, so where we > have DMA-capable devices we should definitely tell the kernel > that that DMA is coherent. > > Our pl011 does not do DMA, though (we do not set the dmas property), so > it's kind of bogus for the kernel to complain about that. > > So I think we should take these changes where they refer to DMA > capable devices and ask the kernel folks to fix the warnings > where they refer to devices that aren't doing DMA. Looking through > the patch, though, my initial impression is that all these are > in the latter category... My initial thought was to fix it in the kernel too. However, through preliminary research, I discovered some 'legacy reasons' mentioned in the discussion above. So I'm worried that the fix will bring new risks. > > > diff --git a/hw/arm/boot.c b/hw/arm/boot.c > > index d480a7da02..cdf99966e6 100644 > > --- a/hw/arm/boot.c > > +++ b/hw/arm/boot.c > > @@ -509,6 +509,7 @@ static void fdt_add_psci_node(void *fdt) > > qemu_fdt_setprop_cell(fdt, "/psci", "cpu_off", cpu_off_fn); > > qemu_fdt_setprop_cell(fdt, "/psci", "cpu_on", cpu_on_fn); > > qemu_fdt_setprop_cell(fdt, "/psci", "migrate", migrate_fn); > > +qemu_fdt_setprop(fdt, "/psci", "dma-coherent", NULL, 0); > > The PSCI node is describing the firmware interface for > HVC or SMC calls -- I don't think it makes any sense > to think of this as doing DMA.
Re: [PATCH RFC] hw/arm/virt: Avoid unexpected warning from Linux guest on host with Fujitsu CPUs
On Fri, Jun 7, 2024 at 6:17 PM Peter Maydell wrote: > > On Thu, 6 Jun 2024 at 22:18, Robin Murphy wrote: > > > > On 2024-06-06 6:13 pm, Jonathan Cameron wrote: > > > On Thu, 6 Jun 2024 12:56:59 +0100 > > > Peter Maydell wrote: > > > > > >> On Thu, 6 Jun 2024 at 11:48, Zhenyu Zhang wrote: > > >>> In Linux, a check is applied to every device which is exposed through > > >>> device-tree > > >>> node. The warning message is raised when the device isn't DMA coherent > > >>> and the > > >>> cache line size is larger than ARCH_DMA_MINALIGN (128 bytes). The cache > > >>> line is > > >>> sorted from CTR_EL0[CWG], which corresponds to 256 bytes on the guest > > >>> CPUs. > > >>> The DMA coherent capability is claimed through 'dma-coherent' in their > > >>> device-tree nodes. > > >> > > >> For QEMU emulated all our DMA is always coherent, so where we > > >> have DMA-capable devices we should definitely tell the kernel > > >> that that DMA is coherent. > > > > The trick for that is to put the "dma-coherent" property right in the > > root of the DT so it plausibly communicates "the whole platform is > > coherent", and is then inherited by all devices, even those which > > shouldn't technically need it. > > Ah, cool -- that's a pretty small change and it makes sense, and > avoids us having potential bugs in future where we forget to > mark a really-does-do-DMA device as dma-coherent. I like that > a lot better than adding incorrect dma-coherent tags to lots > of device nodes that aren't for DMA-capable devices. Hi Robin, Peter, and Jonathan, Thanks for your suggestions, it helps me a lot! I will submit a v2 patch that puts the "dma-coherent" property right in the root of the DT so it plausibly communicates. Yes, these discussions make sense. I will modify and test it again on Fujitsu host. Thanks again Zhenyu > > thanks > -- PMM >
Re: [PATCH RFC] hw/arm/virt: Avoid unexpected warning from Linux guest on host with Fujitsu CPUs
On Thu, 6 Jun 2024 at 22:18, Robin Murphy wrote: > > On 2024-06-06 6:13 pm, Jonathan Cameron wrote: > > On Thu, 6 Jun 2024 12:56:59 +0100 > > Peter Maydell wrote: > > > >> On Thu, 6 Jun 2024 at 11:48, Zhenyu Zhang wrote: > >>> In Linux, a check is applied to every device which is exposed through > >>> device-tree > >>> node. The warning message is raised when the device isn't DMA coherent > >>> and the > >>> cache line size is larger than ARCH_DMA_MINALIGN (128 bytes). The cache > >>> line is > >>> sorted from CTR_EL0[CWG], which corresponds to 256 bytes on the guest > >>> CPUs. > >>> The DMA coherent capability is claimed through 'dma-coherent' in their > >>> device-tree nodes. > >> > >> For QEMU emulated all our DMA is always coherent, so where we > >> have DMA-capable devices we should definitely tell the kernel > >> that that DMA is coherent. > > The trick for that is to put the "dma-coherent" property right in the > root of the DT so it plausibly communicates "the whole platform is > coherent", and is then inherited by all devices, even those which > shouldn't technically need it. Ah, cool -- that's a pretty small change and it makes sense, and avoids us having potential bugs in future where we forget to mark a really-does-do-DMA device as dma-coherent. I like that a lot better than adding incorrect dma-coherent tags to lots of device nodes that aren't for DMA-capable devices. thanks -- PMM
Re: [PATCH RFC] hw/arm/virt: Avoid unexpected warning from Linux guest on host with Fujitsu CPUs
On 2024-06-06 6:13 pm, Jonathan Cameron wrote: On Thu, 6 Jun 2024 12:56:59 +0100 Peter Maydell wrote: On Thu, 6 Jun 2024 at 11:48, Zhenyu Zhang wrote: Multiple warning messages and corresponding backtraces are observed when Linux guest is booted on the host with Fujitsu CPUs. One of them is shown as below. [0.032443] [ cut here ] [0.032446] uart-pl011 900.pl011: ARCH_DMA_MINALIGN smaller than CTR_EL0.CWG (128 < 256) [0.032454] WARNING: CPU: 0 PID: 1 at arch/arm64/mm/dma-mapping.c:54 arch_setup_dma_ops+0xbc/0xcc [0.032470] Modules linked in: [0.032475] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.14.0-452.el9.aarch64 #1 [0.032481] Hardware name: linux,dummy-virt (DT) [0.032484] pstate: 6045 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [0.032490] pc : arch_setup_dma_ops+0xbc/0xcc [0.032496] lr : arch_setup_dma_ops+0xbc/0xcc [0.032501] sp : 80008003b860 [0.032503] x29: 80008003b860 x28: x27: aae4b949049c [0.032510] x26: x25: x24: [0.032517] x23: 0100 x22: x21: [0.032523] x20: 0001 x19: 2f06c02ea400 x18: [0.032529] x17: 208a5f76 x16: 6589dbcb x15: aae4ba071c89 [0.032535] x14: x13: aae4ba071c84 x12: 455f525443206e61 [0.032541] x11: 68742072656c6c61 x10: 0029 x9 : aae4b7d21da4 [0.032547] x8 : 0029 x7 : 4c414e494d5f414d x6 : 0029 [0.032553] x5 : 000f x4 : aae4b9617a00 x3 : 0001 [0.032558] x2 : x1 : x0 : 2f06c029be40 [0.032564] Call trace: [0.032566] arch_setup_dma_ops+0xbc/0xcc [0.032572] of_dma_configure_id+0x138/0x300 [0.032591] amba_dma_configure+0x34/0xc0 [0.032600] really_probe+0x78/0x3dc [0.032614] __driver_probe_device+0x108/0x160 [0.032619] driver_probe_device+0x44/0x114 [0.032624] __device_attach_driver+0xb8/0x14c [0.032629] bus_for_each_drv+0x88/0xe4 [0.032634] __device_attach+0xb0/0x1e0 [0.032638] device_initial_probe+0x18/0x20 [0.032643] bus_probe_device+0xa8/0xb0 [0.032648] device_add+0x4b4/0x6c0 [0.032652] amba_device_try_add.part.0+0x48/0x360 [0.032657] amba_device_add+0x104/0x144 [0.032662] of_amba_device_create.isra.0+0x100/0x1c4 [0.032666] of_platform_bus_create+0x294/0x35c [0.032669] of_platform_populate+0x5c/0x150 [0.032672] of_platform_default_populate_init+0xd0/0xec [0.032697] do_one_initcall+0x4c/0x2e0 [0.032701] do_initcalls+0x100/0x13c [0.032707] kernel_init_freeable+0x1c8/0x21c [0.032712] kernel_init+0x28/0x140 [0.032731] ret_from_fork+0x10/0x20 [0.032735] ---[ end trace ]--- In Linux, a check is applied to every device which is exposed through device-tree node. The warning message is raised when the device isn't DMA coherent and the cache line size is larger than ARCH_DMA_MINALIGN (128 bytes). The cache line is sorted from CTR_EL0[CWG], which corresponds to 256 bytes on the guest CPUs. The DMA coherent capability is claimed through 'dma-coherent' in their device-tree nodes. For QEMU emulated all our DMA is always coherent, so where we have DMA-capable devices we should definitely tell the kernel that that DMA is coherent. The trick for that is to put the "dma-coherent" property right in the root of the DT so it plausibly communicates "the whole platform is coherent", and is then inherited by all devices, even those which shouldn't technically need it. Our pl011 does not do DMA, though (we do not set the dmas property), so it's kind of bogus for the kernel to complain about that. The issue there is, per the history Jonathan dug up, DT on Arm got the assumption baked into it from day one that "dma-ranges" was implied for simple-bus and similar, and thus there is no easy generic way to indicate that any MMIO device *can't* do DMA. For Linux this means we end up having to assume that everything *might* be DMA-capable, since the only thing which knows for sure is a driver, but we have further legacy in the driver model which means we have to do perform the basic DMA setup for any device *before* its driver probes. Yes, it's a bit rubbish; feel free to shake your fist at the past. (At least we learned and got it right in ACPI for arm64 by making the _CCA method mandatory for DMA-capable devices...) So I think we should take these changes where they refer to DMA capable devices and ask the kernel folks to fix the warnings where they refer to devices that aren't doing DMA. Looking through the patch, though, my initial impression is that all these are in the latter category... I was curious and have a very slow test running, so took a look. of_dma_configure() is being passed force_dma = true.
Re: [PATCH RFC] hw/arm/virt: Avoid unexpected warning from Linux guest on host with Fujitsu CPUs
On Thu, 6 Jun 2024 12:56:59 +0100 Peter Maydell wrote: > On Thu, 6 Jun 2024 at 11:48, Zhenyu Zhang wrote: > > > > Multiple warning messages and corresponding backtraces are observed when > > Linux > > guest is booted on the host with Fujitsu CPUs. One of them is shown as > > below. > > > > [0.032443] [ cut here ] > > [0.032446] uart-pl011 900.pl011: ARCH_DMA_MINALIGN smaller than > > CTR_EL0.CWG (128 < 256) > > [0.032454] WARNING: CPU: 0 PID: 1 at arch/arm64/mm/dma-mapping.c:54 > > arch_setup_dma_ops+0xbc/0xcc > > [0.032470] Modules linked in: > > [0.032475] CPU: 0 PID: 1 Comm: swapper/0 Not tainted > > 5.14.0-452.el9.aarch64 #1 > > [0.032481] Hardware name: linux,dummy-virt (DT) > > [0.032484] pstate: 6045 (nZCv daif +PAN -UAO -TCO -DIT -SSBS > > BTYPE=--) > > [0.032490] pc : arch_setup_dma_ops+0xbc/0xcc > > [0.032496] lr : arch_setup_dma_ops+0xbc/0xcc > > [0.032501] sp : 80008003b860 > > [0.032503] x29: 80008003b860 x28: x27: > > aae4b949049c > > [0.032510] x26: x25: x24: > > > > [0.032517] x23: 0100 x22: x21: > > > > [0.032523] x20: 0001 x19: 2f06c02ea400 x18: > > > > [0.032529] x17: 208a5f76 x16: 6589dbcb x15: > > aae4ba071c89 > > [0.032535] x14: x13: aae4ba071c84 x12: > > 455f525443206e61 > > [0.032541] x11: 68742072656c6c61 x10: 0029 x9 : > > aae4b7d21da4 > > [0.032547] x8 : 0029 x7 : 4c414e494d5f414d x6 : > > 0029 > > [0.032553] x5 : 000f x4 : aae4b9617a00 x3 : > > 0001 > > [0.032558] x2 : x1 : x0 : > > 2f06c029be40 > > [0.032564] Call trace: > > [0.032566] arch_setup_dma_ops+0xbc/0xcc > > [0.032572] of_dma_configure_id+0x138/0x300 > > [0.032591] amba_dma_configure+0x34/0xc0 > > [0.032600] really_probe+0x78/0x3dc > > [0.032614] __driver_probe_device+0x108/0x160 > > [0.032619] driver_probe_device+0x44/0x114 > > [0.032624] __device_attach_driver+0xb8/0x14c > > [0.032629] bus_for_each_drv+0x88/0xe4 > > [0.032634] __device_attach+0xb0/0x1e0 > > [0.032638] device_initial_probe+0x18/0x20 > > [0.032643] bus_probe_device+0xa8/0xb0 > > [0.032648] device_add+0x4b4/0x6c0 > > [0.032652] amba_device_try_add.part.0+0x48/0x360 > > [0.032657] amba_device_add+0x104/0x144 > > [0.032662] of_amba_device_create.isra.0+0x100/0x1c4 > > [0.032666] of_platform_bus_create+0x294/0x35c > > [0.032669] of_platform_populate+0x5c/0x150 > > [0.032672] of_platform_default_populate_init+0xd0/0xec > > [0.032697] do_one_initcall+0x4c/0x2e0 > > [0.032701] do_initcalls+0x100/0x13c > > [0.032707] kernel_init_freeable+0x1c8/0x21c > > [0.032712] kernel_init+0x28/0x140 > > [0.032731] ret_from_fork+0x10/0x20 > > [0.032735] ---[ end trace ]--- > > > > In Linux, a check is applied to every device which is exposed through > > device-tree > > node. The warning message is raised when the device isn't DMA coherent and > > the > > cache line size is larger than ARCH_DMA_MINALIGN (128 bytes). The cache > > line is > > sorted from CTR_EL0[CWG], which corresponds to 256 bytes on the guest CPUs. > > The DMA coherent capability is claimed through 'dma-coherent' in their > > device-tree nodes. > > For QEMU emulated all our DMA is always coherent, so where we > have DMA-capable devices we should definitely tell the kernel > that that DMA is coherent. > > Our pl011 does not do DMA, though (we do not set the dmas property), so > it's kind of bogus for the kernel to complain about that. > > So I think we should take these changes where they refer to DMA > capable devices and ask the kernel folks to fix the warnings > where they refer to devices that aren't doing DMA. Looking through > the patch, though, my initial impression is that all these are > in the latter category... I was curious and have a very slow test running, so took a look. of_dma_configure() is being passed force_dma = true. https://elixir.bootlin.com/linux/v6.10-rc2/source/drivers/amba/bus.c#L361 The is a comment in of_dma_configure() /* * For legacy reasons, we have to assume some devices need * DMA configuration regardless of whether "dma-ranges" is * correctly specified or not. */ So this I think this is being triggered by a workaround for broken DT. This was introduced by Robin Murphy +CC though you may need to ask on kernel list because ARM / QEMU fun. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=723288836628b Relevant comment from that patch description: "Certain bus types have a general
Re: [PATCH RFC] hw/arm/virt: Avoid unexpected warning from Linux guest on host with Fujitsu CPUs
On Thu, 6 Jun 2024 at 11:48, Zhenyu Zhang wrote: > > Multiple warning messages and corresponding backtraces are observed when Linux > guest is booted on the host with Fujitsu CPUs. One of them is shown as below. > > [0.032443] [ cut here ] > [0.032446] uart-pl011 900.pl011: ARCH_DMA_MINALIGN smaller than > CTR_EL0.CWG (128 < 256) > [0.032454] WARNING: CPU: 0 PID: 1 at arch/arm64/mm/dma-mapping.c:54 > arch_setup_dma_ops+0xbc/0xcc > [0.032470] Modules linked in: > [0.032475] CPU: 0 PID: 1 Comm: swapper/0 Not tainted > 5.14.0-452.el9.aarch64 #1 > [0.032481] Hardware name: linux,dummy-virt (DT) > [0.032484] pstate: 6045 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) > [0.032490] pc : arch_setup_dma_ops+0xbc/0xcc > [0.032496] lr : arch_setup_dma_ops+0xbc/0xcc > [0.032501] sp : 80008003b860 > [0.032503] x29: 80008003b860 x28: x27: > aae4b949049c > [0.032510] x26: x25: x24: > > [0.032517] x23: 0100 x22: x21: > > [0.032523] x20: 0001 x19: 2f06c02ea400 x18: > > [0.032529] x17: 208a5f76 x16: 6589dbcb x15: > aae4ba071c89 > [0.032535] x14: x13: aae4ba071c84 x12: > 455f525443206e61 > [0.032541] x11: 68742072656c6c61 x10: 0029 x9 : > aae4b7d21da4 > [0.032547] x8 : 0029 x7 : 4c414e494d5f414d x6 : > 0029 > [0.032553] x5 : 000f x4 : aae4b9617a00 x3 : > 0001 > [0.032558] x2 : x1 : x0 : > 2f06c029be40 > [0.032564] Call trace: > [0.032566] arch_setup_dma_ops+0xbc/0xcc > [0.032572] of_dma_configure_id+0x138/0x300 > [0.032591] amba_dma_configure+0x34/0xc0 > [0.032600] really_probe+0x78/0x3dc > [0.032614] __driver_probe_device+0x108/0x160 > [0.032619] driver_probe_device+0x44/0x114 > [0.032624] __device_attach_driver+0xb8/0x14c > [0.032629] bus_for_each_drv+0x88/0xe4 > [0.032634] __device_attach+0xb0/0x1e0 > [0.032638] device_initial_probe+0x18/0x20 > [0.032643] bus_probe_device+0xa8/0xb0 > [0.032648] device_add+0x4b4/0x6c0 > [0.032652] amba_device_try_add.part.0+0x48/0x360 > [0.032657] amba_device_add+0x104/0x144 > [0.032662] of_amba_device_create.isra.0+0x100/0x1c4 > [0.032666] of_platform_bus_create+0x294/0x35c > [0.032669] of_platform_populate+0x5c/0x150 > [0.032672] of_platform_default_populate_init+0xd0/0xec > [0.032697] do_one_initcall+0x4c/0x2e0 > [0.032701] do_initcalls+0x100/0x13c > [0.032707] kernel_init_freeable+0x1c8/0x21c > [0.032712] kernel_init+0x28/0x140 > [0.032731] ret_from_fork+0x10/0x20 > [0.032735] ---[ end trace ]--- > > In Linux, a check is applied to every device which is exposed through > device-tree > node. The warning message is raised when the device isn't DMA coherent and the > cache line size is larger than ARCH_DMA_MINALIGN (128 bytes). The cache line > is > sorted from CTR_EL0[CWG], which corresponds to 256 bytes on the guest CPUs. > The DMA coherent capability is claimed through 'dma-coherent' in their > device-tree nodes. For QEMU emulated all our DMA is always coherent, so where we have DMA-capable devices we should definitely tell the kernel that that DMA is coherent. Our pl011 does not do DMA, though (we do not set the dmas property), so it's kind of bogus for the kernel to complain about that. So I think we should take these changes where they refer to DMA capable devices and ask the kernel folks to fix the warnings where they refer to devices that aren't doing DMA. Looking through the patch, though, my initial impression is that all these are in the latter category... > diff --git a/hw/arm/boot.c b/hw/arm/boot.c > index d480a7da02..cdf99966e6 100644 > --- a/hw/arm/boot.c > +++ b/hw/arm/boot.c > @@ -509,6 +509,7 @@ static void fdt_add_psci_node(void *fdt) > qemu_fdt_setprop_cell(fdt, "/psci", "cpu_off", cpu_off_fn); > qemu_fdt_setprop_cell(fdt, "/psci", "cpu_on", cpu_on_fn); > qemu_fdt_setprop_cell(fdt, "/psci", "migrate", migrate_fn); > +qemu_fdt_setprop(fdt, "/psci", "dma-coherent", NULL, 0); The PSCI node is describing the firmware interface for HVC or SMC calls -- I don't think it makes any sense to think of this as doing DMA. So I would query the kernel folks about this warning. > } > > int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo, > diff --git a/hw/arm/virt.c b/hw/arm/virt.c > index 3c93c0c0a6..d3e5f512e2 100644 > --- a/hw/arm/virt.c > +++ b/hw/arm/virt.c > @@ -652,6 +652,7 @@ static void fdt_add_pmu_nodes(const VirtMachineState *vms) > qemu_fdt_setprop_cells(ms->fdt, "/pmu", "interrupts", > GIC_FDT_IRQ_TYPE_PPI, >
[PATCH RFC] hw/arm/virt: Avoid unexpected warning from Linux guest on host with Fujitsu CPUs
Multiple warning messages and corresponding backtraces are observed when Linux
guest is booted on the host with Fujitsu CPUs. One of them is shown as below.
[0.032443] [ cut here ]
[0.032446] uart-pl011 900.pl011: ARCH_DMA_MINALIGN smaller than
CTR_EL0.CWG (128 < 256)
[0.032454] WARNING: CPU: 0 PID: 1 at arch/arm64/mm/dma-mapping.c:54
arch_setup_dma_ops+0xbc/0xcc
[0.032470] Modules linked in:
[0.032475] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.14.0-452.el9.aarch64
#1
[0.032481] Hardware name: linux,dummy-virt (DT)
[0.032484] pstate: 6045 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[0.032490] pc : arch_setup_dma_ops+0xbc/0xcc
[0.032496] lr : arch_setup_dma_ops+0xbc/0xcc
[0.032501] sp : 80008003b860
[0.032503] x29: 80008003b860 x28: x27: aae4b949049c
[0.032510] x26: x25: x24:
[0.032517] x23: 0100 x22: x21:
[0.032523] x20: 0001 x19: 2f06c02ea400 x18:
[0.032529] x17: 208a5f76 x16: 6589dbcb x15: aae4ba071c89
[0.032535] x14: x13: aae4ba071c84 x12: 455f525443206e61
[0.032541] x11: 68742072656c6c61 x10: 0029 x9 : aae4b7d21da4
[0.032547] x8 : 0029 x7 : 4c414e494d5f414d x6 : 0029
[0.032553] x5 : 000f x4 : aae4b9617a00 x3 : 0001
[0.032558] x2 : x1 : x0 : 2f06c029be40
[0.032564] Call trace:
[0.032566] arch_setup_dma_ops+0xbc/0xcc
[0.032572] of_dma_configure_id+0x138/0x300
[0.032591] amba_dma_configure+0x34/0xc0
[0.032600] really_probe+0x78/0x3dc
[0.032614] __driver_probe_device+0x108/0x160
[0.032619] driver_probe_device+0x44/0x114
[0.032624] __device_attach_driver+0xb8/0x14c
[0.032629] bus_for_each_drv+0x88/0xe4
[0.032634] __device_attach+0xb0/0x1e0
[0.032638] device_initial_probe+0x18/0x20
[0.032643] bus_probe_device+0xa8/0xb0
[0.032648] device_add+0x4b4/0x6c0
[0.032652] amba_device_try_add.part.0+0x48/0x360
[0.032657] amba_device_add+0x104/0x144
[0.032662] of_amba_device_create.isra.0+0x100/0x1c4
[0.032666] of_platform_bus_create+0x294/0x35c
[0.032669] of_platform_populate+0x5c/0x150
[0.032672] of_platform_default_populate_init+0xd0/0xec
[0.032697] do_one_initcall+0x4c/0x2e0
[0.032701] do_initcalls+0x100/0x13c
[0.032707] kernel_init_freeable+0x1c8/0x21c
[0.032712] kernel_init+0x28/0x140
[0.032731] ret_from_fork+0x10/0x20
[0.032735] ---[ end trace ]---
In Linux, a check is applied to every device which is exposed through
device-tree
node. The warning message is raised when the device isn't DMA coherent and the
cache line size is larger than ARCH_DMA_MINALIGN (128 bytes). The cache line is
sorted from CTR_EL0[CWG], which corresponds to 256 bytes on the guest CPUs.
The DMA coherent capability is claimed through 'dma-coherent' in their
device-tree nodes.
I don't see those devices need to be DMA incoherent necessarily and those
devices
will do DMA operations in practice. So lets add 'dma-coherent' property to their
device-tree nodes to avoid the unexpected warnings in the Linux guest.
Signed-off-by: Zhenyu Zhang
---
hw/arm/boot.c| 1 +
hw/arm/virt.c| 4
hw/core/sysbus-fdt.c | 1 +
3 files changed, 6 insertions(+)
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index d480a7da02..cdf99966e6 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -509,6 +509,7 @@ static void fdt_add_psci_node(void *fdt)
qemu_fdt_setprop_cell(fdt, "/psci", "cpu_off", cpu_off_fn);
qemu_fdt_setprop_cell(fdt, "/psci", "cpu_on", cpu_on_fn);
qemu_fdt_setprop_cell(fdt, "/psci", "migrate", migrate_fn);
+qemu_fdt_setprop(fdt, "/psci", "dma-coherent", NULL, 0);
}
int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo,
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 3c93c0c0a6..d3e5f512e2 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -652,6 +652,7 @@ static void fdt_add_pmu_nodes(const VirtMachineState *vms)
qemu_fdt_setprop_cells(ms->fdt, "/pmu", "interrupts",
GIC_FDT_IRQ_TYPE_PPI,
INTID_TO_PPI(VIRTUAL_PMU_IRQ), irqflags);
+qemu_fdt_setprop(ms->fdt, "/pmu", "dma-coherent", NULL, 0);
}
}
@@ -936,6 +937,7 @@ static void create_uart(const VirtMachineState *vms, int
uart,
vms->clock_phandle, vms->clock_phandle);
qemu_fdt_setprop(ms->fdt, nodename, "clock-names",
clocknames, sizeof(clocknames));
+qemu_fdt_setprop(ms->fdt, nodename, "dma-coherent", NULL, 0);
if (uart == VIRT_UART) {
qemu_fdt_setprop_string(ms->fdt, "/chosen", "stdout-path", nodename);
@@ -972,6 +974,7 @@
