Re: [PATCH v8 8/8] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

On Mon, Aug 12, 2024 at 10:48:40AM +1000, Alistair Francis wrote:
> [EXTERNAL MAIL]
> 
> On Fri, Aug 9, 2024 at 8:14 PM Ethan Chen  wrote:
> >
> > On Thu, Aug 08, 2024 at 02:01:13PM +1000, Alistair Francis wrote:
> > >
> > > On Mon, Jul 15, 2024 at 8:15 PM Ethan Chen via  
> > > wrote:
> > > >
> > > > - Add 'iopmp=on' option to enable IOPMP. It adds an iopmp device virt 
> > > > machine
> > > >   to protect all regions of system memory, and configures RRID of CPU.
> > > >
> > > > Signed-off-by: Ethan Chen 
> > > > ---
> > > >  docs/system/riscv/virt.rst |  5 +++
> > > >  hw/riscv/Kconfig   |  1 +
> > > >  hw/riscv/virt.c| 63 ++
> > > >  include/hw/riscv/virt.h|  5 ++-
> > > >  4 files changed, 73 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
> > > > index 9a06f95a34..9fd006ccc2 100644
> > > > --- a/docs/system/riscv/virt.rst
> > > > +++ b/docs/system/riscv/virt.rst
> > > > @@ -116,6 +116,11 @@ The following machine-specific options are 
> > > > supported:
> > > >having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not 
> > > > specified,
> > > >the default number of per-HART VS-level AIA IMSIC pages is 0.
> > > >
> > > > +- iopmp=[on|off]
> > > > +
> > > > +  When this option is "on", an IOPMP device is added to machine. IOPMP 
> > > > checks
> > > > +  memory transcations in system memory. This option is assumed to be 
> > > > "off".
> > >
> > > We probably should have a a little more here. You don't even mention
> > > that this is the rapid-k model.
> >
> > I'll provide more details.
> >
> > >
> > > It might be worth adding a `model` field, to make it easier to add
> > > other models in the future. Thoughts?
> > >
> >
> > I think the IOPMP model should be a device property and not
> > configured here.
> 
> It should be a device property, but then how does a user configure
> that? I guess users can globally set device props, but it's a bit
> clunky
>

Because IOPMP has a lot props, I think it is better to configure them 
through global device props instead of machine option.

Thanks,
Ethan Chen

Re: [PATCH v8 5/8] hw/misc/riscv_iopmp: Add API to set up IOPMP protection for system memory

[Ethan Chen via]

On Mon, Aug 12, 2024 at 10:47:33AM +1000, Alistair Francis wrote:
> [EXTERNAL MAIL]
> 
> On Fri, Aug 9, 2024 at 8:11 PM Ethan Chen  wrote:
> >
> > On Thu, Aug 08, 2024 at 02:23:56PM +1000, Alistair Francis wrote:
> > >
> > > On Mon, Jul 15, 2024 at 8:13 PM Ethan Chen via  
> > > wrote:
> > > >
> > > > To enable system memory transactions through the IOPMP, memory regions 
> > > > must
> > > > be moved to the IOPMP downstream and then replaced with IOMMUs for IOPMP
> > > > translation.
> > > >
> > > > The iopmp_setup_system_memory() function copies subregions of system 
> > > > memory
> > > > to create the IOPMP downstream and then replaces the specified memory
> > > > regions in system memory with the IOMMU regions of the IOPMP. It also
> > > > adds entries to a protection map that records the relationship between
> > > > physical address regions and the IOPMP, which is used by the IOPMP DMA
> > > > API to send transaction information.
> > > >
> > > > Signed-off-by: Ethan Chen 
> > > > ---
> > > >  hw/misc/riscv_iopmp.c | 61 +++
> > > >  include/hw/misc/riscv_iopmp.h |  3 ++
> > > >  2 files changed, 64 insertions(+)
> > > >
> > > > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > > > index db43e3c73f..e62ac57437 100644
> > > > --- a/hw/misc/riscv_iopmp.c
> > > > +++ b/hw/misc/riscv_iopmp.c
> > > > @@ -1151,4 +1151,65 @@ iopmp_register_types(void)
> > > >  type_register_static(&iopmp_iommu_memory_region_info);
> > > >  }
> > > >
> > > > +/*
> > > > + * Copies subregions from the source memory region to the destination 
> > > > memory
> > > > + * region
> > > > + */
> > > > +static void copy_memory_subregions(MemoryRegion *src_mr, MemoryRegion 
> > > > *dst_mr)
> 
> Maybe `alias_memory_subregions()` or `link_memory_subregions()`
> instead of `copy_memory_subregions()`.

Thanks for the suggestion. I will clarify it.

> 
> > > > +{
> > > > +int32_t priority;
> > > > +hwaddr addr;
> > > > +MemoryRegion *alias, *subregion;
> > > > +QTAILQ_FOREACH(subregion, &src_mr->subregions, subregions_link) {
> > > > +priority = subregion->priority;
> > > > +addr = subregion->addr;
> > > > +alias = g_malloc0(sizeof(MemoryRegion));
> > > > +memory_region_init_alias(alias, NULL, subregion->name, 
> > > > subregion, 0,
> > > > + memory_region_size(subregion));
> > > > +memory_region_add_subregion_overlap(dst_mr, addr, alias, 
> > > > priority);
> > > > +}
> > > > +}
> > >
> > > This seems strange. Do we really need to do this?
> > >
> > > I haven't looked at the memory_region stuff for awhile, but this seems
> > > clunky and prone to breakage.
> > >
> > > We already link s->iommu with the system memory
> > >
> >
> > s->iommu occupies the address of the protected devices in system
> > memory. Since IOPMP does not alter address, the target address space
> > must differ from system memory to avoid infinite recursive iommu access.
> >
> > The transaction will be redirected to a downstream memory region, which
> > is almost identical to system memory but without the iommu memory
> > region of IOPMP.
> >
> > This function serves as a helper to create that downstream memory region.
> 
> What I don't understand is that we already have target_mr as a
> subregion of downstream, is that not enough?
>

Did you mean the target_mr in iopmp_setup_system_memory? It specifies
the container that IOPMP wants to protect. We don't create 
separate iommus for each subregion. We create a single iommu for the
entire container (system memory).

The access to target_mr (system memory) which has iommu region of 
IOPMP, will be translated to downstream which has protected device 
regions.

Thanks,
Ethan Chen

Re: [PATCH v8 8/8] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

On Thu, Aug 08, 2024 at 02:01:13PM +1000, Alistair Francis wrote:
> 
> On Mon, Jul 15, 2024 at 8:15 PM Ethan Chen via  wrote:
> >
> > - Add 'iopmp=on' option to enable IOPMP. It adds an iopmp device virt 
> > machine
> >   to protect all regions of system memory, and configures RRID of CPU.
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  docs/system/riscv/virt.rst |  5 +++
> >  hw/riscv/Kconfig   |  1 +
> >  hw/riscv/virt.c| 63 ++
> >  include/hw/riscv/virt.h|  5 ++-
> >  4 files changed, 73 insertions(+), 1 deletion(-)
> >
> > diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
> > index 9a06f95a34..9fd006ccc2 100644
> > --- a/docs/system/riscv/virt.rst
> > +++ b/docs/system/riscv/virt.rst
> > @@ -116,6 +116,11 @@ The following machine-specific options are supported:
> >having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not specified,
> >the default number of per-HART VS-level AIA IMSIC pages is 0.
> >
> > +- iopmp=[on|off]
> > +
> > +  When this option is "on", an IOPMP device is added to machine. IOPMP 
> > checks
> > +  memory transcations in system memory. This option is assumed to be "off".
> 
> We probably should have a a little more here. You don't even mention
> that this is the rapid-k model.

I'll provide more details.

> 
> It might be worth adding a `model` field, to make it easier to add
> other models in the future. Thoughts?
>

I think the IOPMP model should be a device property and not 
configured here.

Thanks,
Ethan Chen

> Alistair
> 
> > +
> >  Running Linux kernel
> >  
> >
> > diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
> > index a2030e3a6f..0b45a5ade2 100644
> > --- a/hw/riscv/Kconfig
> > +++ b/hw/riscv/Kconfig
> > @@ -56,6 +56,7 @@ config RISCV_VIRT
> >  select PLATFORM_BUS
> >  select ACPI
> >  select ACPI_PCI
> > +select RISCV_IOPMP
> >
> >  config SHAKTI_C
> >  bool
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index bc0893e087..5a03c03c4a 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -55,6 +55,7 @@
> >  #include "hw/acpi/aml-build.h"
> >  #include "qapi/qapi-visit-common.h"
> >  #include "hw/virtio/virtio-iommu.h"
> > +#include "hw/misc/riscv_iopmp.h"
> >
> >  /* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by 
> > QEMU. */
> >  static bool virt_use_kvm_aia(RISCVVirtState *s)
> > @@ -82,6 +83,7 @@ static const MemMapEntry virt_memmap[] = {
> >  [VIRT_UART0] ={ 0x1000, 0x100 },
> >  [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
> >  [VIRT_FW_CFG] =   { 0x1010,  0x18 },
> > +[VIRT_IOPMP] ={ 0x1020,  0x10 },
> >  [VIRT_FLASH] ={ 0x2000, 0x400 },
> >  [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
> >  [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
> > @@ -90,6 +92,11 @@ static const MemMapEntry virt_memmap[] = {
> >  [VIRT_DRAM] = { 0x8000,   0x0 },
> >  };
> >
> > +static const MemMapEntry iopmp_protect_memmap[] = {
> > +/* IOPMP protect all regions by default */
> > +{0, 0x},
> > +};
> > +
> >  /* PCIe high mmio is fixed for RV32 */
> >  #define VIRT32_HIGH_PCIE_MMIO_BASE  0x3ULL
> >  #define VIRT32_HIGH_PCIE_MMIO_SIZE  (4 * GiB)
> > @@ -1024,6 +1031,24 @@ static void create_fdt_virtio_iommu(RISCVVirtState 
> > *s, uint16_t bdf)
> > bdf + 1, iommu_phandle, bdf + 1, 0x - bdf);
> >  }
> >
> > +static void create_fdt_iopmp(RISCVVirtState *s, const MemMapEntry *memmap,
> > + uint32_t irq_mmio_phandle) {
> > +g_autofree char *name = NULL;
> > +MachineState *ms = MACHINE(s);
> > +
> > +name = g_strdup_printf("/soc/iopmp@%lx", 
> > (long)memmap[VIRT_IOPMP].base);
> > +qemu_fdt_add_subnode(ms->fdt, name);
> > +qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
> > +qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, 
> > memmap[VIRT_IOPMP].base,
> > +0x0, memmap[VIRT_IOPMP].size);
> > +qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", 
> > irq_mmio_phandle);
> &

Re: [PATCH v8 5/8] hw/misc/riscv_iopmp: Add API to set up IOPMP protection for system memory

[Ethan Chen via]

On Thu, Aug 08, 2024 at 02:23:56PM +1000, Alistair Francis wrote:
> 
> On Mon, Jul 15, 2024 at 8:13 PM Ethan Chen via  wrote:
> >
> > To enable system memory transactions through the IOPMP, memory regions must
> > be moved to the IOPMP downstream and then replaced with IOMMUs for IOPMP
> > translation.
> >
> > The iopmp_setup_system_memory() function copies subregions of system memory
> > to create the IOPMP downstream and then replaces the specified memory
> > regions in system memory with the IOMMU regions of the IOPMP. It also
> > adds entries to a protection map that records the relationship between
> > physical address regions and the IOPMP, which is used by the IOPMP DMA
> > API to send transaction information.
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  hw/misc/riscv_iopmp.c | 61 +++
> >  include/hw/misc/riscv_iopmp.h |  3 ++
> >  2 files changed, 64 insertions(+)
> >
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > index db43e3c73f..e62ac57437 100644
> > --- a/hw/misc/riscv_iopmp.c
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -1151,4 +1151,65 @@ iopmp_register_types(void)
> >  type_register_static(&iopmp_iommu_memory_region_info);
> >  }
> >
> > +/*
> > + * Copies subregions from the source memory region to the destination 
> > memory
> > + * region
> > + */
> > +static void copy_memory_subregions(MemoryRegion *src_mr, MemoryRegion 
> > *dst_mr)
> > +{
> > +int32_t priority;
> > +hwaddr addr;
> > +MemoryRegion *alias, *subregion;
> > +QTAILQ_FOREACH(subregion, &src_mr->subregions, subregions_link) {
> > +priority = subregion->priority;
> > +addr = subregion->addr;
> > +alias = g_malloc0(sizeof(MemoryRegion));
> > +memory_region_init_alias(alias, NULL, subregion->name, subregion, 
> > 0,
> > + memory_region_size(subregion));
> > +memory_region_add_subregion_overlap(dst_mr, addr, alias, priority);
> > +}
> > +}
> 
> This seems strange. Do we really need to do this?
> 
> I haven't looked at the memory_region stuff for awhile, but this seems
> clunky and prone to breakage.
> 
> We already link s->iommu with the system memory
>

s->iommu occupies the address of the protected devices in system
memory. Since IOPMP does not alter address, the target address space 
must differ from system memory to avoid infinite recursive iommu access.

The transaction will be redirected to a downstream memory region, which
is almost identical to system memory but without the iommu memory
region of IOPMP. 

This function serves as a helper to create that downstream memory region.

Thanks,
Ethan Chen

> Alistair
> 
> > +
> > +/*
> > + * Create downstream of system memory for IOPMP, and overlap memory region
> > + * specified in memmap with IOPMP translator. Make sure subregions are 
> > added to
> > + * system memory before call this function. It also add entry to
> > + * iopmp_protection_memmaps for recording the relationship between physical
> > + * address regions and IOPMP.
> > + */
> > +void iopmp_setup_system_memory(DeviceState *dev, const MemMapEntry *memmap,
> > +   uint32_t map_entry_num)
> > +{
> > +IopmpState *s = IOPMP(dev);
> > +uint32_t i;
> > +MemoryRegion *iommu_alias;
> > +MemoryRegion *target_mr = get_system_memory();
> > +MemoryRegion *downstream = g_malloc0(sizeof(MemoryRegion));
> > +memory_region_init(downstream, NULL, "iopmp_downstream",
> > +   memory_region_size(target_mr));
> > +/* Copy subregions of target to downstream */
> > +copy_memory_subregions(target_mr, downstream);
> > +
> > +iopmp_protection_memmap *map;
> > +for (i = 0; i < map_entry_num; i++) {
> > +/* Memory access to protected regions of target are through IOPMP 
> > */
> > +iommu_alias = g_new(MemoryRegion, 1);
> > +memory_region_init_alias(iommu_alias, NULL, "iommu_alias",
> > + MEMORY_REGION(&s->iommu), memmap[i].base,
> > + memmap[i].size);
> > +memory_region_add_subregion_overlap(target_mr, memmap[i].base,
> > +iommu_alias, 1);
> > +/* Record which IOPMP is responsible for the region */
> > +map = g_new0(iopmp_protection_memmap, 1);
> > +map->io

Re: [PATCH v8 4/8] hw/misc/riscv_iopmp: Add RISC-V IOPMP device

[Ethan Chen via]

On Thu, Aug 08, 2024 at 01:56:35PM +1000, Alistair Francis wrote:
> [EXTERNAL MAIL]
> 
> On Mon, Jul 15, 2024 at 7:58 PM Ethan Chen via  wrote:
> >
> > Support basic functions of IOPMP specification v0.9.1 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1
> >
> > The IOPMP checks whether memory access from a device or CPU is valid.
> > This implementation uses an IOMMU to modify the address space accessed
> > by the device.
> >
> > For device access with IOMMUAccessFlags specifying read or write
> > (IOMMU_RO or IOMMU_WO), the IOPMP checks the permission in
> > iopmp_translate. If the access is valid, the target address space is
> > downstream_as. If the access is blocked, it will be redirected to
> > blocked_rwx_as.
> >
> > For CPU access with IOMMUAccessFlags not specifying read or write
> > (IOMMU_NONE), the IOPMP translates the access to the corresponding
> > address space based on the permission. If the access has full permission
> > (rwx), the target address space is downstream_as. If the access has
> > limited permission, the target address space is blocked_ followed by
> > the lacked permissions.
> >
> > The operation of a blocked region can trigger an IOPMP interrupt, a bus
> > error, or it can respond with success and fabricated data, depending on
> > the value of the IOPMP ERR_CFG register.
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  hw/misc/Kconfig   |3 +
> >  hw/misc/meson.build   |1 +
> >  hw/misc/riscv_iopmp.c | 1154 +
> >  hw/misc/trace-events  |3 +
> >  include/hw/misc/riscv_iopmp.h |  168 +
> >  5 files changed, 1329 insertions(+)
> >  create mode 100644 hw/misc/riscv_iopmp.c
> >  create mode 100644 include/hw/misc/riscv_iopmp.h
> >
> > diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
> > index 1e08785b83..427f0c702e 100644
> > --- a/hw/misc/Kconfig
> > +++ b/hw/misc/Kconfig
> > @@ -213,4 +213,7 @@ config IOSB
> >  config XLNX_VERSAL_TRNG
> >  bool
> >
> > +config RISCV_IOPMP
> > +bool
> > +
> >  source macio/Kconfig
> > diff --git a/hw/misc/meson.build b/hw/misc/meson.build
> > index 2ca8717be2..d9006e1d81 100644
> > --- a/hw/misc/meson.build
> > +++ b/hw/misc/meson.build
> > @@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
> > files('sifive_e_prci.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: 
> > files('sifive_e_aon.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: 
> > files('sifive_u_otp.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: 
> > files('sifive_u_prci.c'))
> > +specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: 
> > files('riscv_iopmp.c'))
> >
> >  subdir('macio')
> >
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > new file mode 100644
> > index 00..db43e3c73f
> > --- /dev/null
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -0,0 +1,1154 @@
> > +/*
> > + * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
> > + *
> > + * Copyright (c) 2023-2024 Andes Tech. Corp.
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms and conditions of the GNU General Public License,
> > + * version 2 or later, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope it will be useful, but WITHOUT
> > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License 
> > for
> > + * more details.
> > + *
> > + * You should have received a copy of the GNU General Public License along 
> > with
> > + * this program.  If not, see <http://www.gnu.org/licenses/>.
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/log.h"
> > +#include "qapi/error.h"
> > +#include "trace.h"
> > +#include "exec/exec-all.h"
> > +#include "exec/address-spaces.h"
> > +#include "hw/qdev-properties.h"
> > +#include "hw/sysbus.h"
> > +#include "hw/misc/riscv_iopmp.h"
> > +#include "memory.h"

Re: [PATCH v8 6/8] hw/misc/riscv_iopmp: Add API to configure RISCV CPU IOPMP support

[Ethan Chen via]

On Thu, Aug 08, 2024 at 02:25:04PM +1000, Alistair Francis wrote:
> 
> On Mon, Jul 15, 2024 at 8:15 PM Ethan Chen via  wrote:
> >
> > The iopmp_setup_cpu() function configures the RISCV CPU to support IOPMP and
> > specifies the CPU's RRID.
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  hw/misc/riscv_iopmp.c | 6 ++
> >  include/hw/misc/riscv_iopmp.h | 1 +
> >  2 files changed, 7 insertions(+)
> >
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > index e62ac57437..374bf5c610 100644
> > --- a/hw/misc/riscv_iopmp.c
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -1211,5 +1211,11 @@ void iopmp_setup_system_memory(DeviceState *dev, 
> > const MemMapEntry *memmap,
> > "iopmp-downstream-as");
> >  }
> >
> > +void iopmp_setup_cpu(RISCVCPU *cpu, uint32_t rrid)
> > +{
> > +cpu->cfg.iopmp = true;
> > +cpu->cfg.iopmp_rrid = rrid;
> > +}
> 
> This should just be a normal CPU property, which the machine can then
> set to true if required

I will add CPU properties for IOPMP config.

Thanks,
Ethan Chen

> 
> Alistair
> 
> > +
> >
> >  type_init(iopmp_register_types);
> > diff --git a/include/hw/misc/riscv_iopmp.h b/include/hw/misc/riscv_iopmp.h
> > index ebe9c4bc4a..7e7da56d10 100644
> > --- a/include/hw/misc/riscv_iopmp.h
> > +++ b/include/hw/misc/riscv_iopmp.h
> > @@ -167,5 +167,6 @@ typedef struct IopmpState {
> >
> >  void iopmp_setup_system_memory(DeviceState *dev, const MemMapEntry *memmap,
> > uint32_t mapentry_num);
> > +void iopmp_setup_cpu(RISCVCPU *cpu, uint32_t rrid);
> >
> >  #endif
> > --
> > 2.34.1
> >
> >

Re: [PATCH v8 4/8] hw/misc/riscv_iopmp: Add RISC-V IOPMP device

[Ethan Chen via]

On Thu, Aug 08, 2024 at 01:56:35PM +1000, Alistair Francis wrote:
> [EXTERNAL MAIL]
> 
> On Mon, Jul 15, 2024 at 7:58 PM Ethan Chen via  wrote:
> >
> > Support basic functions of IOPMP specification v0.9.1 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1
> >
> > The IOPMP checks whether memory access from a device or CPU is valid.
> > This implementation uses an IOMMU to modify the address space accessed
> > by the device.
> >
> > For device access with IOMMUAccessFlags specifying read or write
> > (IOMMU_RO or IOMMU_WO), the IOPMP checks the permission in
> > iopmp_translate. If the access is valid, the target address space is
> > downstream_as. If the access is blocked, it will be redirected to
> > blocked_rwx_as.
> >
> > For CPU access with IOMMUAccessFlags not specifying read or write
> > (IOMMU_NONE), the IOPMP translates the access to the corresponding
> > address space based on the permission. If the access has full permission
> > (rwx), the target address space is downstream_as. If the access has
> > limited permission, the target address space is blocked_ followed by
> > the lacked permissions.
> >
> > The operation of a blocked region can trigger an IOPMP interrupt, a bus
> > error, or it can respond with success and fabricated data, depending on
> > the value of the IOPMP ERR_CFG register.
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  hw/misc/Kconfig   |3 +
> >  hw/misc/meson.build   |1 +
> >  hw/misc/riscv_iopmp.c | 1154 +
> >  hw/misc/trace-events  |3 +
> >  include/hw/misc/riscv_iopmp.h |  168 +
> >  5 files changed, 1329 insertions(+)
> >  create mode 100644 hw/misc/riscv_iopmp.c
> >  create mode 100644 include/hw/misc/riscv_iopmp.h
> >
> > diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
> > index 1e08785b83..427f0c702e 100644
> > --- a/hw/misc/Kconfig
> > +++ b/hw/misc/Kconfig
> > @@ -213,4 +213,7 @@ config IOSB
> >  config XLNX_VERSAL_TRNG
> >  bool
> >
> > +config RISCV_IOPMP
> > +bool
> > +
> >  source macio/Kconfig
> > diff --git a/hw/misc/meson.build b/hw/misc/meson.build
> > index 2ca8717be2..d9006e1d81 100644
> > --- a/hw/misc/meson.build
> > +++ b/hw/misc/meson.build
> > @@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
> > files('sifive_e_prci.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: 
> > files('sifive_e_aon.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: 
> > files('sifive_u_otp.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: 
> > files('sifive_u_prci.c'))
> > +specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: 
> > files('riscv_iopmp.c'))
> >
> >  subdir('macio')
> >
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > new file mode 100644
> > index 00..db43e3c73f
> > --- /dev/null
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -0,0 +1,1154 @@
> > +/*
> > + * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
> > + *
> > + * Copyright (c) 2023-2024 Andes Tech. Corp.
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms and conditions of the GNU General Public License,
> > + * version 2 or later, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope it will be useful, but WITHOUT
> > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License 
> > for
> > + * more details.
> > + *
> > + * this program.  If not, see <http://www.gnu.org/licenses/>.
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/log.h"
> > +#include "qapi/error.h"
> > +#include "trace.h"
> > +#include "exec/exec-all.h"
> > +#include "exec/address-spaces.h"
> > +#include "hw/qdev-properties.h"
> > +#include "hw/sysbus.h"
> > +#include "hw/misc/riscv_iopmp.h"
> > +#include "memory.h"
> > +#include "hw/irq.h"
> > +#include "hw/registerfields.h"
> &g

[PATCH v8 8/8] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

- Add 'iopmp=on' option to enable IOPMP. It adds an iopmp device virt machine
  to protect all regions of system memory, and configures RRID of CPU.

Signed-off-by: Ethan Chen 
---
 docs/system/riscv/virt.rst |  5 +++
 hw/riscv/Kconfig   |  1 +
 hw/riscv/virt.c| 63 ++
 include/hw/riscv/virt.h|  5 ++-
 4 files changed, 73 insertions(+), 1 deletion(-)

diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
index 9a06f95a34..9fd006ccc2 100644
--- a/docs/system/riscv/virt.rst
+++ b/docs/system/riscv/virt.rst
@@ -116,6 +116,11 @@ The following machine-specific options are supported:
   having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not specified,
   the default number of per-HART VS-level AIA IMSIC pages is 0.
 
+- iopmp=[on|off]
+
+  When this option is "on", an IOPMP device is added to machine. IOPMP checks
+  memory transcations in system memory. This option is assumed to be "off".
+
 Running Linux kernel
 
 
diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index a2030e3a6f..0b45a5ade2 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -56,6 +56,7 @@ config RISCV_VIRT
 select PLATFORM_BUS
 select ACPI
 select ACPI_PCI
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index bc0893e087..5a03c03c4a 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -55,6 +55,7 @@
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
 #include "hw/virtio/virtio-iommu.h"
+#include "hw/misc/riscv_iopmp.h"
 
 /* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */
 static bool virt_use_kvm_aia(RISCVVirtState *s)
@@ -82,6 +83,7 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -90,6 +92,11 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_DRAM] = { 0x8000,   0x0 },
 };
 
+static const MemMapEntry iopmp_protect_memmap[] = {
+/* IOPMP protect all regions by default */
+{0, 0x},
+};
+
 /* PCIe high mmio is fixed for RV32 */
 #define VIRT32_HIGH_PCIE_MMIO_BASE  0x3ULL
 #define VIRT32_HIGH_PCIE_MMIO_SIZE  (4 * GiB)
@@ -1024,6 +1031,24 @@ static void create_fdt_virtio_iommu(RISCVVirtState *s, 
uint16_t bdf)
bdf + 1, iommu_phandle, bdf + 1, 0x - bdf);
 }
 
+static void create_fdt_iopmp(RISCVVirtState *s, const MemMapEntry *memmap,
+ uint32_t irq_mmio_phandle) {
+g_autofree char *name = NULL;
+MachineState *ms = MACHINE(s);
+
+name = g_strdup_printf("/soc/iopmp@%lx", (long)memmap[VIRT_IOPMP].base);
+qemu_fdt_add_subnode(ms->fdt, name);
+qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
+qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, memmap[VIRT_IOPMP].base,
+0x0, memmap[VIRT_IOPMP].size);
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", irq_mmio_phandle);
+if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", IOPMP_IRQ);
+} else {
+qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", IOPMP_IRQ, 0x4);
+}
+}
+
 static void finalize_fdt(RISCVVirtState *s)
 {
 uint32_t phandle = 1, irq_mmio_phandle = 1, msi_pcie_phandle = 1;
@@ -1042,6 +1067,10 @@ static void finalize_fdt(RISCVVirtState *s)
 create_fdt_uart(s, virt_memmap, irq_mmio_phandle);
 
 create_fdt_rtc(s, virt_memmap, irq_mmio_phandle);
+
+if (s->have_iopmp) {
+create_fdt_iopmp(s, virt_memmap, irq_mmio_phandle);
+}
 }
 
 static void create_fdt(RISCVVirtState *s, const MemMapEntry *memmap)
@@ -1425,6 +1454,7 @@ static void virt_machine_init(MachineState *machine)
 DeviceState *mmio_irqchip, *virtio_irqchip, *pcie_irqchip;
 int i, base_hartid, hart_count;
 int socket_count = riscv_socket_count(machine);
+int cpu, socket;
 
 /* Check socket count limit */
 if (VIRT_SOCKETS_MAX < socket_count) {
@@ -1606,6 +1636,19 @@ static void virt_machine_init(MachineState *machine)
 }
 virt_flash_map(s, system_memory);
 
+if (s->have_iopmp) {
+DeviceState *iopmp_dev = sysbus_create_simple(TYPE_IOPMP,
+memmap[VIRT_IOPMP].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
+
+for (socket = 0; socket < socket_count; socket++) {
+for (cpu = s->soc[socket].num_harts - 1; cpu >= 0; cpu--) {
+iopmp_setup_cpu(&s->soc[socket].harts[cpu], 0);
+}
+}
+iopmp_setup_system_memory(iopmp_dev, iopmp_p

[PATCH v8 7/8] hw/misc/riscv_iopmp: Add DMA operation with IOPMP support API

[Ethan Chen via]

The iopmp_dma_rw() function performs memory read/write operations to system
memory with support for IOPMP. It sends transaction information to the IOPMP
for partial hit detection.

Signed-off-by: Ethan Chen 
---
 hw/misc/riscv_iopmp.c | 68 +++
 include/hw/misc/riscv_iopmp.h |  3 +-
 2 files changed, 70 insertions(+), 1 deletion(-)

diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
index 374bf5c610..0be32ca819 100644
--- a/hw/misc/riscv_iopmp.c
+++ b/hw/misc/riscv_iopmp.c
@@ -1217,5 +1217,73 @@ void iopmp_setup_cpu(RISCVCPU *cpu, uint32_t rrid)
 cpu->cfg.iopmp_rrid = rrid;
 }
 
+static void send_transaction_start(IopmpState *s, uint32_t rrid)
+{
+int flag = 0;
+if (rrid < s->rrid_num) {
+while (flag == 0) {
+/* Wait last transaction of rrid complete */
+while (s->transaction_state[rrid].running) {
+;
+}
+qemu_mutex_lock(&s->iopmp_transaction_mutex);
+/* Check status again */
+if (s->transaction_state[rrid].running == false) {
+s->transaction_state[rrid].running = true;
+s->transaction_state[rrid].supported = true;
+flag = 1;
+}
+qemu_mutex_unlock(&s->iopmp_transaction_mutex);
+}
+}
+}
+
+static void send_transaction_complete(IopmpState *s, uint32_t rrid)
+{
+if (rrid < s->rrid_num) {
+qemu_mutex_lock(&s->iopmp_transaction_mutex);
+s->transaction_state[rrid].running = false;
+s->transaction_state[rrid].supported = false;
+qemu_mutex_unlock(&s->iopmp_transaction_mutex);
+}
+}
+
+static void send_transaction_info(IopmpState *s, uint32_t rrid,
+  hwaddr start_addr, hwaddr size)
+{
+if (rrid < s->rrid_num) {
+s->transaction_state[rrid].start_addr = start_addr;
+s->transaction_state[rrid].end_addr = start_addr + size - 1;
+}
+}
+
+/*
+ * Perform address_space_rw to system memory and send transaction information
+ * to correspond IOPMP for partially hit detection.
+ */
+MemTxResult iopmp_dma_rw(hwaddr addr, uint32_t rrid, void *buf, hwaddr len,
+ bool is_write)
+{
+MemTxResult result;
+MemTxAttrs attrs;
+iopmp_protection_memmap *map;
+/* Find which IOPMP is responsible for receiving transaction information */
+QLIST_FOREACH(map, &iopmp_protection_memmaps, list) {
+if (addr >= map->entry.base &&
+addr < map->entry.base + map->entry.size) {
+send_transaction_start(map->iopmp_s, rrid);
+send_transaction_info(map->iopmp_s, rrid, addr, len);
+break;
+}
+}
+
+attrs.requester_id = rrid;
+result = address_space_rw(&address_space_memory, addr, attrs, buf, len,
+  is_write);
+if (map) {
+send_transaction_complete(map->iopmp_s, rrid);
+}
+return result;
+}
 
 type_init(iopmp_register_types);
diff --git a/include/hw/misc/riscv_iopmp.h b/include/hw/misc/riscv_iopmp.h
index 7e7da56d10..d87395170d 100644
--- a/include/hw/misc/riscv_iopmp.h
+++ b/include/hw/misc/riscv_iopmp.h
@@ -168,5 +168,6 @@ typedef struct IopmpState {
 void iopmp_setup_system_memory(DeviceState *dev, const MemMapEntry *memmap,
uint32_t mapentry_num);
 void iopmp_setup_cpu(RISCVCPU *cpu, uint32_t rrid);
-
+MemTxResult iopmp_dma_rw(hwaddr addr, uint32_t rrid, void *buf, hwaddr len,
+ bool is_write);
 #endif
-- 
2.34.1

[PATCH v8 6/8] hw/misc/riscv_iopmp: Add API to configure RISCV CPU IOPMP support

[Ethan Chen via]

The iopmp_setup_cpu() function configures the RISCV CPU to support IOPMP and
specifies the CPU's RRID.

Signed-off-by: Ethan Chen 
---
 hw/misc/riscv_iopmp.c | 6 ++
 include/hw/misc/riscv_iopmp.h | 1 +
 2 files changed, 7 insertions(+)

diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
index e62ac57437..374bf5c610 100644
--- a/hw/misc/riscv_iopmp.c
+++ b/hw/misc/riscv_iopmp.c
@@ -1211,5 +1211,11 @@ void iopmp_setup_system_memory(DeviceState *dev, const 
MemMapEntry *memmap,
"iopmp-downstream-as");
 }
 
+void iopmp_setup_cpu(RISCVCPU *cpu, uint32_t rrid)
+{
+cpu->cfg.iopmp = true;
+cpu->cfg.iopmp_rrid = rrid;
+}
+
 
 type_init(iopmp_register_types);
diff --git a/include/hw/misc/riscv_iopmp.h b/include/hw/misc/riscv_iopmp.h
index ebe9c4bc4a..7e7da56d10 100644
--- a/include/hw/misc/riscv_iopmp.h
+++ b/include/hw/misc/riscv_iopmp.h
@@ -167,5 +167,6 @@ typedef struct IopmpState {
 
 void iopmp_setup_system_memory(DeviceState *dev, const MemMapEntry *memmap,
uint32_t mapentry_num);
+void iopmp_setup_cpu(RISCVCPU *cpu, uint32_t rrid);
 
 #endif
-- 
2.34.1

[PATCH v8 5/8] hw/misc/riscv_iopmp: Add API to set up IOPMP protection for system memory

[Ethan Chen via]

To enable system memory transactions through the IOPMP, memory regions must
be moved to the IOPMP downstream and then replaced with IOMMUs for IOPMP
translation.

The iopmp_setup_system_memory() function copies subregions of system memory
to create the IOPMP downstream and then replaces the specified memory
regions in system memory with the IOMMU regions of the IOPMP. It also
adds entries to a protection map that records the relationship between
physical address regions and the IOPMP, which is used by the IOPMP DMA
API to send transaction information.

Signed-off-by: Ethan Chen 
---
 hw/misc/riscv_iopmp.c | 61 +++
 include/hw/misc/riscv_iopmp.h |  3 ++
 2 files changed, 64 insertions(+)

diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
index db43e3c73f..e62ac57437 100644
--- a/hw/misc/riscv_iopmp.c
+++ b/hw/misc/riscv_iopmp.c
@@ -1151,4 +1151,65 @@ iopmp_register_types(void)
 type_register_static(&iopmp_iommu_memory_region_info);
 }
 
+/*
+ * Copies subregions from the source memory region to the destination memory
+ * region
+ */
+static void copy_memory_subregions(MemoryRegion *src_mr, MemoryRegion *dst_mr)
+{
+int32_t priority;
+hwaddr addr;
+MemoryRegion *alias, *subregion;
+QTAILQ_FOREACH(subregion, &src_mr->subregions, subregions_link) {
+priority = subregion->priority;
+addr = subregion->addr;
+alias = g_malloc0(sizeof(MemoryRegion));
+memory_region_init_alias(alias, NULL, subregion->name, subregion, 0,
+ memory_region_size(subregion));
+memory_region_add_subregion_overlap(dst_mr, addr, alias, priority);
+}
+}
+
+/*
+ * Create downstream of system memory for IOPMP, and overlap memory region
+ * specified in memmap with IOPMP translator. Make sure subregions are added to
+ * system memory before call this function. It also add entry to
+ * iopmp_protection_memmaps for recording the relationship between physical
+ * address regions and IOPMP.
+ */
+void iopmp_setup_system_memory(DeviceState *dev, const MemMapEntry *memmap,
+   uint32_t map_entry_num)
+{
+IopmpState *s = IOPMP(dev);
+uint32_t i;
+MemoryRegion *iommu_alias;
+MemoryRegion *target_mr = get_system_memory();
+MemoryRegion *downstream = g_malloc0(sizeof(MemoryRegion));
+memory_region_init(downstream, NULL, "iopmp_downstream",
+   memory_region_size(target_mr));
+/* Copy subregions of target to downstream */
+copy_memory_subregions(target_mr, downstream);
+
+iopmp_protection_memmap *map;
+for (i = 0; i < map_entry_num; i++) {
+/* Memory access to protected regions of target are through IOPMP */
+iommu_alias = g_new(MemoryRegion, 1);
+memory_region_init_alias(iommu_alias, NULL, "iommu_alias",
+ MEMORY_REGION(&s->iommu), memmap[i].base,
+ memmap[i].size);
+memory_region_add_subregion_overlap(target_mr, memmap[i].base,
+iommu_alias, 1);
+/* Record which IOPMP is responsible for the region */
+map = g_new0(iopmp_protection_memmap, 1);
+map->iopmp_s = s;
+map->entry.base = memmap[i].base;
+map->entry.size = memmap[i].size;
+QLIST_INSERT_HEAD(&iopmp_protection_memmaps, map, list);
+}
+s->downstream = downstream;
+address_space_init(&s->downstream_as, s->downstream,
+   "iopmp-downstream-as");
+}
+
+
 type_init(iopmp_register_types);
diff --git a/include/hw/misc/riscv_iopmp.h b/include/hw/misc/riscv_iopmp.h
index b8fe479108..ebe9c4bc4a 100644
--- a/include/hw/misc/riscv_iopmp.h
+++ b/include/hw/misc/riscv_iopmp.h
@@ -165,4 +165,7 @@ typedef struct IopmpState {
 uint32_t fabricated_v;
 } IopmpState;
 
+void iopmp_setup_system_memory(DeviceState *dev, const MemMapEntry *memmap,
+   uint32_t mapentry_num);
+
 #endif
-- 
2.34.1

[PATCH v8 1/8] memory: Introduce memory region fetch operation

[Ethan Chen via]

Allow memory regions to have different behaviors for read and fetch
operations.

For example, the RISC-V IOPMP could raise an interrupt when the CPU
tries to fetch from a non-executable region.

If the fetch operation for a memory region is not implemented, the read
operation will still be used for fetch operations.

Signed-off-by: Ethan Chen 
---
 accel/tcg/cputlb.c|   9 +++-
 include/exec/memory.h |  30 
 system/memory.c   | 104 ++
 system/trace-events   |   2 +
 4 files changed, 143 insertions(+), 2 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 117b516739..edb3715017 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1942,8 +1942,13 @@ static uint64_t int_ld_mmio_beN(CPUState *cpu, 
CPUTLBEntryFull *full,
 this_size = 1 << this_mop;
 this_mop |= MO_BE;
 
-r = memory_region_dispatch_read(mr, mr_offset, &val,
-this_mop, full->attrs);
+if (type == MMU_INST_FETCH) {
+r = memory_region_dispatch_fetch(mr, mr_offset, &val,
+ this_mop, full->attrs);
+} else {
+r = memory_region_dispatch_read(mr, mr_offset, &val,
+this_mop, full->attrs);
+}
 if (unlikely(r != MEMTX_OK)) {
 io_failed(cpu, full, addr, this_size, type, mmu_idx, r, ra);
 }
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 02f7528ec0..d837d7d7eb 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -274,6 +274,13 @@ struct MemoryRegionOps {
   uint64_t data,
   unsigned size);
 
+/* Fetch from the memory region. @addr is relative to @mr; @size is
+ * in bytes. */
+uint64_t (*fetch)(void *opaque,
+  hwaddr addr,
+  unsigned size);
+
+
 MemTxResult (*read_with_attrs)(void *opaque,
hwaddr addr,
uint64_t *data,
@@ -284,6 +291,12 @@ struct MemoryRegionOps {
 uint64_t data,
 unsigned size,
 MemTxAttrs attrs);
+MemTxResult (*fetch_with_attrs)(void *opaque,
+hwaddr addr,
+uint64_t *data,
+unsigned size,
+MemTxAttrs attrs);
+
 
 enum device_endian endianness;
 /* Guest-visible constraints: */
@@ -2602,6 +2615,23 @@ MemTxResult memory_region_dispatch_write(MemoryRegion 
*mr,
  MemOp op,
  MemTxAttrs attrs);
 
+
+/**
+ * memory_region_dispatch_fetch: perform a fetch directly to the specified
+ * MemoryRegion.
+ *
+ * @mr: #MemoryRegion to access
+ * @addr: address within that region
+ * @pval: pointer to uint64_t which the data is written to
+ * @op: size, sign, and endianness of the memory operation
+ * @attrs: memory transaction attributes to use for the access
+ */
+MemTxResult memory_region_dispatch_fetch(MemoryRegion *mr,
+ hwaddr addr,
+ uint64_t *pval,
+ MemOp op,
+ MemTxAttrs attrs);
+
 /**
  * address_space_init: initializes an address space
  *
diff --git a/system/memory.c b/system/memory.c
index 5e6eb459d5..b46721446c 100644
--- a/system/memory.c
+++ b/system/memory.c
@@ -477,6 +477,51 @@ static MemTxResult 
memory_region_read_with_attrs_accessor(MemoryRegion *mr,
 return r;
 }
 
+static MemTxResult memory_region_fetch_accessor(MemoryRegion *mr,
+hwaddr addr,
+uint64_t *value,
+unsigned size,
+signed shift,
+uint64_t mask,
+MemTxAttrs attrs)
+{
+uint64_t tmp;
+
+tmp = mr->ops->fetch(mr->opaque, addr, size);
+if (mr->subpage) {
+trace_memory_region_subpage_fetch(get_cpu_index(), mr, addr, tmp, 
size);
+} else if (trace_event_get_state_backends(TRACE_MEMORY_REGION_OPS_FETCH)) {
+hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
+trace_memory_region_ops_fetch(get_cpu_index(), mr, abs_addr, tmp, size,
+ memory_region_name(mr));
+}
+memory_region_shift_read_access(value, shift, mask, tmp);
+return MEMTX_OK;
+}
+
+static MemTxResult memory_region_fetch_with_attrs_accessor(MemoryRegion *mr,
+  hwaddr addr,
+

[PATCH v8 2/8] system/physmem: Support IOMMU granularity smaller than TARGET_PAGE size

[Ethan Chen via]

If the IOMMU granularity is smaller than the TARGET_PAGE size, there may be
 multiple entries within the same page. To obtain the correct result, pass
the original address to the IOMMU.

Similar to the RISC-V PMP solution, the TLB_INVALID_MASK will be set when
there are multiple entries in the same page, ensuring that the IOMMU is
checked on every access.

Signed-off-by: Ethan Chen 
---
 accel/tcg/cputlb.c | 20 
 system/physmem.c   |  4 
 2 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index edb3715017..7df106fea3 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1062,8 +1062,23 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx,
 
 prot = full->prot;
 asidx = cpu_asidx_from_attrs(cpu, full->attrs);
-section = address_space_translate_for_iotlb(cpu, asidx, paddr_page,
+section = address_space_translate_for_iotlb(cpu, asidx, full->phys_addr,
 &xlat, &sz, full->attrs, 
&prot);
+/* Update page size */
+full->lg_page_size = ctz64(sz);
+if (full->lg_page_size > TARGET_PAGE_BITS) {
+full->lg_page_size = TARGET_PAGE_BITS;
+} else {
+sz = TARGET_PAGE_SIZE;
+}
+
+is_ram = memory_region_is_ram(section->mr);
+is_romd = memory_region_is_romd(section->mr);
+/* If the translated mr is ram/rom, make xlat align the TARGET_PAGE */
+if (is_ram || is_romd) {
+xlat &= TARGET_PAGE_MASK;
+}
+
 assert(sz >= TARGET_PAGE_SIZE);
 
 tlb_debug("vaddr=%016" VADDR_PRIx " paddr=0x" HWADDR_FMT_plx
@@ -1076,9 +1091,6 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx,
 read_flags |= TLB_INVALID_MASK;
 }
 
-is_ram = memory_region_is_ram(section->mr);
-is_romd = memory_region_is_romd(section->mr);
-
 if (is_ram || is_romd) {
 /* RAM and ROMD both have associated host memory. */
 addend = (uintptr_t)memory_region_get_ram_ptr(section->mr) + xlat;
diff --git a/system/physmem.c b/system/physmem.c
index 2154432cb6..346b015447 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -702,6 +702,10 @@ address_space_translate_for_iotlb(CPUState *cpu, int 
asidx, hwaddr orig_addr,
 iotlb = imrc->translate(iommu_mr, addr, IOMMU_NONE, iommu_idx);
 addr = ((iotlb.translated_addr & ~iotlb.addr_mask)
 | (addr & iotlb.addr_mask));
+/* Update size */
+if (iotlb.addr_mask != -1 && *plen > iotlb.addr_mask + 1) {
+*plen = iotlb.addr_mask + 1;
+}
 /* Update the caller's prot bits to remove permissions the IOMMU
  * is giving us a failure response for. If we get down to no
  * permissions left at all we can give up now.
-- 
2.34.1

[PATCH v8 4/8] hw/misc/riscv_iopmp: Add RISC-V IOPMP device

[Ethan Chen via]

Support basic functions of IOPMP specification v0.9.1 rapid-k model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1

The IOPMP checks whether memory access from a device or CPU is valid.
This implementation uses an IOMMU to modify the address space accessed
by the device.

For device access with IOMMUAccessFlags specifying read or write
(IOMMU_RO or IOMMU_WO), the IOPMP checks the permission in
iopmp_translate. If the access is valid, the target address space is
downstream_as. If the access is blocked, it will be redirected to
blocked_rwx_as.

For CPU access with IOMMUAccessFlags not specifying read or write
(IOMMU_NONE), the IOPMP translates the access to the corresponding
address space based on the permission. If the access has full permission
(rwx), the target address space is downstream_as. If the access has
limited permission, the target address space is blocked_ followed by
the lacked permissions.

The operation of a blocked region can trigger an IOPMP interrupt, a bus
error, or it can respond with success and fabricated data, depending on
the value of the IOPMP ERR_CFG register.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |3 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1154 +
 hw/misc/trace-events  |3 +
 include/hw/misc/riscv_iopmp.h |  168 +
 5 files changed, 1329 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 1e08785b83..427f0c702e 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -213,4 +213,7 @@ config IOSB
 config XLNX_VERSAL_TRNG
 bool
 
+config RISCV_IOPMP
+bool
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index 2ca8717be2..d9006e1d81 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..db43e3c73f
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,1154 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023-2024 Andes Tech. Corp.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+#include "hw/registerfields.h"
+#include "trace.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+
+REG32(VERSION, 0x00)
+FIELD(VERSION, VENDOR, 0, 24)
+FIELD(VERSION, SPECVER , 24, 8)
+REG32(IMP, 0x04)
+FIELD(IMP, IMPID, 0, 32)
+REG32(HWCFG0, 0x08)
+FIELD(HWCFG0, MODEL, 0, 4)
+FIELD(HWCFG0, TOR_EN, 4, 1)
+FIELD(HWCFG0, SPS_EN, 5, 1)
+FIELD(HWCFG0, USER_CFG_EN, 6, 1)
+FIELD(HWCFG0, PRIENT_PROG, 7, 1)
+FIELD(HWCFG0, RRID_TRANSL_EN, 8, 1)
+FIELD(HWCFG0, RRID_TRANSL_PROG, 9, 1)
+FIELD(HWCFG0, CHK_X, 10, 1)
+FIELD(HWCFG0, NO_X, 11, 1)
+FIELD(HWCFG0, NO_W, 12, 1)
+FIELD(HWCFG0, STALL_EN, 13, 1)
+FIELD(HWCFG0, PEIS, 14, 1)
+FIELD(HWCFG0, PEES, 15, 1)
+FIELD(HWCFG0, MFR_EN, 16, 1)
+FIELD(HWCFG0, MD_NUM, 24, 7)
+FIELD(HWCFG0, ENABLE, 31, 1)
+REG32(HWCFG1, 0x0C)
+FIELD(HWCFG1, RRID_NUM, 0, 16)
+FIELD(HWCFG1, ENTRY_NUM, 16, 16)
+REG32(HWCFG2, 0x10)
+FIELD(HWCFG2, PRIO_ENTRY, 0, 16)
+FIELD(HWCFG2, RRID_TRANSL, 16, 16)
+REG32(ENTRYOFFSET, 0x14)
+FIELD(ENTRYOFFSET, OFFSET, 0, 32)
+REG32(MDSTALL, 0x30)
+FIELD(MDSTALL, EXEMPT, 0, 1)
+FIELD(MDSTALL, MD, 1, 31)
+REG32(MDSTALLH, 0x34)
+FIELD(MDSTALLH, MD, 0, 32)
+REG32(RRIDSCP, 0x38)
+FIELD(RRIDSCP, RRID, 0, 16)
+FIELD(RRIDSCP, OP, 30, 2)
+REG32(MDLCK, 0x40)
+FIELD(MDLCK, L, 0, 1)
+   

[PATCH v8 3/8] target/riscv: Add support for IOPMP

[Ethan Chen via]

Signed-off-by: Ethan Chen 
---
 target/riscv/cpu_cfg.h|  2 ++
 target/riscv/cpu_helper.c | 18 +++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/target/riscv/cpu_cfg.h b/target/riscv/cpu_cfg.h
index fb7eebde52..2946fec20c 100644
--- a/target/riscv/cpu_cfg.h
+++ b/target/riscv/cpu_cfg.h
@@ -164,6 +164,8 @@ struct RISCVCPUConfig {
 bool pmp;
 bool debug;
 bool misa_w;
+bool iopmp;
+uint32_t iopmp_rrid;
 
 bool short_isa_string;
 
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 6709622dd3..c2d6a874da 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -1418,9 +1418,21 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int 
size,
 }
 
 if (ret == TRANSLATE_SUCCESS) {
-tlb_set_page(cs, address & ~(tlb_size - 1), pa & ~(tlb_size - 1),
- prot, mmu_idx, tlb_size);
-return true;
+if (cpu->cfg.iopmp) {
+/*
+ * Do not align address on early stage because IOPMP needs origin
+ * address for permission check.
+ */
+tlb_set_page_with_attrs(cs, address, pa,
+(MemTxAttrs)
+{
+  .requester_id = cpu->cfg.iopmp_rrid,
+},
+prot, mmu_idx, tlb_size);
+} else {
+tlb_set_page(cs, address & ~(tlb_size - 1), pa & ~(tlb_size - 1),
+ prot, mmu_idx, tlb_size);
+}
 } else if (probe) {
 return false;
 } else {
-- 
2.34.1

[PATCH v8 0/8] Support RISC-V IOPMP

[Ethan Chen via]

This series implements basic functions of IOPMP specification v0.9.1 rapid-k
model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1

When IOPMP is enabled, memory access to system memory from devices and
the CPU will be checked by the IOPMP.

The issue of CPU access to non-CPU address space via IOMMU was previously
mentioned by Jim Shu, who provided a patch[1] to fix it. IOPMP also requires
this patch.

[1] accel/tcg: Store section pointer in CPUTLBEntryFull

https://patchew.org/QEMU/20240612081416.29704-1-jim@sifive.com/20240612081416.29704-2-jim@sifive.com/


Changes for v8:

  - Support transactions from CPU
  - Add an API to set up IOPMP protection for system memory
  - Add an API to configure the RISCV CPU to support IOPMP and specify the
CPU's RRID
  - Add an API for DMA operation with IOPMP support
  - Add SPDX license identifiers to new files (Stefan W.)
  - Remove IOPMP PCI interface(pci_setup_iommu) (Zhiwei)

Changes for v7:

  - Change the specification version to v0.9.1
  - Remove the sps extension
  - Remove stall support, transaction information which need requestor device
support.
  - Remove iopmp_cascade option for virt machine
  - Refine 'addr' range checks switch case (Daniel)

Ethan Chen (8):
  memory: Introduce memory region fetch operation
  system/physmem: Support IOMMU granularity smaller than TARGET_PAGE
size
  target/riscv: Add support for IOPMP
  hw/misc/riscv_iopmp: Add RISC-V IOPMP device
  hw/misc/riscv_iopmp: Add API to set up IOPMP protection for system
memory
  hw/misc/riscv_iopmp: Add API to configure RISCV CPU IOPMP support
  hw/misc/riscv_iopmp:  Add DMA operation with IOPMP support API
  hw/riscv/virt: Add IOPMP support

 accel/tcg/cputlb.c|   29 +-
 docs/system/riscv/virt.rst|5 +
 hw/misc/Kconfig   |3 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1289 +
 hw/misc/trace-events  |3 +
 hw/riscv/Kconfig  |1 +
 hw/riscv/virt.c   |   63 ++
 include/exec/memory.h |   30 +
 include/hw/misc/riscv_iopmp.h |  173 +
 include/hw/riscv/virt.h   |5 +-
 system/memory.c   |  104 +++
 system/physmem.c  |4 +
 system/trace-events   |2 +
 target/riscv/cpu_cfg.h|2 +
 target/riscv/cpu_helper.c |   18 +-
 16 files changed, 1722 insertions(+), 10 deletions(-)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h

-- 
2.34.1

Re: [PATCH v7 2/2] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

On Wed, Jun 26, 2024 at 11:22:46AM +1000, Alistair Francis wrote:
> 
> On Mon, Jun 24, 2024 at 11:47 AM Ethan Chen  wrote:
> >
> > Hi Alistair,
> >
> > IOPMP can applies all device. In this patch series, PCI devices on the 
> > bridge
> > can connect to IOPMP by pci_setup_iommu(), but other devices need change 
> > their
> > memory access address space from system memory to IOPMP by themself.
> 
> We should be really clear about that then. The documentation and the
> flag `iopmp=[on|off]` implies that either the IOPMP is on or off.
> 
> For example, what happens in the future if we extend support to apply
> to all devices? That will be a breaking change for anyone currently
> using `iopmp=on`.
> 
> Maybe we should have use something like `iopmp=[pci|off]` instead, and
> then be really clear in the docs what is and isn't going through the
> IOPMP.
> 
> Alistair

Hi Alistair,

According to Zhiwei's suggestion in this patch series, we will remove
iopmp_setup_pci because it will be exclusive with IOMMU integration.

We are looking for an interface to make device memory access to be
checked by IOPMP. After iopmp_setup_pci is removed, all devices need
to change memory access target to iommu memory region in IOPMP
themselves in current method. Therefore, by default, all devices won't
go through the IOPMP even if iopmp=on.

Another method is to replace the memory region of protected device
in the system memory by iommu memory region in IOPMP (similar to
MPC in arm/mps2-tz) when iopmp=on. With this method, all devices are
going through the IOPMP by default when iopmp=on.

Which method is more suitable for the RISC-V virt machine?

Thanks,
Ethan

> 
> >
> > Thanks,
> > Ethan
> >
> > On Fri, Jun 21, 2024 at 03:54:15PM +1000, Alistair Francis wrote:
> > > On Wed, Jun 12, 2024 at 1:25 PM Ethan Chen via  
> > > wrote:
> > > >
> > > > If a requestor device is connected to the IOPMP device, its memory 
> > > > access will
> > > > be checked by the IOPMP rule.
> > > >
> > > > - Add 'iopmp=on' option to add an iopmp device and make the Generic PCI 
> > > > Express
> > > >   Bridge connect to IOPMP.
> > >
> > > I have only had a chance to have a quick look at this series and the spec.
> > >
> > > But the IOPMP spec applies to all devices right, but this series seems
> > > to only work with PCI. Am I missing something?
> > >
> > > Alistair
> > >
> > > >
> > > > Signed-off-by: Ethan Chen 
> > > > ---
> > > >  docs/system/riscv/virt.rst |  6 
> > > >  hw/riscv/Kconfig   |  1 +
> > > >  hw/riscv/virt.c| 57 --
> > > >  include/hw/riscv/virt.h|  5 +++-
> > > >  4 files changed, 66 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
> > > > index 9a06f95a34..3b2576f905 100644
> > > > --- a/docs/system/riscv/virt.rst
> > > > +++ b/docs/system/riscv/virt.rst
> > > > @@ -116,6 +116,12 @@ The following machine-specific options are 
> > > > supported:
> > > >having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not 
> > > > specified,
> > > >the default number of per-HART VS-level AIA IMSIC pages is 0.
> > > >
> > > > +- iopmp=[on|off]
> > > > +
> > > > +  When this option is "on", an IOPMP device is added to machine. It 
> > > > checks dma
> > > > +  operations from the generic PCIe host bridge. This option is assumed 
> > > > to be
> > > > +  "off".
> > > > +
> > > >  Running Linux kernel
> > > >  
> > > >
> > > > diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
> > > > index a2030e3a6f..0b45a5ade2 100644
> > > > --- a/hw/riscv/Kconfig
> > > > +++ b/hw/riscv/Kconfig
> > > > @@ -56,6 +56,7 @@ config RISCV_VIRT
> > > >  select PLATFORM_BUS
> > > >  select ACPI
> > > >  select ACPI_PCI
> > > > +select RISCV_IOPMP
> > > >
> > > >  config SHAKTI_C
> > > >  bool
> > > > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > > > index 4fdb660525..53a1b71c71 100644
> > > > --- a/hw/riscv/virt.c
> > > > +++ b/hw/riscv/virt.c
> > > > @@ -55

Re: [PATCH v7 2/2] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

Hi Alistair,

IOPMP can applies all device. In this patch series, PCI devices on the bridge 
can connect to IOPMP by pci_setup_iommu(), but other devices need change their
memory access address space from system memory to IOPMP by themself.

Thanks,
Ethan

On Fri, Jun 21, 2024 at 03:54:15PM +1000, Alistair Francis wrote:
> On Wed, Jun 12, 2024 at 1:25 PM Ethan Chen via  wrote:
> >
> > If a requestor device is connected to the IOPMP device, its memory access 
> > will
> > be checked by the IOPMP rule.
> >
> > - Add 'iopmp=on' option to add an iopmp device and make the Generic PCI 
> > Express
> >   Bridge connect to IOPMP.
> 
> I have only had a chance to have a quick look at this series and the spec.
> 
> But the IOPMP spec applies to all devices right, but this series seems
> to only work with PCI. Am I missing something?
> 
> Alistair
> 
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  docs/system/riscv/virt.rst |  6 
> >  hw/riscv/Kconfig   |  1 +
> >  hw/riscv/virt.c| 57 --
> >  include/hw/riscv/virt.h|  5 +++-
> >  4 files changed, 66 insertions(+), 3 deletions(-)
> >
> > diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
> > index 9a06f95a34..3b2576f905 100644
> > --- a/docs/system/riscv/virt.rst
> > +++ b/docs/system/riscv/virt.rst
> > @@ -116,6 +116,12 @@ The following machine-specific options are supported:
> >having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not specified,
> >the default number of per-HART VS-level AIA IMSIC pages is 0.
> >
> > +- iopmp=[on|off]
> > +
> > +  When this option is "on", an IOPMP device is added to machine. It checks 
> > dma
> > +  operations from the generic PCIe host bridge. This option is assumed to 
> > be
> > +  "off".
> > +
> >  Running Linux kernel
> >  
> >
> > diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
> > index a2030e3a6f..0b45a5ade2 100644
> > --- a/hw/riscv/Kconfig
> > +++ b/hw/riscv/Kconfig
> > @@ -56,6 +56,7 @@ config RISCV_VIRT
> >  select PLATFORM_BUS
> >  select ACPI
> >  select ACPI_PCI
> > +select RISCV_IOPMP
> >
> >  config SHAKTI_C
> >  bool
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index 4fdb660525..53a1b71c71 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -55,6 +55,7 @@
> >  #include "hw/acpi/aml-build.h"
> >  #include "qapi/qapi-visit-common.h"
> >  #include "hw/virtio/virtio-iommu.h"
> > +#include "hw/misc/riscv_iopmp.h"
> >
> >  /* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by 
> > QEMU. */
> >  static bool virt_use_kvm_aia(RISCVVirtState *s)
> > @@ -82,6 +83,7 @@ static const MemMapEntry virt_memmap[] = {
> >  [VIRT_UART0] ={ 0x1000, 0x100 },
> >  [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
> >  [VIRT_FW_CFG] =   { 0x1010,  0x18 },
> > +[VIRT_IOPMP] ={ 0x1020,  0x10 },
> >  [VIRT_FLASH] ={ 0x2000, 0x400 },
> >  [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
> >  [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
> > @@ -1006,6 +1008,24 @@ static void create_fdt_virtio_iommu(RISCVVirtState 
> > *s, uint16_t bdf)
> > bdf + 1, iommu_phandle, bdf + 1, 0x - bdf);
> >  }
> >
> > +static void create_fdt_iopmp(RISCVVirtState *s, const MemMapEntry *memmap,
> > + uint32_t irq_mmio_phandle) {
> > +g_autofree char *name = NULL;
> > +MachineState *ms = MACHINE(s);
> > +
> > +name = g_strdup_printf("/soc/iopmp@%lx", 
> > (long)memmap[VIRT_IOPMP].base);
> > +qemu_fdt_add_subnode(ms->fdt, name);
> > +qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
> > +qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, 
> > memmap[VIRT_IOPMP].base,
> > +0x0, memmap[VIRT_IOPMP].size);
> > +qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", 
> > irq_mmio_phandle);
> > +if (s->aia_type == VIRT_AIA_TYPE_NONE) {
> > +qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", IOPMP_IRQ);
> > +} else {
> > +qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", IOPMP_IRQ, 

Re: [PATCH v7 1/2] hw/misc/riscv_iopmp: Add RISC-V IOPMP device

[Ethan Chen via]

On Mon, Jun 17, 2024 at 02:09:34PM +0200, Stefan Weil wrote:
> [EXTERNAL MAIL]
> 
> Am 12.06.24 um 05:17 schrieb Ethan Chen via:
> > Support basic functions of IOPMP specification v0.9.1 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1
> > 
> > IOPMP check memory access from device is valid or not. This implementation 
> > uses
> > IOMMU to change address space that device access. There are three possible
> > results of an access: valid, blocked, and stalled(stall is not supported in 
> > this
> >   patch).
> > 
> > If an access is valid, target address space is downstream_as.
> > If an access is blocked, it will go to blocked_io_as. The operation of
> > blocked_io_as could be a bus error, or it can respond a success with 
> > fabricated
> > data depending on IOPMP ERR_CFG register value.
> > 
> > Signed-off-by: Ethan Chen 
> > ---
> >   hw/misc/Kconfig   |3 +
> >   hw/misc/meson.build   |1 +
> >   hw/misc/riscv_iopmp.c | 1002 +
> >   hw/misc/trace-events  |4 +
> >   include/hw/misc/riscv_iopmp.h |  152 +
> >   5 files changed, 1162 insertions(+)
> >   create mode 100644 hw/misc/riscv_iopmp.c
> >   create mode 100644 include/hw/misc/riscv_iopmp.h
> 
> Should both new files have SPDX license identifiers?
> 
> Regards,
> Stefan W.

Thank you for the reminder, I will add them.

Thanks,
Ethan

Re: [PATCH v7 1/2] hw/misc/riscv_iopmp: Add RISC-V IOPMP device

[Ethan Chen via]

On Mon, Jun 17, 2024 at 07:28:33PM +0800, LIU Zhiwei wrote:
> 
> On 2024/6/12 11:17, Ethan Chen wrote:
> > Support basic functions of IOPMP specification v0.9.1 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1
> > 
> > IOPMP check memory access from device is valid or not. This implementation 
> > uses
> > IOMMU to change address space that device access. There are three possible
> > results of an access: valid, blocked, and stalled(stall is not supported in 
> > this
> >   patch).
> > 
> > If an access is valid, target address space is downstream_as.
> > If an access is blocked, it will go to blocked_io_as. The operation of
> > blocked_io_as could be a bus error, or it can respond a success with 
> > fabricated
> > data depending on IOPMP ERR_CFG register value.
> > 
> > Signed-off-by: Ethan Chen 
> > ---
> >   hw/misc/Kconfig   |3 +
> >   hw/misc/meson.build   |1 +
> >   hw/misc/riscv_iopmp.c | 1002 +
> >   hw/misc/trace-events  |4 +
> >   include/hw/misc/riscv_iopmp.h |  152 +
> >   5 files changed, 1162 insertions(+)
> >   create mode 100644 hw/misc/riscv_iopmp.c
> >   create mode 100644 include/hw/misc/riscv_iopmp.h
> > 
> > diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
> > index 1e08785b83..427f0c702e 100644
> > --- a/hw/misc/Kconfig
> > +++ b/hw/misc/Kconfig
> > @@ -213,4 +213,7 @@ config IOSB
> >   config XLNX_VERSAL_TRNG
> >   bool
> > 
> > +config RISCV_IOPMP
> > +bool
> > +
> >   source macio/Kconfig
> > diff --git a/hw/misc/meson.build b/hw/misc/meson.build
> > index 86596a3888..f83cd108f8 100644
> > --- a/hw/misc/meson.build
> > +++ b/hw/misc/meson.build
> > @@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
> > files('sifive_e_prci.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: 
> > files('sifive_e_aon.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: 
> > files('sifive_u_otp.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: 
> > files('sifive_u_prci.c'))
> > +specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: 
> > files('riscv_iopmp.c'))
> > 
> >   subdir('macio')
> > 
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > new file mode 100644
> > index 00..75b28dc559
> > --- /dev/null
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -0,0 +1,1002 @@
> > +/*
> > + * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
> > + *
> > + * Copyright (c) 2023 Andes Tech. Corp.
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms and conditions of the GNU General Public License,
> > + * version 2 or later, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope it will be useful, but WITHOUT
> > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License 
> > for
> > + * more details.
> > + *
> > + * You should have received a copy of the GNU General Public License along 
> > with
> > + * this program.  If not, see .
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/log.h"
> > +#include "qapi/error.h"
> > +#include "trace.h"
> > +#include "exec/exec-all.h"
> > +#include "exec/address-spaces.h"
> > +#include "hw/qdev-properties.h"
> > +#include "hw/sysbus.h"
> > +#include "hw/misc/riscv_iopmp.h"
> > +#include "memory.h"
> > +#include "hw/irq.h"
> > +#include "hw/registerfields.h"
> > +#include "trace.h"
> > +
> > +#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
> > +
> > +REG32(VERSION, 0x00)
> > +FIELD(VERSION, VENDOR, 0, 24)
> > +FIELD(VERSION, SPECVER , 24, 8)
> > +REG32(IMP, 0x04)
> > +FIELD(IMP, IMPID, 0, 32)
> > +REG32(HWCFG0, 0x08)
> > +FIELD(HWCFG0, MODEL, 0, 4)
> > +FIELD(HWCFG0, TOR_EN, 4, 1)
> > +FIELD(HWCFG0, SPS_EN, 5, 1)
> > +FIELD(HWCFG0, USER_CFG_EN, 6, 1)
> > +FIELD(HWCFG0, PRIENT_PROG, 7, 1)
> > +FIELD(HWCFG0, RRID_TRANSL_EN, 8, 1)
> > +FIELD(HWCFG0, RRID_TRANSL_PROG, 9, 1)
> > +FIELD(HWCFG0, CHK_X, 10, 1)
> > +FIELD(HWCFG0, NO_X, 11, 1)
> > +FIELD(HWCFG0, NO_W, 12, 1)
> > +FIELD(HWCFG0, STALL_EN, 13, 1)
> > +FIELD(HWCFG0, PEIS, 14, 1)
> > +FIELD(HWCFG0, PEES, 15, 1)
> > +FIELD(HWCFG0, MFR_EN, 16, 1)
> > +FIELD(HWCFG0, MD_NUM, 24, 7)
> > +FIELD(HWCFG0, ENABLE, 31, 1)
> > +REG32(HWCFG1, 0x0C)
> > +FIELD(HWCFG1, RRID_NUM, 0, 16)
> > +FIELD(HWCFG1, ENTRY_NUM, 16, 16)
> > +REG32(HWCFG2, 0x10)
> > +FIELD(HWCFG2, PRIO_ENTRY, 0, 16)
> > +FIELD(HWCFG2, RRID_TRANSL, 16, 16)
> > +REG32(ENTRYOFFSET, 0x14)
> > +FIELD(ENTRYOFFSET, OFFSET, 0, 32)
> > +REG32(MDSTALL, 0x30)
> > +FIELD(MDSTALL, EXEMPT, 0, 1)
> > +FIELD(MDSTALL, MD, 1, 31)
> > +REG32(MDSTALLH, 0x34)
> > +FIELD(MDSTALLH, MD, 0, 32)
> > +REG32

Re: [PATCH v7 1/2] hw/misc/riscv_iopmp: Add RISC-V IOPMP device

[Ethan Chen via]

On Thu, Jun 13, 2024 at 05:26:03PM +0800, LIU Zhiwei wrote:
> 
> Hi Ethan,
> 
> On 2024/6/12 11:17, Ethan Chen wrote:
> > Support basic functions of IOPMP specification v0.9.1 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1
> > 
> > IOPMP check memory access from device is valid or not. This implementation 
> > uses
> > IOMMU to change address space that device access. There are three possible
> > results of an access: valid, blocked, and stalled(stall is not supported in 
> > this
> >   patch).
> > 
> > If an access is valid, target address space is downstream_as.
> > If an access is blocked, it will go to blocked_io_as. The operation of
> > blocked_io_as could be a bus error, or it can respond a success with 
> > fabricated
> > data depending on IOPMP ERR_CFG register value.
> > 
> > Signed-off-by: Ethan Chen 
> > ---
> >   hw/misc/Kconfig   |3 +
> >   hw/misc/meson.build   |1 +
> >   hw/misc/riscv_iopmp.c | 1002 +
> >   hw/misc/trace-events  |4 +
> >   include/hw/misc/riscv_iopmp.h |  152 +
> >   5 files changed, 1162 insertions(+)
> >   create mode 100644 hw/misc/riscv_iopmp.c
> >   create mode 100644 include/hw/misc/riscv_iopmp.h
> > 
> > diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
> > index 1e08785b83..427f0c702e 100644
> > --- a/hw/misc/Kconfig
> > +++ b/hw/misc/Kconfig
> > @@ -213,4 +213,7 @@ config IOSB
> >   config XLNX_VERSAL_TRNG
> >   bool
> > 
> > +config RISCV_IOPMP
> > +bool
> > +
> >   source macio/Kconfig
> > diff --git a/hw/misc/meson.build b/hw/misc/meson.build
> > index 86596a3888..f83cd108f8 100644
> > --- a/hw/misc/meson.build
> > +++ b/hw/misc/meson.build
> > @@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
> > files('sifive_e_prci.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: 
> > files('sifive_e_aon.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: 
> > files('sifive_u_otp.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: 
> > files('sifive_u_prci.c'))
> > +specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: 
> > files('riscv_iopmp.c'))
> > 
> >   subdir('macio')
> > 
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > new file mode 100644
> > index 00..75b28dc559
> > --- /dev/null
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -0,0 +1,1002 @@
> > +/*
> > + * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
> > + *
> > + * Copyright (c) 2023 Andes Tech. Corp.
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms and conditions of the GNU General Public License,
> > + * version 2 or later, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope it will be useful, but WITHOUT
> > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License 
> > for
> > + * more details.
> > + *
> > + * You should have received a copy of the GNU General Public License along 
> > with
> > + * this program.  If not, see .
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/log.h"
> > +#include "qapi/error.h"
> > +#include "trace.h"
> > +#include "exec/exec-all.h"
> > +#include "exec/address-spaces.h"
> > +#include "hw/qdev-properties.h"
> > +#include "hw/sysbus.h"
> > +#include "hw/misc/riscv_iopmp.h"
> > +#include "memory.h"
> > +#include "hw/irq.h"
> > +#include "hw/registerfields.h"
> > +#include "trace.h"
> > +
> > +#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
> > +
> > +REG32(VERSION, 0x00)
> > +FIELD(VERSION, VENDOR, 0, 24)
> > +FIELD(VERSION, SPECVER , 24, 8)
> > +REG32(IMP, 0x04)
> > +FIELD(IMP, IMPID, 0, 32)
> > +REG32(HWCFG0, 0x08)
> > +FIELD(HWCFG0, MODEL, 0, 4)
> > +FIELD(HWCFG0, TOR_EN, 4, 1)
> > +FIELD(HWCFG0, SPS_EN, 5, 1)
> > +FIELD(HWCFG0, USER_CFG_EN, 6, 1)
> > +FIELD(HWCFG0, PRIENT_PROG, 7, 1)
> > +FIELD(HWCFG0, RRID_TRANSL_EN, 8, 1)
> > +FIELD(HWCFG0, RRID_TRANSL_PROG, 9, 1)
> > +FIELD(HWCFG0, CHK_X, 10, 1)
> > +FIELD(HWCFG0, NO_X, 11, 1)
> > +FIELD(HWCFG0, NO_W, 12, 1)
> > +FIELD(HWCFG0, STALL_EN, 13, 1)
> > +FIELD(HWCFG0, PEIS, 14, 1)
> > +FIELD(HWCFG0, PEES, 15, 1)
> > +FIELD(HWCFG0, MFR_EN, 16, 1)
> > +FIELD(HWCFG0, MD_NUM, 24, 7)
> > +FIELD(HWCFG0, ENABLE, 31, 1)
> > +REG32(HWCFG1, 0x0C)
> > +FIELD(HWCFG1, RRID_NUM, 0, 16)
> > +FIELD(HWCFG1, ENTRY_NUM, 16, 16)
> > +REG32(HWCFG2, 0x10)
> > +FIELD(HWCFG2, PRIO_ENTRY, 0, 16)
> > +FIELD(HWCFG2, RRID_TRANSL, 16, 16)
> > +REG32(ENTRYOFFSET, 0x14)
> > +FIELD(ENTRYOFFSET, OFFSET, 0, 32)
> > +REG32(MDSTALL, 0x30)
> > +FIELD(MDSTALL, EXEMPT, 0, 1)
> > +FIELD(MDSTALL, MD, 1, 31)
> > +REG32(MDSTALLH, 0x34)
> > +FIELD(MDSTALLH, MD, 0,

Re: [RFC PATCH 02/16] accel/tcg: memory access from CPU will pass access_type to IOMMU

[Ethan Chen via]

On Wed, Jun 12, 2024 at 04:14:02PM +0800, Jim Shu wrote:
> [EXTERNAL MAIL]
> 
> It is the preparation patch for upcoming RISC-V wgChecker device.
> 
> Since RISC-V wgChecker could permit access in RO/WO permission, the
> IOMMUMemoryRegion could return different section for read & write
> access. The memory access from CPU should also pass the access_type to
> IOMMU translate function so that IOMMU could return the correct section
> of specified access_type.
> 

Hi Jim,

Does this method take into account the situation where the CPU access type is
different from the access type when creating iotlb? I think the section
might be wrong in this situation.

Thanks,
Ethan
> 
> 

Re: [RFC PATCH] memory: Introduce memory region fetch operation

[Ethan Chen via]

On Wed, Jun 12, 2024 at 01:43:41PM +0100, Peter Maydell wrote:
> 
> On Wed, 12 Jun 2024 at 10:02, Ethan Chen via  wrote:
> >
> > Allow the memory region to have different behaviors for read and fetch
> > operations.
> >
> > For example RISCV IOPMP will raise interrupt when cpu try to fetch a
> > non-excutable region.
> 
> It actually raises an interrupt rather than it being a permissions fault?

Device can return bus error, interrupt or success with fabricated data.

> 
> > If fetch operation of a memory region is not implemented, it still uses the
> > read operation for fetch.
> 
> This patch should probably be part of the series with the device that
> needs it.

I will add this patch to IOPMP patch series.

Thanks,
Ethan

[RFC PATCH] system/physmem: Support IOMMU granularity smaller than TARGET_PAGE size

[Ethan Chen via]

If the IOMMU granularity is smaller than TARGET_PAGE size, there may have
multiple entries in the same page. Pass the origin address to IOMMU to get
correct result

Similar to the RISCV PMP solution, TLB_INVALID_MASK will be set when there have
multiple entries in the same page to check the IOMMU on every access.

Signed-off-by: Ethan Chen 
---
 accel/tcg/cputlb.c | 17 -
 system/physmem.c   |  4 
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 117b516739..9c0db4d9e2 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1062,8 +1062,23 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx,
 
 prot = full->prot;
 asidx = cpu_asidx_from_attrs(cpu, full->attrs);
-section = address_space_translate_for_iotlb(cpu, asidx, paddr_page,
+section = address_space_translate_for_iotlb(cpu, asidx, full->phys_addr,
 &xlat, &sz, full->attrs, 
&prot);
+/* Update page size */
+full->lg_page_size = ctz64(sz);
+if (full->lg_page_size > TARGET_PAGE_BITS) {
+full->lg_page_size = TARGET_PAGE_BITS;
+} else {
+sz = TARGET_PAGE_SIZE;
+}
+
+is_ram = memory_region_is_ram(section->mr);
+is_romd = memory_region_is_romd(section->mr);
+/* If the translated mr is ram/rom, make xlat align the TARGET_PAGE */
+if (is_ram || is_romd) {
+xlat &= TARGET_PAGE_MASK;
+}
+
 assert(sz >= TARGET_PAGE_SIZE);
 
 tlb_debug("vaddr=%016" VADDR_PRIx " paddr=0x" HWADDR_FMT_plx
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..0fd0326714 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -702,6 +702,10 @@ address_space_translate_for_iotlb(CPUState *cpu, int 
asidx, hwaddr orig_addr,
 iotlb = imrc->translate(iommu_mr, addr, IOMMU_NONE, iommu_idx);
 addr = ((iotlb.translated_addr & ~iotlb.addr_mask)
 | (addr & iotlb.addr_mask));
+/* Update size */
+if (iotlb.addr_mask != -1 && *plen > iotlb.addr_mask + 1) {
+*plen = iotlb.addr_mask + 1;
+}
 /* Update the caller's prot bits to remove permissions the IOMMU
  * is giving us a failure response for. If we get down to no
  * permissions left at all we can give up now.
-- 
2.34.1

[RFC PATCH] memory: Introduce memory region fetch operation

[Ethan Chen via]

Allow the memory region to have different behaviors for read and fetch
operations.

For example RISCV IOPMP will raise interrupt when cpu try to fetch a
non-excutable region.

If fetch operation of a memory region is not implemented, it still uses the
read operation for fetch.

Signed-off-by: Ethan Chen 
---
 accel/tcg/cputlb.c|   9 +++-
 include/exec/memory.h |  30 
 system/memory.c   | 104 ++
 3 files changed, 141 insertions(+), 2 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 117b516739..edb3715017 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1942,8 +1942,13 @@ static uint64_t int_ld_mmio_beN(CPUState *cpu, 
CPUTLBEntryFull *full,
 this_size = 1 << this_mop;
 this_mop |= MO_BE;
 
-r = memory_region_dispatch_read(mr, mr_offset, &val,
-this_mop, full->attrs);
+if (type == MMU_INST_FETCH) {
+r = memory_region_dispatch_fetch(mr, mr_offset, &val,
+ this_mop, full->attrs);
+} else {
+r = memory_region_dispatch_read(mr, mr_offset, &val,
+this_mop, full->attrs);
+}
 if (unlikely(r != MEMTX_OK)) {
 io_failed(cpu, full, addr, this_size, type, mmu_idx, r, ra);
 }
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 1be58f694c..6d61ec443e 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -274,6 +274,13 @@ struct MemoryRegionOps {
   uint64_t data,
   unsigned size);
 
+/* Fetch to the memory region. @addr is relative to @mr; @size is
+ * in bytes. */
+uint64_t (*fetch)(void *opaque,
+  hwaddr addr,
+  unsigned size);
+
+
 MemTxResult (*read_with_attrs)(void *opaque,
hwaddr addr,
uint64_t *data,
@@ -284,6 +291,12 @@ struct MemoryRegionOps {
 uint64_t data,
 unsigned size,
 MemTxAttrs attrs);
+MemTxResult (*fetch_with_attrs)(void *opaque,
+hwaddr addr,
+uint64_t *data,
+unsigned size,
+MemTxAttrs attrs);
+
 
 enum device_endian endianness;
 /* Guest-visible constraints: */
@@ -2672,6 +2685,23 @@ MemTxResult memory_region_dispatch_write(MemoryRegion 
*mr,
  MemOp op,
  MemTxAttrs attrs);
 
+
+/**
+ * memory_region_dispatch_fetch: perform a fetch directly to the specified
+ * MemoryRegion.
+ *
+ * @mr: #MemoryRegion to access
+ * @addr: address within that region
+ * @pval: pointer to uint64_t which the data is written to
+ * @op: size, sign, and endianness of the memory operation
+ * @attrs: memory transaction attributes to use for the access
+ */
+MemTxResult memory_region_dispatch_fetch(MemoryRegion *mr,
+ hwaddr addr,
+ uint64_t *pval,
+ MemOp op,
+ MemTxAttrs attrs);
+
 /**
  * address_space_init: initializes an address space
  *
diff --git a/system/memory.c b/system/memory.c
index 74cd73ebc7..ecd3020b1c 100644
--- a/system/memory.c
+++ b/system/memory.c
@@ -477,6 +477,51 @@ static MemTxResult 
memory_region_read_with_attrs_accessor(MemoryRegion *mr,
 return r;
 }
 
+static MemTxResult memory_region_fetch_accessor(MemoryRegion *mr,
+hwaddr addr,
+uint64_t *value,
+unsigned size,
+signed shift,
+uint64_t mask,
+MemTxAttrs attrs)
+{
+uint64_t tmp;
+
+tmp = mr->ops->fetch(mr->opaque, addr, size);
+if (mr->subpage) {
+trace_memory_region_subpage_fetch(get_cpu_index(), mr, addr, tmp, 
size);
+} else if (trace_event_get_state_backends(TRACE_MEMORY_REGION_OPS_FETCH)) {
+hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
+trace_memory_region_ops_fetch(get_cpu_index(), mr, abs_addr, tmp, size,
+ memory_region_name(mr));
+}
+memory_region_shift_read_access(value, shift, mask, tmp);
+return MEMTX_OK;
+}
+
+static MemTxResult memory_region_fetch_with_attrs_accessor(MemoryRegion *mr,
+  hwaddr addr,
+  uint64_t *value,
+ 

[PATCH v7 1/2] hw/misc/riscv_iopmp: Add RISC-V IOPMP device

[Ethan Chen via]

Support basic functions of IOPMP specification v0.9.1 rapid-k model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1

IOPMP check memory access from device is valid or not. This implementation uses
IOMMU to change address space that device access. There are three possible
results of an access: valid, blocked, and stalled(stall is not supported in this
 patch).

If an access is valid, target address space is downstream_as.
If an access is blocked, it will go to blocked_io_as. The operation of
blocked_io_as could be a bus error, or it can respond a success with fabricated
data depending on IOPMP ERR_CFG register value.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |3 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1002 +
 hw/misc/trace-events  |4 +
 include/hw/misc/riscv_iopmp.h |  152 +
 5 files changed, 1162 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 1e08785b83..427f0c702e 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -213,4 +213,7 @@ config IOSB
 config XLNX_VERSAL_TRNG
 bool
 
+config RISCV_IOPMP
+bool
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index 86596a3888..f83cd108f8 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..75b28dc559
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,1002 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+#include "hw/registerfields.h"
+#include "trace.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+
+REG32(VERSION, 0x00)
+FIELD(VERSION, VENDOR, 0, 24)
+FIELD(VERSION, SPECVER , 24, 8)
+REG32(IMP, 0x04)
+FIELD(IMP, IMPID, 0, 32)
+REG32(HWCFG0, 0x08)
+FIELD(HWCFG0, MODEL, 0, 4)
+FIELD(HWCFG0, TOR_EN, 4, 1)
+FIELD(HWCFG0, SPS_EN, 5, 1)
+FIELD(HWCFG0, USER_CFG_EN, 6, 1)
+FIELD(HWCFG0, PRIENT_PROG, 7, 1)
+FIELD(HWCFG0, RRID_TRANSL_EN, 8, 1)
+FIELD(HWCFG0, RRID_TRANSL_PROG, 9, 1)
+FIELD(HWCFG0, CHK_X, 10, 1)
+FIELD(HWCFG0, NO_X, 11, 1)
+FIELD(HWCFG0, NO_W, 12, 1)
+FIELD(HWCFG0, STALL_EN, 13, 1)
+FIELD(HWCFG0, PEIS, 14, 1)
+FIELD(HWCFG0, PEES, 15, 1)
+FIELD(HWCFG0, MFR_EN, 16, 1)
+FIELD(HWCFG0, MD_NUM, 24, 7)
+FIELD(HWCFG0, ENABLE, 31, 1)
+REG32(HWCFG1, 0x0C)
+FIELD(HWCFG1, RRID_NUM, 0, 16)
+FIELD(HWCFG1, ENTRY_NUM, 16, 16)
+REG32(HWCFG2, 0x10)
+FIELD(HWCFG2, PRIO_ENTRY, 0, 16)
+FIELD(HWCFG2, RRID_TRANSL, 16, 16)
+REG32(ENTRYOFFSET, 0x14)
+FIELD(ENTRYOFFSET, OFFSET, 0, 32)
+REG32(MDSTALL, 0x30)
+FIELD(MDSTALL, EXEMPT, 0, 1)
+FIELD(MDSTALL, MD, 1, 31)
+REG32(MDSTALLH, 0x34)
+FIELD(MDSTALLH, MD, 0, 32)
+REG32(RRIDSCP, 0x38)
+FIELD(RRIDSCP, RRID, 0, 16)
+FIELD(RRIDSCP, OP, 30, 2)
+REG32(MDLCK, 0x40)
+FIELD(MDLCK, L, 0, 1)
+FIELD(MDLCK, MD, 1, 31)
+REG32(MDLCKH, 0x44)
+FIELD(MDLCKH, MDH, 0, 32)
+REG32(MDCFGLCK, 0x48)
+FIELD(MDCFGLCK, L, 0, 1)
+FIELD(MDCFGLCK, F, 1, 7)
+REG32(ENTRYLCK, 0x4C)
+FIELD(ENTRYLCK, L, 0, 1)
+FIELD(ENTRYLCK, F, 1, 16)
+REG32(ERR_CFG, 0x60)
+FIELD(ERR_CFG, L, 0, 1)
+FIELD(ERR_CFG, IE, 1, 1)
+FIELD(ERR_CFG, IRE, 2, 1)
+FIELD(ERR_CFG, IWE, 3, 1)
+FIELD(ERR_CFG, IXE, 4, 1)
+FIELD(ERR_CFG, RRE, 5, 1)
+FIELD(ERR_CFG, RWE, 6, 1)
+FIELD(ERR_CFG, RXE, 7, 1)
+REG32(ERR_REQINFO, 0x6

[PATCH v7 2/2] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

If a requestor device is connected to the IOPMP device, its memory access will
be checked by the IOPMP rule.

- Add 'iopmp=on' option to add an iopmp device and make the Generic PCI Express
  Bridge connect to IOPMP.

Signed-off-by: Ethan Chen 
---
 docs/system/riscv/virt.rst |  6 
 hw/riscv/Kconfig   |  1 +
 hw/riscv/virt.c| 57 --
 include/hw/riscv/virt.h|  5 +++-
 4 files changed, 66 insertions(+), 3 deletions(-)

diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
index 9a06f95a34..3b2576f905 100644
--- a/docs/system/riscv/virt.rst
+++ b/docs/system/riscv/virt.rst
@@ -116,6 +116,12 @@ The following machine-specific options are supported:
   having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not specified,
   the default number of per-HART VS-level AIA IMSIC pages is 0.
 
+- iopmp=[on|off]
+
+  When this option is "on", an IOPMP device is added to machine. It checks dma
+  operations from the generic PCIe host bridge. This option is assumed to be
+  "off".
+
 Running Linux kernel
 
 
diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index a2030e3a6f..0b45a5ade2 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -56,6 +56,7 @@ config RISCV_VIRT
 select PLATFORM_BUS
 select ACPI
 select ACPI_PCI
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 4fdb660525..53a1b71c71 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -55,6 +55,7 @@
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
 #include "hw/virtio/virtio-iommu.h"
+#include "hw/misc/riscv_iopmp.h"
 
 /* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */
 static bool virt_use_kvm_aia(RISCVVirtState *s)
@@ -82,6 +83,7 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -1006,6 +1008,24 @@ static void create_fdt_virtio_iommu(RISCVVirtState *s, 
uint16_t bdf)
bdf + 1, iommu_phandle, bdf + 1, 0x - bdf);
 }
 
+static void create_fdt_iopmp(RISCVVirtState *s, const MemMapEntry *memmap,
+ uint32_t irq_mmio_phandle) {
+g_autofree char *name = NULL;
+MachineState *ms = MACHINE(s);
+
+name = g_strdup_printf("/soc/iopmp@%lx", (long)memmap[VIRT_IOPMP].base);
+qemu_fdt_add_subnode(ms->fdt, name);
+qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
+qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, memmap[VIRT_IOPMP].base,
+0x0, memmap[VIRT_IOPMP].size);
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", irq_mmio_phandle);
+if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", IOPMP_IRQ);
+} else {
+qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", IOPMP_IRQ, 0x4);
+}
+}
+
 static void finalize_fdt(RISCVVirtState *s)
 {
 uint32_t phandle = 1, irq_mmio_phandle = 1, msi_pcie_phandle = 1;
@@ -1024,6 +1044,10 @@ static void finalize_fdt(RISCVVirtState *s)
 create_fdt_uart(s, virt_memmap, irq_mmio_phandle);
 
 create_fdt_rtc(s, virt_memmap, irq_mmio_phandle);
+
+if (s->have_iopmp) {
+create_fdt_iopmp(s, virt_memmap, irq_mmio_phandle);
+}
 }
 
 static void create_fdt(RISCVVirtState *s, const MemMapEntry *memmap)
@@ -1404,7 +1428,7 @@ static void virt_machine_init(MachineState *machine)
 RISCVVirtState *s = RISCV_VIRT_MACHINE(machine);
 MemoryRegion *system_memory = get_system_memory();
 MemoryRegion *mask_rom = g_new(MemoryRegion, 1);
-DeviceState *mmio_irqchip, *virtio_irqchip, *pcie_irqchip;
+DeviceState *mmio_irqchip, *virtio_irqchip, *pcie_irqchip, *gpex_dev;
 int i, base_hartid, hart_count;
 int socket_count = riscv_socket_count(machine);
 
@@ -1570,7 +1594,7 @@ static void virt_machine_init(MachineState *machine)
 qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
 }
 
-gpex_pcie_init(system_memory, pcie_irqchip, s);
+gpex_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
 
 create_platform_bus(s, mmio_irqchip);
 
@@ -1581,6 +1605,14 @@ static void virt_machine_init(MachineState *machine)
 sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base,
 qdev_get_gpio_in(mmio_irqchip, RTC_IRQ));
 
+if (s->have_iopmp) {
+DeviceState *iopmp_dev = sysbus_create_simple(TYPE_IOPMP,
+memmap[VIRT_IOPMP].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
+
+iopmp_setup_pci(iopmp_dev, PCI_HOST_BRIDGE(gpex_dev)->bus);
+

[PATCH v7 0/2] Support RISC-V IOPMP

[Ethan Chen via]

Due to changing the referenced specification version, this patch has changed
a lot in this version.

This series implements basic functions of IOPMP specification v0.9.1 rapid-k
model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/releases/tag/v0.9.1

When IOPMP is enabled, memory access from devices will check by IOPMP.

CPU as an IOPMP requestor has not been implemented because the IOTLB does not
support recording sections outside the current CPU address space.

Changes for v7:

  - Change the specification version to v0.9.1
  - Remove the sps extension
  - Remove stall support, transaction information which need requestor device
support.
  - Remove iopmp_cascade option for virt machine
  - Refine 'addr' range checks switch case (Daniel)


Ethan Chen (2):
  hw/misc/riscv_iopmp: Add RISC-V IOPMP device
  hw/riscv/virt: Add IOPMP support

 docs/system/riscv/virt.rst|6 +
 hw/misc/Kconfig   |3 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1002 +
 hw/misc/trace-events  |4 +
 hw/riscv/Kconfig  |1 +
 hw/riscv/virt.c   |   57 +-
 include/hw/misc/riscv_iopmp.h |  152 +
 include/hw/riscv/virt.h   |5 +-
 9 files changed, 1228 insertions(+), 3 deletions(-)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h

-- 
2.34.1

Re: [PATCH v6 0/3] Support RISC-V IOPMP

[Ethan Chen via]

Hi Dainel,

Sorry for the delayed response. I've been busy over the past two months.
I plan to submit the next version of the patch within two weeks.

Thanks,
Ethan Chen

On Mon, May 27, 2024 at 09:09:49AM -0300, Daniel Henrique Barboza wrote:
> Hi Ethan,
> 
> 
> Did you send v7 already and I somehow missed it? This is the latest version I
> have for this IOPMP work in my inbox.
> 
> If this is the latest, do you plan to send a new version? Most of the comments
> made in patche 2 are simple code style changes and there's no technical 
> reason to
> not get this work merged for this release.
> 
> 
> Thanks,
> 
> Daniel
> 
> On 2/7/24 06:34, Ethan Chen wrote:
> > This series implements IOPMP specification v1.0.0-draft4 rapid-k model and 
> > add
> > IOPMP device to RISC-V virt machine.
> > 
> > Patch 1 add config STREAM make other device can reuse /hw/core/stream.c, 
> > IOPMP
> > implementation will use it. Patch 2 implement IOPMP deivce. Patch 3 add 
> > IOPMP
> > device to RISC-V virt machine.
> > 
> > The IOPMP specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> > 
> > Changes for v6:
> >- Rebased
> >- IOPMP: Fix MDLCK_MD and ENTRYLCK_F did not lock correspond registers
> > Fix iopmp_write for ENTRY_ADDRH and ENTRYLCK_L
> >- Refine coding style (Daniel Henrique Barboza)
> > 
> > Thanks,
> > Ethan Chen
> > 
> > Ethan Chen (3):
> >hw/core: Add config stream
> >Add RISC-V IOPMP support
> >hw/riscv/virt: Add IOPMP support
> > 
> >   docs/system/riscv/virt.rst|   12 +
> >   hw/Kconfig|1 +
> >   hw/core/Kconfig   |3 +
> >   hw/core/meson.build   |2 +-
> >   hw/misc/Kconfig   |4 +
> >   hw/misc/meson.build   |1 +
> >   hw/misc/riscv_iopmp.c | 1202 +
> >   hw/misc/trace-events  |4 +
> >   hw/riscv/Kconfig  |1 +
> >   hw/riscv/virt.c   |  110 +-
> >   include/hw/misc/riscv_iopmp.h |  187 +++
> >   .../hw/misc/riscv_iopmp_transaction_info.h|   28 +
> >   include/hw/riscv/virt.h   |8 +-
> >   13 files changed, 1559 insertions(+), 4 deletions(-)
> >   create mode 100644 hw/misc/riscv_iopmp.c
> >   create mode 100644 include/hw/misc/riscv_iopmp.h
> >   create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h
> > 
> 

Re: [PATCH v6 2/3] Add RISC-V IOPMP support

[Ethan Chen via]

On Wed, Feb 14, 2024 at 11:34:55AM -0300, Daniel Henrique Barboza wrote:
>
>
> On 2/7/24 06:34, Ethan Chen wrote:
> > Support specification Version 1.0.0-draft4 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> >
> > The memory transaction from source devices connected to IOPMP will be
> > checked by IOPMP rule. The method of connecting the source device to
> > IOPMP as follows:
> > For a system bus device, device connects to IOPMP by calling
> > address_space_rw to address space iopmp_sysbus_as with
> > MemTxAttrs.requester_id=SID.
> > For a PCI bus device, device connects to IOPMP when it is on a host
> > bridge with iopmp_setup_pci. PCI bus device has default SID from PCI
> > BDF.
> >
> > IOPMP have two optional features need source device support.
> > 1. Partially hit detection: A Source device support IOPMP partially
> > hit detection need to send a transaction_info before transaction
> > start and send a transaction_info with eop after transaction end.
> > IOPMP will additionally check partially hit by transaction_info.
> >
> > 2. Stall: A Source device support IOPMP stall need to resend the
> > transaction when it gets the MEMTX_IOPMP_STALL result
> >
> > There are three possible results of a transaction: valid, blocked, and
> > stalled. If a transaction is valid, target address space is
> > downstream_as(system_memory). If a transaction is blocked, it will go
> > to blocked_io_as. The operation of blocked_io_as could be a bus error,
> > a decode error, or it can respond a success with fabricated data
> > depending on IOPMP ERRREACT register value. If a transaction is
> > stalled, it will go to stall_io_as. The operation of stall_io_as does
> > nothing but return a stall result to source device. Source device
> > should retry the transaction if it gets a stall result.
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >   hw/misc/Kconfig   |4 +
> >   hw/misc/meson.build   |1 +
> >   hw/misc/riscv_iopmp.c | 1202 +
> >   hw/misc/trace-events  |4 +
> >   include/hw/misc/riscv_iopmp.h |  187 +++
> >   .../hw/misc/riscv_iopmp_transaction_info.h|   28 +
> >   6 files changed, 1426 insertions(+)
> >   create mode 100644 hw/misc/riscv_iopmp.c
> >   create mode 100644 include/hw/misc/riscv_iopmp.h
> >   create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h
> >
> > diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
> > index 4fc6b29b43..dd0a2da29b 100644
> > --- a/hw/misc/Kconfig
> > +++ b/hw/misc/Kconfig
> > @@ -206,4 +206,8 @@ config IOSB
> >   config XLNX_VERSAL_TRNG
> >   bool
> > +config RISCV_IOPMP
> > +bool
> > +select STREAM
> > +
> >   source macio/Kconfig
> > diff --git a/hw/misc/meson.build b/hw/misc/meson.build
> > index e4ef1da5a5..74221e48f9 100644
> > --- a/hw/misc/meson.build
> > +++ b/hw/misc/meson.build
> > @@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
> > files('sifive_e_prci.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: 
> > files('sifive_e_aon.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: 
> > files('sifive_u_otp.c'))
> >   system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: 
> > files('sifive_u_prci.c'))
> > +specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: 
> > files('riscv_iopmp.c'))
> >   subdir('macio')
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > new file mode 100644
> > index 00..e9d5d171a6
> > --- /dev/null
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -0,0 +1,1202 @@
> > +/*
> > + * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
> > + *
> > + * Copyright (c) 2023 Andes Tech. Corp.
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms and conditions of the GNU General Public License,
> > + * version 2 or later, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope it will be useful, but WITHOUT
> > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License 
> > for
> > + * more details.
> > + *
> > + * You should have received a copy of the GNU General Public License along 
> > with
> > + * this program.  If not, see .
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/log.h"
> > +#include "qapi/error.h"
> > +#include "trace.h"
> > +#include "exec/exec-all.h"
> > +#include "exec/address-spaces.h"
> > +#include "hw/qdev-properties.h"
> > +#include "hw/sysbus.h"
> > +#include "hw/misc/riscv_iopmp.h"
> > +#include "memory.h"
> > +#include "hw/irq.h"
> > +#include "hw/registerfields.h"
> > +#include "trace.h"
> > +
> > +#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
> > +

[PATCH v6 3/3] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

If a source device is connected to the IOPMP device, its memory
transaction will be checked by the IOPMP rule.

When using RISC-V virt machine option "iopmp=on", the generic PCIe host
bridge connects to IOPMP. The PCI devices on the bridge will connect to
IOPMP with default source id(SID) from PCI BDF.

- Add 'iopmp=on' option to add an iopmp device. It checks dma
  operations from the generic PCIe host bridge. This option is assumed
  to be "off"
- Add 'iopmp_cascade=on' option to add second iopmp device which is
  cascaded by first iopmp device to machine. When iopmp option is "off"
  , this option has no effect.

Signed-off-by: Ethan Chen 
---
 docs/system/riscv/virt.rst |  12 
 hw/riscv/Kconfig   |   1 +
 hw/riscv/virt.c| 110 -
 include/hw/riscv/virt.h|   8 ++-
 4 files changed, 128 insertions(+), 3 deletions(-)

diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
index 9a06f95a34..ffcbe3a562 100644
--- a/docs/system/riscv/virt.rst
+++ b/docs/system/riscv/virt.rst
@@ -116,6 +116,18 @@ The following machine-specific options are supported:
   having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not specified,
   the default number of per-HART VS-level AIA IMSIC pages is 0.
 
+- iopmp=[on|off]
+
+  When this option is "on", an IOPMP device is added to machine. It checks dma
+  operations from the generic PCIe host bridge. This option is assumed to be
+  "off".
+
+- iopmp_cascade=[on|off]
+
+  When this option is "on", a second IOPMP device which is cascaded by first
+  IOPMP device is added to machine. When IOPMP option is "off", this option has
+  no effect. This option is assumed to be "off".
+
 Running Linux kernel
 
 
diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index a50717be87..c207b94747 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -46,6 +46,7 @@ config RISCV_VIRT
 select PLATFORM_BUS
 select ACPI
 select ACPI_PCI
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index f9fd1341fc..9a3afca8d7 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -52,6 +52,7 @@
 #include "hw/display/ramfb.h"
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
+#include "hw/misc/riscv_iopmp.h"
 
 /* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */
 static bool virt_use_kvm_aia(RISCVVirtState *s)
@@ -74,6 +75,8 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
+[VIRT_IOPMP2] =   { 0x1030,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -1011,6 +1014,42 @@ static void create_fdt_fw_cfg(RISCVVirtState *s, const 
MemMapEntry *memmap)
 g_free(nodename);
 }
 
+static void create_fdt_iopmp(RISCVVirtState *s, const MemMapEntry *memmap,
+ uint32_t irq_mmio_phandle) {
+g_autofree char *name = NULL;
+MachineState *ms = MACHINE(s);
+
+name = g_strdup_printf("/soc/iopmp@%lx", (long)memmap[VIRT_IOPMP].base);
+qemu_fdt_add_subnode(ms->fdt, name);
+qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
+qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, memmap[VIRT_IOPMP].base,
+0x0, memmap[VIRT_IOPMP].size);
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", irq_mmio_phandle);
+if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", IOPMP_IRQ);
+} else {
+qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", IOPMP_IRQ, 0x4);
+}
+}
+
+static void create_fdt_iopmp2(RISCVVirtState *s, const MemMapEntry *memmap,
+ uint32_t irq_mmio_phandle) {
+g_autofree char *name;
+MachineState *ms = MACHINE(s);
+
+name = g_strdup_printf("/soc/iopmp2@%lx", (long)memmap[VIRT_IOPMP2].base);
+qemu_fdt_add_subnode(ms->fdt, name);
+qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
+qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, memmap[VIRT_IOPMP2].base,
+0x0, memmap[VIRT_IOPMP2].size);
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", irq_mmio_phandle);
+if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", IOPMP2_IRQ);
+} else {
+qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", IOPMP2_IRQ, 0x4);
+}
+}
+
 static void finalize_fdt(RISCVVirtState *s)
 {
 uint32_t phandle = 1, irq_mmio_phandle = 1, msi_pcie_phandle = 1;
@@ -1029,6 +1068,13 @@ static void finalize_fdt(RISCVVirtState *s)
 create_fdt_uart(s, virt_memmap, irq_mmio_pha

[PATCH v6 1/3] hw/core: Add config stream

[Ethan Chen via]

Make other device can use /hw/core/stream.c by select this config.

Reviewed-by: Alistair Francis 
Signed-off-by: Ethan Chen 
---
 hw/Kconfig  | 1 +
 hw/core/Kconfig | 3 +++
 hw/core/meson.build | 2 +-
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/Kconfig b/hw/Kconfig
index 2c00936c28..5554b0663f 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -80,6 +80,7 @@ config XILINX
 config XILINX_AXI
 bool
 select PTIMER # for hw/dma/xilinx_axidma.c
+select STREAM
 
 config XLNX_ZYNQMP
 bool
diff --git a/hw/core/Kconfig b/hw/core/Kconfig
index 9397503656..e89ffa728b 100644
--- a/hw/core/Kconfig
+++ b/hw/core/Kconfig
@@ -27,3 +27,6 @@ config REGISTER
 
 config SPLIT_IRQ
 bool
+
+config STREAM
+bool
diff --git a/hw/core/meson.build b/hw/core/meson.build
index 67dad04de5..0893917b12 100644
--- a/hw/core/meson.build
+++ b/hw/core/meson.build
@@ -32,8 +32,8 @@ system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: 
files('platform-bus.c'))
 system_ss.add(when: 'CONFIG_PTIMER', if_true: files('ptimer.c'))
 system_ss.add(when: 'CONFIG_REGISTER', if_true: files('register.c'))
 system_ss.add(when: 'CONFIG_SPLIT_IRQ', if_true: files('split-irq.c'))
-system_ss.add(when: 'CONFIG_XILINX_AXI', if_true: files('stream.c'))
 system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: files('sysbus-fdt.c'))
+system_ss.add(when: 'CONFIG_STREAM', if_true: files('stream.c'))
 
 system_ss.add(files(
   'cpu-sysemu.c',
-- 
2.34.1

[PATCH v6 0/3] Support RISC-V IOPMP

[Ethan Chen via]

This series implements IOPMP specification v1.0.0-draft4 rapid-k model and add
IOPMP device to RISC-V virt machine.

Patch 1 add config STREAM make other device can reuse /hw/core/stream.c, IOPMP
implementation will use it. Patch 2 implement IOPMP deivce. Patch 3 add IOPMP
device to RISC-V virt machine.

The IOPMP specification url:
https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf

Changes for v6:
  - Rebased
  - IOPMP: Fix MDLCK_MD and ENTRYLCK_F did not lock correspond registers
   Fix iopmp_write for ENTRY_ADDRH and ENTRYLCK_L
  - Refine coding style (Daniel Henrique Barboza)

Thanks,
Ethan Chen

Ethan Chen (3):
  hw/core: Add config stream
  Add RISC-V IOPMP support
  hw/riscv/virt: Add IOPMP support

 docs/system/riscv/virt.rst|   12 +
 hw/Kconfig|1 +
 hw/core/Kconfig   |3 +
 hw/core/meson.build   |2 +-
 hw/misc/Kconfig   |4 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1202 +
 hw/misc/trace-events  |4 +
 hw/riscv/Kconfig  |1 +
 hw/riscv/virt.c   |  110 +-
 include/hw/misc/riscv_iopmp.h |  187 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|   28 +
 include/hw/riscv/virt.h   |8 +-
 13 files changed, 1559 insertions(+), 4 deletions(-)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

-- 
2.34.1

[PATCH v6 2/3] Add RISC-V IOPMP support

[Ethan Chen via]

Support specification Version 1.0.0-draft4 rapid-k model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf

The memory transaction from source devices connected to IOPMP will be
checked by IOPMP rule. The method of connecting the source device to
IOPMP as follows:
For a system bus device, device connects to IOPMP by calling
address_space_rw to address space iopmp_sysbus_as with
MemTxAttrs.requester_id=SID.
For a PCI bus device, device connects to IOPMP when it is on a host
bridge with iopmp_setup_pci. PCI bus device has default SID from PCI
BDF.

IOPMP have two optional features need source device support.
1. Partially hit detection: A Source device support IOPMP partially
   hit detection need to send a transaction_info before transaction
   start and send a transaction_info with eop after transaction end.
   IOPMP will additionally check partially hit by transaction_info.

2. Stall: A Source device support IOPMP stall need to resend the
   transaction when it gets the MEMTX_IOPMP_STALL result

There are three possible results of a transaction: valid, blocked, and
stalled. If a transaction is valid, target address space is
downstream_as(system_memory). If a transaction is blocked, it will go
to blocked_io_as. The operation of blocked_io_as could be a bus error,
a decode error, or it can respond a success with fabricated data
depending on IOPMP ERRREACT register value. If a transaction is
stalled, it will go to stall_io_as. The operation of stall_io_as does
nothing but return a stall result to source device. Source device
should retry the transaction if it gets a stall result.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |4 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1202 +
 hw/misc/trace-events  |4 +
 include/hw/misc/riscv_iopmp.h |  187 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|   28 +
 6 files changed, 1426 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 4fc6b29b43..dd0a2da29b 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -206,4 +206,8 @@ config IOSB
 config XLNX_VERSAL_TRNG
 bool
 
+config RISCV_IOPMP
+bool
+select STREAM
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index e4ef1da5a5..74221e48f9 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..e9d5d171a6
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,1202 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+#include "hw/registerfields.h"
+#include "trace.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+#define TYPE_IOPMP_TRASACTION_INFO_SINK "iopmp_transaction_info_sink"
+
+DECLARE_INSTANCE_CHECKER(Iopmp_StreamSink, IOPMP_TRASACTION_INFO_SINK,
+ TYPE_IOPMP_TRASACTION_INFO_SINK)
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+REG32(VERSION, 0x00)
+FIELD(VERSION, VENDOR, 0, 24)
+FIELD(VERSION, SPECVER , 24, 8)
+REG32(IMP, 0x04)
+FIELD(IMP, IMPID, 0, 32)
+REG32(HWCFG0, 0x08)
+FIELD(HWCFG0, SID_NUM, 0, 16)
+FIELD(HWCFG0, ENTRY_NUM, 16, 16)
+REG32(HWCFG1, 0x0C)
+FIELD(HWCFG1, MODEL, 0, 4)
+FIELD(HWCFG1, TOR_EN, 4, 1)
+FIELD(HWCFG1, SPS_EN,

Re: [PATCH v4 0/4] Support RISC-V IOPMP

[Ethan Chen via]

On Mon, Jan 22, 2024 at 04:01:12PM +1000, Alistair Francis wrote:
> On Thu, Dec 21, 2023 at 4:38 PM Ethan Chen  wrote:
> >
> > On Mon, Dec 18, 2023 at 02:18:58PM +1000, Alistair Francis wrote:
> > > On Wed, Nov 22, 2023 at 3:36 PM Ethan Chen via  
> > > wrote:
> > > >
> > > > This series implements IOPMP specification v1.0.0-draft4 rapid-k model.
> > > > The specification url:
> > > > https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> > > >
> > > > When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
> > > > platform. This DMA device is connected to the IOPMP and has the 
> > > > functionalities
> > >
> > > I don't think we want to add an Andes DMA device to the virt machine.
> > >
> > > I can't even find the spec for the ATCDMAC300, which isn't great
> > >
> > > Alistair
> >
> > Since the IOPMP does not take effect when there is no other device connects 
> > to
> > IOPMP, I think it is necessary to have a DMA device for IOPMP demonstration.
> 
> That is true, but that device shouldn't be a vendor specific device
> for the virt machine.
> 
> >
> > Do you have any suggestions for supporting IOPMP on RISC-V virt machine?
> 
> A RVI device would be fine. Otherwise something that has become a
> defacto standard by being commonly used (the SiFive PLIC for example).
> 
> I really don't think it should be some vendor IP, especially one that
> doesn't have a public datasheet.
> 
> You could add an Andes machine that can use your vendor IP. Otherwise
> we can look at adding IOPMP and not connecting it, but that is a pain.

In submitted patch v5, I removed vendor IP and made generic PCIe host
bridge on RISC-V virt machine connect to IOPMP. DMA operation from PCI
devices on the bridge will be check by IOPMP.

> 
> What is the IOPMP spec group doing for testing?

IOPMP TG is doing an implementation(RTL) testing.

NVidia will provide SystemC stimulus from different ports to test or 
observe the object under testing. A test bench will be provided in 
the form of TLM-2.0 transaction level modeling.

Thanks,
Ethan Chen

[PATCH v5 3/3] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

If a source device is connected to the IOPMP device, its memory
transaction will be checked by the IOPMP rule.

When using RISC-V virt machine option "iopmp=on", the generic PCIe host
bridge connects to IOPMP. The PCI devices on the brigde will connets to
IOPMP with default source id(SID) from PCI BDF.

- Add 'iopmp=on' option to add an iopmp device. It checks dma
  operations from the generic PCIe host bridge. This option is assumed
  to be "off"
- Add 'iopmp_cascade=on' option to add second iopmp device which is
  cascaded by first iopmp device to machine. When iopmp option is "off"
  , this option has no effect.

Signed-off-by: Ethan Chen 
---
 docs/system/riscv/virt.rst |  12 
 hw/riscv/Kconfig   |   1 +
 hw/riscv/virt.c| 110 -
 include/hw/riscv/virt.h|   8 ++-
 4 files changed, 129 insertions(+), 2 deletions(-)

diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
index 9a06f95a34..e07ce2cd28 100644
--- a/docs/system/riscv/virt.rst
+++ b/docs/system/riscv/virt.rst
@@ -116,6 +116,18 @@ The following machine-specific options are supported:
   having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not specified,
   the default number of per-HART VS-level AIA IMSIC pages is 0.
 
+- iopmp=[on|off]
+
+  When this option is "on".  An IOPMP device is added to machine. It checks dma
+  operations from the generic PCIe host bridge. This option is assumed to be
+  "off".
+
+- iopmp_cascade=[on|off]
+
+  When this option is "on". Second IOPMP device which is cascaded by first 
IOPMP
+  device is added to machine. When IOPMP option is "off", this option has no
+  effect. This option is assumed to be "off".
+
 Running Linux kernel
 
 
diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index a50717be87..c207b94747 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -46,6 +46,7 @@ config RISCV_VIRT
 select PLATFORM_BUS
 select ACPI
 select ACPI_PCI
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index f9fd1341fc..c7a5035074 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -52,6 +52,7 @@
 #include "hw/display/ramfb.h"
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
+#include "hw/misc/riscv_iopmp.h"
 
 /* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */
 static bool virt_use_kvm_aia(RISCVVirtState *s)
@@ -74,6 +75,8 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
+[VIRT_IOPMP2] =   { 0x1030,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -1011,6 +1014,44 @@ static void create_fdt_fw_cfg(RISCVVirtState *s, const 
MemMapEntry *memmap)
 g_free(nodename);
 }
 
+static void create_fdt_iopmp(RISCVVirtState *s, const MemMapEntry *memmap,
+ uint32_t irq_mmio_phandle) {
+char *name;
+MachineState *ms = MACHINE(s);
+
+name = g_strdup_printf("/soc/iopmp@%lx", (long)memmap[VIRT_IOPMP].base);
+qemu_fdt_add_subnode(ms->fdt, name);
+qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
+qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, memmap[VIRT_IOPMP].base,
+0x0, memmap[VIRT_IOPMP].size);
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", irq_mmio_phandle);
+if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", IOPMP_IRQ);
+} else {
+qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", IOPMP_IRQ, 0x4);
+}
+g_free(name);
+}
+
+static void create_fdt_iopmp2(RISCVVirtState *s, const MemMapEntry *memmap,
+ uint32_t irq_mmio_phandle) {
+char *name;
+MachineState *ms = MACHINE(s);
+
+name = g_strdup_printf("/soc/iopmp2@%lx", (long)memmap[VIRT_IOPMP2].base);
+qemu_fdt_add_subnode(ms->fdt, name);
+qemu_fdt_setprop_string(ms->fdt, name, "compatible", "riscv_iopmp");
+qemu_fdt_setprop_cells(ms->fdt, name, "reg", 0x0, memmap[VIRT_IOPMP2].base,
+0x0, memmap[VIRT_IOPMP2].size);
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", irq_mmio_phandle);
+if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", IOPMP2_IRQ);
+} else {
+qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", IOPMP2_IRQ, 0x4);
+}
+g_free(name);
+}
+
 static void finalize_fdt(RISCVVirtState *s)
 {
 uint32_t phandle = 1, irq_mmio_phandle = 1, msi_pcie_phandle = 1;
@@ -1029,6 +1070,13 @@ static void finalize_fdt(RISCVVirtState *s)
 create_fdt_uart(s, virt_memmap, irq

[PATCH v5 2/3] Add RISC-V IOPMP support

[Ethan Chen via]

Support specification Version 1.0.0-draft4 rapid-k model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf

The memory transaction from source devices connected to IOPMP will be
checked by IOPMP rule. The method of connecting the source device to
IOPMP as follows:
For a system bus device, device connects to IOPMP by calling
address_space_rw to address space iopmp_sysbus_as with
MemTxAttrs.requester_id=SID.
For a PCI bus device, device connects to IOPMP when it is on a host
bridge with iopmp_setup_pci. PCI bus device has default SID from PCI
BDF.

IOPMP have two optional features need source device support.
1. Partially hit detection: A Source device support IOPMP partially
   hit detection need to send a transaction_info before transaction
   start and send a transaction_info with eop after transaction end.
   IOPMP will additionally check partially hit by transaction_info.

2. Stall: A Source device support IOPMP stall need to resend the
   transaction when it gets the MEMTX_IOPMP_STALL result

There are three possible results of a transaction: valid, blocked, and
stalled. If a transaction is valid, target address space is
downstream_as(system_memory). If a transaction is blocked, it will go
to blocked_io_as. The operation of blocked_io_as could be a bus error,
a decode error, or it can respond a success with fabricated data
depending on IOPMP ERRREACT register value. If a transaction is
stalled, it will go to stall_io_as. The operation of stall_io_as does
nothing but return a stall result to source device. Source device
should retry the transaction if it gets a stall result.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |4 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1130 +
 hw/misc/trace-events  |4 +
 include/hw/misc/riscv_iopmp.h |  187 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|   28 +
 6 files changed, 1354 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index cc8a8c1418..953569e682 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -200,4 +200,8 @@ config IOSB
 config XLNX_VERSAL_TRNG
 bool
 
+config RISCV_IOPMP
+bool
+select STREAM
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index 36c20d5637..86b81e1690 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..468863bccf
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,1130 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+#include "hw/registerfields.h"
+#include "trace.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+#define TYPE_IOPMP_TRASACTION_INFO_SINK "iopmp_transaction_info_sink"
+
+DECLARE_INSTANCE_CHECKER(Iopmp_StreamSink, IOPMP_TRASACTION_INFO_SINK,
+ TYPE_IOPMP_TRASACTION_INFO_SINK)
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+REG32(VERSION, 0x00)
+FIELD(VERSION, VENDOR, 0, 24)
+FIELD(VERSION, SPECVER , 24, 8)
+REG32(IMP, 0x04)
+FIELD(IMP, IMPID, 0, 32)
+REG32(HWCFG0, 0x08)
+FIELD(HWCFG0, SID_NUM, 0, 16)
+FIELD(HWCFG0, ENTRY_NUM, 16, 16)
+REG32(HWCFG1, 0x0C)
+FIELD(HWCFG1, MODEL, 0, 4)
+FIELD(HWCFG1, TOR_EN, 4, 1)
+FIELD(HWCFG1, SPS_EN,

[PATCH v5 1/3] hw/core: Add config stream

[Ethan Chen via]

Make other device can use /hw/core/stream.c by select this config.

Reviewed-by: Alistair Francis 
Signed-off-by: Ethan Chen 
---
 hw/Kconfig  | 1 +
 hw/core/Kconfig | 3 +++
 hw/core/meson.build | 2 +-
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/Kconfig b/hw/Kconfig
index 9ca7b38c31..e4d153dce7 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -79,6 +79,7 @@ config XILINX
 config XILINX_AXI
 bool
 select PTIMER # for hw/dma/xilinx_axidma.c
+select STREAM
 
 config XLNX_ZYNQMP
 bool
diff --git a/hw/core/Kconfig b/hw/core/Kconfig
index 9397503656..e89ffa728b 100644
--- a/hw/core/Kconfig
+++ b/hw/core/Kconfig
@@ -27,3 +27,6 @@ config REGISTER
 
 config SPLIT_IRQ
 bool
+
+config STREAM
+bool
diff --git a/hw/core/meson.build b/hw/core/meson.build
index 67dad04de5..0893917b12 100644
--- a/hw/core/meson.build
+++ b/hw/core/meson.build
@@ -32,8 +32,8 @@ system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: 
files('platform-bus.c'))
 system_ss.add(when: 'CONFIG_PTIMER', if_true: files('ptimer.c'))
 system_ss.add(when: 'CONFIG_REGISTER', if_true: files('register.c'))
 system_ss.add(when: 'CONFIG_SPLIT_IRQ', if_true: files('split-irq.c'))
-system_ss.add(when: 'CONFIG_XILINX_AXI', if_true: files('stream.c'))
 system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: files('sysbus-fdt.c'))
+system_ss.add(when: 'CONFIG_STREAM', if_true: files('stream.c'))
 
 system_ss.add(files(
   'cpu-sysemu.c',
-- 
2.34.1

[PATCH v5 0/3] Support RISC-V IOPMP

[Ethan Chen via]

This series implements IOPMP specification v1.0.0-draft4 rapid-k model and add
IOPMP device to RISC-V virt machine.

Patch 1 add config STREAM make other device can reuse /hw/core/stream.c, IOPMP
implementation will use it. Patch 2 implement IOPMP deivce. Patch 3 add IOPMP
device to RISC-V virt machine.

The IOPMP specification url:
https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf

Changes for v5:
  - Rebase
  - IOPMP: Support tracing (Alistair Francis)
   Use macros from registerfields.h for register (Alistair 
Francis)
   Support PCI device
   Drop IOPMP device create helper function (Alistair Francis)
  - Remove ATCDMAC300 (Alistair Francis)
  - VIRT: Make PCIe bridge connect to IOPMP
  Modify document for IOPMP options
  Add IOPMP fdt

Thanks,
Ethan Chen

Ethan Chen (3):
  hw/core: Add config stream
  Add RISC-V IOPMP support
  hw/riscv/virt: Add IOPMP support

 docs/system/riscv/virt.rst|   12 +
 hw/Kconfig|1 +
 hw/core/Kconfig   |3 +
 hw/core/meson.build   |2 +-
 hw/misc/Kconfig   |4 +
 hw/misc/meson.build   |1 +
 hw/misc/riscv_iopmp.c | 1130 +
 hw/misc/trace-events  |4 +
 hw/riscv/Kconfig  |1 +
 hw/riscv/virt.c   |  110 +-
 include/hw/misc/riscv_iopmp.h |  187 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|   28 +
 include/hw/riscv/virt.h   |8 +-
 13 files changed, 1488 insertions(+), 3 deletions(-)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

-- 
2.34.1

Re: [PATCH v4 0/4] Support RISC-V IOPMP

[Ethan Chen via]

On Mon, Dec 18, 2023 at 02:18:58PM +1000, Alistair Francis wrote:
> On Wed, Nov 22, 2023 at 3:36 PM Ethan Chen via  wrote:
> >
> > This series implements IOPMP specification v1.0.0-draft4 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> >
> > When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
> > platform. This DMA device is connected to the IOPMP and has the 
> > functionalities
> 
> I don't think we want to add an Andes DMA device to the virt machine.
> 
> I can't even find the spec for the ATCDMAC300, which isn't great
> 

AndeShape ATCDMAC110 data sheet is available on Andes website
http://www.andestech.com/en/products-solutions/product-documentation/

ATCDMAC300 is compatible with ATCDMAC110.

Thanks,
Ethan Chen

> 
> > required by IOPMP, including:
> > - Support setup the connection to IOPMP
> > - Support asynchronous I/O to handle stall transactions
> > - Send transaction information
> >
> > IOPMP takes a transaction which partially match an entry as a partially hit
> > error. The transaction size is depending on source device, destination 
> > device
> > and bus.
> >
> > Source device can send a transaction_info to IOPMP. IOPMP will check 
> > partially
> > hit by transaction_info. If source device does not send a transaction_info,
> > IOPMP checks information in IOMMU and dose not check partially hit.
> >
> > Changes for v4:
> >
> >   - Add descriptions of IOPMP and ATCDMAC300
> >   - Refine coding style and comments
> >   - config XILINX_AXI does not include file stream.c but selects config 
> > STREAM
> > instead.
> >   - ATCDMAC300: INT_STATUS is write 1 clear per bit
> > Rename iopmp_address_sink to 
> > transcation_info_sink
> >   - IOPMP: Refine error message and remove unused variable
> >   - VIRT: Document new options
> > atcdmac300 is only added when iopmp is enabled
> >   serial setting should not be changed
> >
> > Ethan Chen (4):
> >   hw/core: Add config stream
> >   Add RISC-V IOPMP support
> >   hw/dma: Add Andes ATCDMAC300 support
> >   hw/riscv/virt: Add IOPMP support
> >
> >  docs/system/riscv/virt.rst|  11 +
> >  hw/Kconfig|   1 +
> >  hw/core/Kconfig   |   3 +
> >  hw/core/meson.build   |   2 +-
> >  hw/dma/Kconfig|   4 +
> >  hw/dma/atcdmac300.c   | 566 ++
> >  hw/dma/meson.build|   1 +
> >  hw/misc/Kconfig   |   4 +
> >  hw/misc/meson.build   |   1 +
> >  hw/misc/riscv_iopmp.c | 966 ++
> >  hw/riscv/Kconfig  |   2 +
> >  hw/riscv/virt.c   |  65 ++
> >  include/hw/dma/atcdmac300.h   | 180 
> >  include/hw/misc/riscv_iopmp.h | 341 +++
> >  .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
> >  include/hw/riscv/virt.h   |  10 +-
> >  16 files changed, 2183 insertions(+), 2 deletions(-)
> >  create mode 100644 hw/dma/atcdmac300.c
> >  create mode 100644 hw/misc/riscv_iopmp.c
> >  create mode 100644 include/hw/dma/atcdmac300.h
> >  create mode 100644 include/hw/misc/riscv_iopmp.h
> >  create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h
> >
> > --
> > 2.34.1
> >
> >

Re: [PATCH v4 2/4] Add RISC-V IOPMP support

[Ethan Chen via]

On Mon, Dec 18, 2023 at 02:04:06PM +1000, Alistair Francis wrote:
> On Wed, Nov 22, 2023 at 3:35 PM Ethan Chen via  wrote:
> >
> > Support specification Version 1.0.0-draft4 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> >
> > IOPMP check memory access from deivce is valid or not. This implementation 
> > uses
> > IOMMU to change address space that device access. There are three possible
> > results of an access: valid, blocked, and stalled.
> >
> > If an access is valid, target address spcae is downstream_as(system_memory).
> > If an access is blocked, it will go to blocked_io_as. The operation of
> > blocked_io_as could be a bus error, a decode error, or it can respond a 
> > success
> > with fabricated data depending on IOPMP ERRREACT register value.
> > If an access is stalled, it will go to stall_io_as. The operation of 
> > stall_io_as
> > does nothing but return a stall result to source device. Source device 
> > should
> > retry the access if it gets a stall result.
> >
> > IOPMP implementation is rely on bus signal. For example IOPMP on AXI bus 
> > checks
> > the AXI burst transaction. A streamsink to receive general 
> > transaction_info(sid,
> > start address, end address) is added to IOPMP.
> >
> > If the source device support transaction_info, it can first send a
> > transaction_info to IOPMP streamsink then do the memory access. IOPMP will 
> > do
> > additional partially hit check with transaction info.
> > If the source device does not support transaction info. IOPMP will not check
> > partially hit.
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  hw/misc/Kconfig   |   4 +
> >  hw/misc/meson.build   |   1 +
> >  hw/misc/riscv_iopmp.c | 966 ++
> >  include/hw/misc/riscv_iopmp.h | 341 +++
> >  .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
> >  5 files changed, 1340 insertions(+)
> >  create mode 100644 hw/misc/riscv_iopmp.c
> >  create mode 100644 include/hw/misc/riscv_iopmp.h
> >  create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h
> >
> > diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
> > index cc8a8c1418..953569e682 100644
> > --- a/hw/misc/Kconfig
> > +++ b/hw/misc/Kconfig
> > @@ -200,4 +200,8 @@ config IOSB
> >  config XLNX_VERSAL_TRNG
> >  bool
> >
> > +config RISCV_IOPMP
> > +bool
> > +select STREAM
> > +
> >  source macio/Kconfig
> > diff --git a/hw/misc/meson.build b/hw/misc/meson.build
> > index 36c20d5637..86b81e1690 100644
> > --- a/hw/misc/meson.build
> > +++ b/hw/misc/meson.build
> > @@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
> > files('sifive_e_prci.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: 
> > files('sifive_e_aon.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: 
> > files('sifive_u_otp.c'))
> >  system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: 
> > files('sifive_u_prci.c'))
> > +specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: 
> > files('riscv_iopmp.c'))
> >
> >  subdir('macio')
> >
> > diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
> > new file mode 100644
> > index 00..098a3b81fd
> > --- /dev/null
> > +++ b/hw/misc/riscv_iopmp.c
> > @@ -0,0 +1,966 @@
> > +/*
> > + * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
> > + *
> > + * Copyright (c) 2023 Andes Tech. Corp.
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms and conditions of the GNU General Public License,
> > + * version 2 or later, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope it will be useful, but WITHOUT
> > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License 
> > for
> > + * more details.
> > + *
> > + * You should have received a copy of the GNU General Public License along 
> > with
> > + * this program.  If not, see <http://www.gnu.org/licenses/>.
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/log

Re: [PATCH v4 0/4] Support RISC-V IOPMP

[Ethan Chen via]

On Mon, Dec 18, 2023 at 02:18:58PM +1000, Alistair Francis wrote:
> On Wed, Nov 22, 2023 at 3:36 PM Ethan Chen via  wrote:
> >
> > This series implements IOPMP specification v1.0.0-draft4 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> >
> > When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
> > platform. This DMA device is connected to the IOPMP and has the 
> > functionalities
> 
> I don't think we want to add an Andes DMA device to the virt machine.
> 
> I can't even find the spec for the ATCDMAC300, which isn't great
> 
> Alistair

Since the IOPMP does not take effect when there is no other device connects to
IOPMP, I think it is necessary to have a DMA device for IOPMP demonstration.

Do you have any suggestions for supporting IOPMP on RISC-V virt machine?

Thanks,
Ethan Chen

> 
> > required by IOPMP, including:
> > - Support setup the connection to IOPMP
> > - Support asynchronous I/O to handle stall transactions
> > - Send transaction information
> >
> > IOPMP takes a transaction which partially match an entry as a partially hit
> > error. The transaction size is depending on source device, destination 
> > device
> > and bus.
> >
> > Source device can send a transaction_info to IOPMP. IOPMP will check 
> > partially
> > hit by transaction_info. If source device does not send a transaction_info,
> > IOPMP checks information in IOMMU and dose not check partially hit.
> >
> > Changes for v4:
> >
> >   - Add descriptions of IOPMP and ATCDMAC300
> >   - Refine coding style and comments
> >   - config XILINX_AXI does not include file stream.c but selects config 
> > STREAM
> > instead.
> >   - ATCDMAC300: INT_STATUS is write 1 clear per bit
> > Rename iopmp_address_sink to 
> > transcation_info_sink
> >   - IOPMP: Refine error message and remove unused variable
> >   - VIRT: Document new options
> > atcdmac300 is only added when iopmp is enabled
> >   serial setting should not be changed
> >
> > Ethan Chen (4):
> >   hw/core: Add config stream
> >   Add RISC-V IOPMP support
> >   hw/dma: Add Andes ATCDMAC300 support
> >   hw/riscv/virt: Add IOPMP support
> >
> >  docs/system/riscv/virt.rst|  11 +
> >  hw/Kconfig|   1 +
> >  hw/core/Kconfig   |   3 +
> >  hw/core/meson.build   |   2 +-
> >  hw/dma/Kconfig|   4 +
> >  hw/dma/atcdmac300.c   | 566 ++
> >  hw/dma/meson.build|   1 +
> >  hw/misc/Kconfig   |   4 +
> >  hw/misc/meson.build   |   1 +
> >  hw/misc/riscv_iopmp.c | 966 ++
> >  hw/riscv/Kconfig  |   2 +
> >  hw/riscv/virt.c   |  65 ++
> >  include/hw/dma/atcdmac300.h   | 180 
> >  include/hw/misc/riscv_iopmp.h | 341 +++
> >  .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
> >  include/hw/riscv/virt.h   |  10 +-
> >  16 files changed, 2183 insertions(+), 2 deletions(-)
> >  create mode 100644 hw/dma/atcdmac300.c
> >  create mode 100644 hw/misc/riscv_iopmp.c
> >  create mode 100644 include/hw/dma/atcdmac300.h
> >  create mode 100644 include/hw/misc/riscv_iopmp.h
> >  create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h
> >
> > --
> > 2.34.1
> >
> >

Re: [PATCH v4 0/4] Support RISC-V IOPMP

[Ethan Chen via]

Ping again.

On Tue, Dec 05, 2023 at 03:48:07PM +0800, Ethan Chen wrote:
> Ping.
> https://patchew.org/QEMU/20231122053251.440723-1-etha...@andestech.com/
> 
> On Wed, Nov 22, 2023 at 01:32:47PM +0800, Ethan Chen wrote:
> > This series implements IOPMP specification v1.0.0-draft4 rapid-k model.
> > The specification url:
> > https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> > 
> > When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
> > platform. This DMA device is connected to the IOPMP and has the 
> > functionalities
> > required by IOPMP, including:
> > - Support setup the connection to IOPMP
> > - Support asynchronous I/O to handle stall transactions
> > - Send transaction information
> > 
> > IOPMP takes a transaction which partially match an entry as a partially hit
> > error. The transaction size is depending on source device, destination 
> > device
> > and bus.
> > 
> > Source device can send a transaction_info to IOPMP. IOPMP will check 
> > partially
> > hit by transaction_info. If source device does not send a transaction_info,
> > IOPMP checks information in IOMMU and dose not check partially hit.
> > 
> > Changes for v4:
> > 
> >   - Add descriptions of IOPMP and ATCDMAC300
> >   - Refine coding style and comments
> >   - config XILINX_AXI does not include file stream.c but selects config 
> > STREAM
> > instead.
> >   - ATCDMAC300: INT_STATUS is write 1 clear per bit
> > Rename iopmp_address_sink to transcation_info_sink
> >   - IOPMP: Refine error message and remove unused variable
> >   - VIRT: Document new options
> > atcdmac300 is only added when iopmp is enabled
> >   serial setting should not be changed
> > 
> > Ethan Chen (4):
> >   hw/core: Add config stream
> >   Add RISC-V IOPMP support
> >   hw/dma: Add Andes ATCDMAC300 support
> >   hw/riscv/virt: Add IOPMP support
> > 
> >  docs/system/riscv/virt.rst|  11 +
> >  hw/Kconfig|   1 +
> >  hw/core/Kconfig   |   3 +
> >  hw/core/meson.build   |   2 +-
> >  hw/dma/Kconfig|   4 +
> >  hw/dma/atcdmac300.c   | 566 ++
> >  hw/dma/meson.build|   1 +
> >  hw/misc/Kconfig   |   4 +
> >  hw/misc/meson.build   |   1 +
> >  hw/misc/riscv_iopmp.c | 966 ++
> >  hw/riscv/Kconfig  |   2 +
> >  hw/riscv/virt.c   |  65 ++
> >  include/hw/dma/atcdmac300.h   | 180 
> >  include/hw/misc/riscv_iopmp.h | 341 +++
> >  .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
> >  include/hw/riscv/virt.h   |  10 +-
> >  16 files changed, 2183 insertions(+), 2 deletions(-)
> >  create mode 100644 hw/dma/atcdmac300.c
> >  create mode 100644 hw/misc/riscv_iopmp.c
> >  create mode 100644 include/hw/dma/atcdmac300.h
> >  create mode 100644 include/hw/misc/riscv_iopmp.h
> >  create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h
> > 
> > -- 
> > 2.34.1
> > 

Re: [PATCH v4 0/4] Support RISC-V IOPMP

[Ethan Chen via]

Ping.
https://patchew.org/QEMU/20231122053251.440723-1-etha...@andestech.com/

On Wed, Nov 22, 2023 at 01:32:47PM +0800, Ethan Chen wrote:
> This series implements IOPMP specification v1.0.0-draft4 rapid-k model.
> The specification url:
> https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf
> 
> When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
> platform. This DMA device is connected to the IOPMP and has the 
> functionalities
> required by IOPMP, including:
> - Support setup the connection to IOPMP
> - Support asynchronous I/O to handle stall transactions
> - Send transaction information
> 
> IOPMP takes a transaction which partially match an entry as a partially hit
> error. The transaction size is depending on source device, destination device
> and bus.
> 
> Source device can send a transaction_info to IOPMP. IOPMP will check partially
> hit by transaction_info. If source device does not send a transaction_info,
> IOPMP checks information in IOMMU and dose not check partially hit.
> 
> Changes for v4:
> 
>   - Add descriptions of IOPMP and ATCDMAC300
>   - Refine coding style and comments
>   - config XILINX_AXI does not include file stream.c but selects config STREAM
> instead.
>   - ATCDMAC300: INT_STATUS is write 1 clear per bit
>   Rename iopmp_address_sink to transcation_info_sink
>   - IOPMP: Refine error message and remove unused variable
>   - VIRT: Document new options
>   atcdmac300 is only added when iopmp is enabled
>   serial setting should not be changed
> 
> Ethan Chen (4):
>   hw/core: Add config stream
>   Add RISC-V IOPMP support
>   hw/dma: Add Andes ATCDMAC300 support
>   hw/riscv/virt: Add IOPMP support
> 
>  docs/system/riscv/virt.rst|  11 +
>  hw/Kconfig|   1 +
>  hw/core/Kconfig   |   3 +
>  hw/core/meson.build   |   2 +-
>  hw/dma/Kconfig|   4 +
>  hw/dma/atcdmac300.c   | 566 ++
>  hw/dma/meson.build|   1 +
>  hw/misc/Kconfig   |   4 +
>  hw/misc/meson.build   |   1 +
>  hw/misc/riscv_iopmp.c | 966 ++
>  hw/riscv/Kconfig  |   2 +
>  hw/riscv/virt.c   |  65 ++
>  include/hw/dma/atcdmac300.h   | 180 
>  include/hw/misc/riscv_iopmp.h | 341 +++
>  .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
>  include/hw/riscv/virt.h   |  10 +-
>  16 files changed, 2183 insertions(+), 2 deletions(-)
>  create mode 100644 hw/dma/atcdmac300.c
>  create mode 100644 hw/misc/riscv_iopmp.c
>  create mode 100644 include/hw/dma/atcdmac300.h
>  create mode 100644 include/hw/misc/riscv_iopmp.h
>  create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h
> 
> -- 
> 2.34.1
> 

[PATCH v4 4/4] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

If a source device is connected to the IOPMP device, its memory access will be
checked by the IOPMP rule.

- Add 'iopmp=on' option to add an iopmp device and a dma device which is
  connected to the iopmp to machine. This option is assumed to be "off"
- Add 'iopmp_cascade=on' option to add second iopmp device which is cascaded by
  first iopmp device to machine. When iopmp option is "off", this option has no
  effect.

Signed-off-by: Ethan Chen 
---
 docs/system/riscv/virt.rst | 11 +++
 hw/riscv/Kconfig   |  2 ++
 hw/riscv/virt.c| 65 ++
 include/hw/riscv/virt.h| 10 +-
 4 files changed, 87 insertions(+), 1 deletion(-)

diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
index f5fa7b8b29..e1b4aa4f94 100644
--- a/docs/system/riscv/virt.rst
+++ b/docs/system/riscv/virt.rst
@@ -111,6 +111,17 @@ The following machine-specific options are supported:
   having AIA IMSIC (i.e. "aia=aplic-imsic" selected). When not specified,
   the default number of per-HART VS-level AIA IMSIC pages is 0.
 
+- iopmp=[on|off]
+
+  When this option is "on".  An iopmp device and a dma device which is 
connected
+  to the iopmp are added to machine. This option is assumed to be "off".
+
+- iopmp_cascade=[on|off]
+
+  When this option is "on". Second iopmp device which is cascaded by first 
iopmp
+  device is added to machine. When iopmp option is "off", this option has no
+  effect. This option is assumed to be "off".
+
 Running Linux kernel
 
 
diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index b6a5eb4452..c30a104aa4 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -45,6 +45,8 @@ config RISCV_VIRT
 select FW_CFG_DMA
 select PLATFORM_BUS
 select ACPI
+select ATCDMAC300
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index c7fc97e273..92b748bfc7 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -53,6 +53,8 @@
 #include "hw/display/ramfb.h"
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "hw/dma/atcdmac300.h"
 
 /*
  * The virt machine physical address space used by some of the devices
@@ -97,6 +99,9 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
+[VIRT_IOPMP2] =   { 0x1030,  0x10 },
+[VIRT_DMAC] = { 0x1040,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -1534,6 +1539,23 @@ static void virt_machine_init(MachineState *machine)
 sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base,
 qdev_get_gpio_in(mmio_irqchip, RTC_IRQ));
 
+if (s->have_iopmp) {
+/* IOPMP */
+DeviceState *iopmp_dev = iopmp_create(memmap[VIRT_IOPMP].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
+/* DMA with IOPMP */
+DeviceState *dmac_dev = atcdmac300_create("atcdmac300",
+memmap[VIRT_DMAC].base, memmap[VIRT_DMAC].size,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), DMAC_IRQ));
+atcdmac300_connect_iopmp(dmac_dev, &(IOPMP(iopmp_dev)->iopmp_as),
+(StreamSink *)&(IOPMP(iopmp_dev)->transaction_info_sink), 0);
+if (s->have_iopmp_cascade) {
+DeviceState *iopmp_dev2 = iopmp_create(memmap[VIRT_IOPMP2].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP2_IRQ));
+cascade_iopmp(iopmp_dev, iopmp_dev2);
+}
+}
+
 for (i = 0; i < ARRAY_SIZE(s->flash); i++) {
 /* Map legacy -drive if=pflash to machine properties */
 pflash_cfi01_legacy_drive(s->flash[i],
@@ -1628,6 +1650,35 @@ static void virt_set_aclint(Object *obj, bool value, 
Error **errp)
 s->have_aclint = value;
 }
 
+static bool virt_get_iopmp(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp;
+}
+
+static void virt_set_iopmp(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp = value;
+}
+
+static bool virt_get_iopmp_cascade(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp_cascade;
+}
+
+static void virt_set_iopmp_cascade(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp_cascade = value;
+}
+
+
 bool virt_is_acpi_enabled(RISCVVirtState *s)
 {
 return s->acpi != ON_OFF_AUTO_OFF;
@@ -1730,6 +1781,20 @@ static void virt_machine_class_init(ObjectClass *oc, 
void *data)
   NULL, NULL);
 object_class_p

[PATCH v4 1/4] hw/core: Add config stream

[Ethan Chen via]

Make other device can use /hw/core/stream.c by select this config.

Signed-off-by: Ethan Chen 
---
 hw/Kconfig  | 1 +
 hw/core/Kconfig | 3 +++
 hw/core/meson.build | 2 +-
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/Kconfig b/hw/Kconfig
index 9ca7b38c31..e4d153dce7 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -79,6 +79,7 @@ config XILINX
 config XILINX_AXI
 bool
 select PTIMER # for hw/dma/xilinx_axidma.c
+select STREAM
 
 config XLNX_ZYNQMP
 bool
diff --git a/hw/core/Kconfig b/hw/core/Kconfig
index 9397503656..e89ffa728b 100644
--- a/hw/core/Kconfig
+++ b/hw/core/Kconfig
@@ -27,3 +27,6 @@ config REGISTER
 
 config SPLIT_IRQ
 bool
+
+config STREAM
+bool
diff --git a/hw/core/meson.build b/hw/core/meson.build
index 67dad04de5..0893917b12 100644
--- a/hw/core/meson.build
+++ b/hw/core/meson.build
@@ -32,8 +32,8 @@ system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: 
files('platform-bus.c'))
 system_ss.add(when: 'CONFIG_PTIMER', if_true: files('ptimer.c'))
 system_ss.add(when: 'CONFIG_REGISTER', if_true: files('register.c'))
 system_ss.add(when: 'CONFIG_SPLIT_IRQ', if_true: files('split-irq.c'))
-system_ss.add(when: 'CONFIG_XILINX_AXI', if_true: files('stream.c'))
 system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: files('sysbus-fdt.c'))
+system_ss.add(when: 'CONFIG_STREAM', if_true: files('stream.c'))
 
 system_ss.add(files(
   'cpu-sysemu.c',
-- 
2.34.1

[PATCH v4 3/4] hw/dma: Add Andes ATCDMAC300 support

[Ethan Chen via]

ATCDMAC300 is a direct memory access controller (DMAC) which transfers data
efficiently between devices on the AMBA AXI4 bus.

ATCDMAC300 supports up to 8 DMA channels. Each DMA channel provides a set of
registers to describe the intended data transfers

To support RISC-V IOPMP, a memory access device needs to
- Support setup the connection to IOPMP
- Support asynchronous I/O to handle stall transactions
- Support transaction information (optional)

To setup the connection to IOPMP, function atcdmac300_connect_iopmp is called.
The iopmp_as and sid are needed, and transaction_info_sink is optional (null if
it is not supported).

To handle IOPMP stall transaction, this device uses asynchronous I/O by doing
memory access in bottom half coroutine. If it receives an IOPMP stall, the
coroutine yields to let the cpu execute then will retry at the bottom half
called next time. You can set the iothread property to make the device run on
iothread.

To send transaction information to IOPMP streamsink, function
transaction_info_push is called before memory access.

Signed-off-by: Ethan Chen 
---
 hw/dma/Kconfig  |   4 +
 hw/dma/atcdmac300.c | 566 
 hw/dma/meson.build  |   1 +
 include/hw/dma/atcdmac300.h | 180 
 4 files changed, 751 insertions(+)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 include/hw/dma/atcdmac300.h

diff --git a/hw/dma/Kconfig b/hw/dma/Kconfig
index 98fbb1bb04..a1d335b52f 100644
--- a/hw/dma/Kconfig
+++ b/hw/dma/Kconfig
@@ -30,3 +30,7 @@ config SIFIVE_PDMA
 config XLNX_CSU_DMA
 bool
 select REGISTER
+
+config ATCDMAC300
+bool
+select STREAM
diff --git a/hw/dma/atcdmac300.c b/hw/dma/atcdmac300.c
new file mode 100644
index 00..7db408aa54
--- /dev/null
+++ b/hw/dma/atcdmac300.c
@@ -0,0 +1,566 @@
+/*
+ * Andes ATCDMAC300 (Andes Technology DMA Controller)
+ *
+ * Copyright (c) 2022 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see 
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/dma/atcdmac300.h"
+#include "hw/irq.h"
+#include "hw/qdev-properties.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "exec/memattrs.h"
+#include "exec/address-spaces.h"
+#include "hw/stream.h"
+#include "hw/misc/riscv_iopmp_transaction_info.h"
+
+/* #define DEBUG_ANDES_ATCDMAC300 */
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_ANDES_ATCDMAC300
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+static void atcdmac300_dma_int_stat_update(ATCDMAC300State *s, int status,
+   int ch)
+{
+s->IntStatus |= (1 << (status + ch));
+}
+
+static void atcdmac300_dma_reset_chan(ATCDMAC300State *s, int ch)
+{
+if (s) {
+s->chan[ch].ChnCtrl &= ~(1 << CHAN_CTL_ENABLE);
+s->ChEN &= ~(1 << ch);
+}
+}
+
+static void atcdmac300_dma_reset(ATCDMAC300State *s)
+{
+int ch;
+for (ch = 0; ch < ATCDMAC300_MAX_CHAN; ch++) {
+atcdmac300_dma_reset_chan(s, ch);
+}
+}
+
+static uint64_t atcdmac300_read(void *opaque, hwaddr offset, unsigned size)
+{
+ATCDMAC300State *s = opaque;
+int ch = 0;
+uint64_t result = 0;
+
+if (offset >= 0x40) {
+ch = ATCDMAC300_GET_CHAN(offset);
+offset = ATCDMAC300_GET_OFF(offset, ch);
+}
+
+switch (offset) {
+case ATCDMAC300_DMA_CFG:
+result = s->DMACfg;
+break;
+case ATCDMAC300_DMAC_CTRL:
+break;
+case ATCDMAC300_CHN_ABT:
+break;
+case ATCDMAC300_INT_STATUS:
+result = s->IntStatus;
+break;
+case ATCDMAC300_CHAN_ENABLE:
+result = s->ChEN;
+break;
+case ATCDMAC300_CHAN_CTL:
+result = s->chan[ch].ChnCtrl;
+break;
+default:
+LOGGE("%s: Bad offset 0x%" HWADDR_PRIX "\n",
+  __func__, offset);
+break;
+}
+
+LOG("### atcdmac300_read()=0x%lx, val=0x%lx\n", offset, result);
+return result;
+}
+
+static void transaction_info_push(StreamSink *sink, uint8_t *buf,
+bool eop)
+{
+if (sink == NULL) {
+/* Do nothing if streamsink is not connected */
+return;
+}
+if (eop) {
+while (stream_push(sink, buf, sizeof(iopmp_transaction_info), true)
+  

[PATCH v4 2/4] Add RISC-V IOPMP support

[Ethan Chen via]

Support specification Version 1.0.0-draft4 rapid-k model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf

IOPMP check memory access from deivce is valid or not. This implementation uses
IOMMU to change address space that device access. There are three possible
results of an access: valid, blocked, and stalled.

If an access is valid, target address spcae is downstream_as(system_memory).
If an access is blocked, it will go to blocked_io_as. The operation of
blocked_io_as could be a bus error, a decode error, or it can respond a success
with fabricated data depending on IOPMP ERRREACT register value.
If an access is stalled, it will go to stall_io_as. The operation of stall_io_as
does nothing but return a stall result to source device. Source device should
retry the access if it gets a stall result.

IOPMP implementation is rely on bus signal. For example IOPMP on AXI bus checks
the AXI burst transaction. A streamsink to receive general transaction_info(sid,
start address, end address) is added to IOPMP.

If the source device support transaction_info, it can first send a
transaction_info to IOPMP streamsink then do the memory access. IOPMP will do
additional partially hit check with transaction info.
If the source device does not support transaction info. IOPMP will not check
partially hit.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |   4 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 966 ++
 include/hw/misc/riscv_iopmp.h | 341 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
 5 files changed, 1340 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index cc8a8c1418..953569e682 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -200,4 +200,8 @@ config IOSB
 config XLNX_VERSAL_TRNG
 bool
 
+config RISCV_IOPMP
+bool
+select STREAM
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index 36c20d5637..86b81e1690 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..098a3b81fd
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,966 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+#define TYPE_IOPMP_TRASACTION_INFO_SINK "iopmp_transaction_info_sink"
+
+DECLARE_INSTANCE_CHECKER(Iopmp_StreamSink, IOPMP_TRASACTION_INFO_SINK,
+ TYPE_IOPMP_TRASACTION_INFO_SINK)
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_RISCV_IOPMP
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+
+static void iopmp_decode_napot(target_ulong a, target_ulong *sa,
+   target_ulong *ea)
+{
+/*
+ * ...aaa0   8-byte NAPOT range
+ * ...aa01   16-byte NAPOT range
+ * ...a011   32-byte NAPOT range
+ * ...
+ * aa01...   2^XLEN-byte NAPOT range
+ * a011...   2^(XLEN+1)-byte NAPOT range
+ * 0111...   2^(XLEN+2)-byte NAPOT range
+ *  ...   Reserved
+ */
+
+a = (a << 2) | 0x3;
+*sa = a & (a + 1);
+*ea = a | (a + 1);
+}
+
+static vo

[PATCH v4 0/4] Support RISC-V IOPMP

[Ethan Chen via]

This series implements IOPMP specification v1.0.0-draft4 rapid-k model.
The specification url:
https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf

When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
platform. This DMA device is connected to the IOPMP and has the functionalities
required by IOPMP, including:
- Support setup the connection to IOPMP
- Support asynchronous I/O to handle stall transactions
- Send transaction information

IOPMP takes a transaction which partially match an entry as a partially hit
error. The transaction size is depending on source device, destination device
and bus.

Source device can send a transaction_info to IOPMP. IOPMP will check partially
hit by transaction_info. If source device does not send a transaction_info,
IOPMP checks information in IOMMU and dose not check partially hit.

Changes for v4:

  - Add descriptions of IOPMP and ATCDMAC300
  - Refine coding style and comments
  - config XILINX_AXI does not include file stream.c but selects config STREAM
instead.
  - ATCDMAC300: INT_STATUS is write 1 clear per bit
Rename iopmp_address_sink to transcation_info_sink
  - IOPMP: Refine error message and remove unused variable
  - VIRT: Document new options
atcdmac300 is only added when iopmp is enabled
  serial setting should not be changed

Ethan Chen (4):
  hw/core: Add config stream
  Add RISC-V IOPMP support
  hw/dma: Add Andes ATCDMAC300 support
  hw/riscv/virt: Add IOPMP support

 docs/system/riscv/virt.rst|  11 +
 hw/Kconfig|   1 +
 hw/core/Kconfig   |   3 +
 hw/core/meson.build   |   2 +-
 hw/dma/Kconfig|   4 +
 hw/dma/atcdmac300.c   | 566 ++
 hw/dma/meson.build|   1 +
 hw/misc/Kconfig   |   4 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 966 ++
 hw/riscv/Kconfig  |   2 +
 hw/riscv/virt.c   |  65 ++
 include/hw/dma/atcdmac300.h   | 180 
 include/hw/misc/riscv_iopmp.h | 341 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
 include/hw/riscv/virt.h   |  10 +-
 16 files changed, 2183 insertions(+), 2 deletions(-)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/dma/atcdmac300.h
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

-- 
2.34.1

Re: [PATCH v3 1/4] hw/core: Add config stream

[Ethan Chen via]

On Tue, Nov 21, 2023 at 03:28:13PM +1000, Alistair Francis wrote:
> On Tue, Nov 21, 2023 at 3:24 PM Alistair Francis  wrote:
> >
> > On Tue, Nov 14, 2023 at 7:49 PM Ethan Chen via  
> > wrote:
> > >
> > > Make other device can use /hw/core/stream.c by select this config.
> > >
> > > Signed-off-by: Ethan Chen 
> > > ---
> > >  hw/core/Kconfig | 3 +++
> > >  hw/core/meson.build | 1 +
> > >  2 files changed, 4 insertions(+)
> > >
> > > diff --git a/hw/core/Kconfig b/hw/core/Kconfig
> > > index 9397503656..628dc3d883 100644
> > > --- a/hw/core/Kconfig
> > > +++ b/hw/core/Kconfig
> > > @@ -27,3 +27,6 @@ config REGISTER
> > >
> > >  config SPLIT_IRQ
> > >  bool
> > > +
> > > +config STREAM
> > > +bool
> > > \ No newline at end of file
> >
> > You are missing a newline here. I think checkpatch should catch this,
> > make sure you run it on all of your patches

Sorry for that. It is wired that this was not catched by checkpatch.

> >
> > > diff --git a/hw/core/meson.build b/hw/core/meson.build
> > > index 67dad04de5..d6ce14d5ce 100644
> > > --- a/hw/core/meson.build
> > > +++ b/hw/core/meson.build
> > > @@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_REGISTER', if_true: 
> > > files('register.c'))
> > >  system_ss.add(when: 'CONFIG_SPLIT_IRQ', if_true: files('split-irq.c'))
> > >  system_ss.add(when: 'CONFIG_XILINX_AXI', if_true: files('stream.c'))
> > >  system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: 
> > > files('sysbus-fdt.c'))
> > > +system_ss.add(when: 'CONFIG_STREAM', if_true: files('stream.c'))
> >
> > You have added the build but not the file. This will fail to compile.
> >
> > Each patch must compile and run when applied individually in order.
> > That way we maintain git bisectability. Can you please make sure that
> > the build is not broken as your patches are applied
> 
> Whoops! The file already exists.
> 
> We should only include the file stream.c once. So we should change the
> CONFIG_XILINX_AXI to select CONFIG_STREAM in this patch
> 

I will fix that in next revision.

Thanks,
Ethan Chen

Re: [PATCH v3 4/4] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

On Tue, Nov 21, 2023 at 03:22:18PM +1000, Alistair Francis wrote:
> On Tue, Nov 14, 2023 at 7:48 PM Ethan Chen via  wrote:
> >
> > - Add 'iopmp=on' option to enable a iopmp device and a dma device
> >  connect to the iopmp device
> > - Add 'iopmp_cascade=on' option to enable iopmp cascading.
> 
> Can we document these in docs/system/riscv/virt.rst
> 
> Alistair

Sure. I will document these.

Thanks,
Ethan Chen

> 
> >
> > Signed-off-by: Ethan Chen 
> > ---
> >  hw/riscv/Kconfig|  2 ++
> >  hw/riscv/virt.c | 72 +++--
> >  include/hw/riscv/virt.h | 10 +-
> >  3 files changed, 81 insertions(+), 3 deletions(-)
> >
> > diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
> > index b6a5eb4452..c30a104aa4 100644
> > --- a/hw/riscv/Kconfig
> > +++ b/hw/riscv/Kconfig
> > @@ -45,6 +45,8 @@ config RISCV_VIRT
> >  select FW_CFG_DMA
> >  select PLATFORM_BUS
> >  select ACPI
> > +select ATCDMAC300
> > +select RISCV_IOPMP
> >
> >  config SHAKTI_C
> >  bool
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index c7fc97e273..3e23ee3afc 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -53,6 +53,8 @@
> >  #include "hw/display/ramfb.h"
> >  #include "hw/acpi/aml-build.h"
> >  #include "qapi/qapi-visit-common.h"
> > +#include "hw/misc/riscv_iopmp.h"
> > +#include "hw/dma/atcdmac300.h"
> >
> >  /*
> >   * The virt machine physical address space used by some of the devices
> > @@ -97,6 +99,9 @@ static const MemMapEntry virt_memmap[] = {
> >  [VIRT_UART0] ={ 0x1000, 0x100 },
> >  [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
> >  [VIRT_FW_CFG] =   { 0x1010,  0x18 },
> > +[VIRT_IOPMP] ={ 0x1020,  0x10 },
> > +[VIRT_IOPMP2] =   { 0x1030,  0x10 },
> > +[VIRT_DMAC] = { 0x1040,  0x10 },
> >  [VIRT_FLASH] ={ 0x2000, 0x400 },
> >  [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
> >  [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
> > @@ -1527,13 +1532,33 @@ static void virt_machine_init(MachineState *machine)
> >
> >  create_platform_bus(s, mmio_irqchip);
> >
> > -serial_mm_init(system_memory, memmap[VIRT_UART0].base,
> > -0, qdev_get_gpio_in(mmio_irqchip, UART0_IRQ), 399193,
> > +serial_mm_init(system_memory, memmap[VIRT_UART0].base + 0x20,
> > +0x2, qdev_get_gpio_in(mmio_irqchip, UART0_IRQ), 38400,
> >  serial_hd(0), DEVICE_LITTLE_ENDIAN);
> >
> >  sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base,
> >  qdev_get_gpio_in(mmio_irqchip, RTC_IRQ));
> >
> > +/* DMAC */
> > +DeviceState *dmac_dev = atcdmac300_create("atcdmac300",
> > +memmap[VIRT_DMAC].base, memmap[VIRT_DMAC].size,
> > +qdev_get_gpio_in(DEVICE(mmio_irqchip), DMAC_IRQ));
> > +
> > +if (s->have_iopmp) {
> > +/* IOPMP */
> > +DeviceState *iopmp_dev = iopmp_create(memmap[VIRT_IOPMP].base,
> > +qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
> > +/* DMA with IOPMP */
> > +atcdmac300_connect_iopmp(dmac_dev, &(IOPMP(iopmp_dev)->iopmp_as),
> > +(StreamSink *)&(IOPMP(iopmp_dev)->transaction_info_sink), 0);
> > +if (s->have_iopmp_cascade) {
> > +DeviceState *iopmp_dev2 = 
> > iopmp_create(memmap[VIRT_IOPMP2].base,
> > +qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP2_IRQ));
> > +cascade_iopmp(iopmp_dev, iopmp_dev2);
> > +}
> > +}
> > +
> > +
> >  for (i = 0; i < ARRAY_SIZE(s->flash); i++) {
> >  /* Map legacy -drive if=pflash to machine properties */
> >  pflash_cfi01_legacy_drive(s->flash[i],
> > @@ -1628,6 +1653,35 @@ static void virt_set_aclint(Object *obj, bool value, 
> > Error **errp)
> >  s->have_aclint = value;
> >  }
> >
> > +static bool virt_get_iopmp(Object *obj, Error **errp)
> > +{
> > +RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
> > +
> > +return s->have_iopmp;
> > +}
> > +
> > +static void virt_set_iopmp(Object *obj, bool value, Error **errp)
> > +{
> > +RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
&

Re: [PATCH v3 4/4] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

On Tue, Nov 14, 2023 at 02:50:21PM -0300, Daniel Henrique Barboza wrote:
> 
> 
> On 11/14/23 06:47, Ethan Chen wrote:
> > - Add 'iopmp=on' option to enable a iopmp device and a dma device
> >   connect to the iopmp device
> > - Add 'iopmp_cascade=on' option to enable iopmp cascading.
> > 
> > Signed-off-by: Ethan Chen 
> > ---
> >   hw/riscv/Kconfig|  2 ++
> >   hw/riscv/virt.c | 72 +++--
> >   include/hw/riscv/virt.h | 10 +-
> >   3 files changed, 81 insertions(+), 3 deletions(-)
> > 
> > diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
> > index b6a5eb4452..c30a104aa4 100644
> > --- a/hw/riscv/Kconfig
> > +++ b/hw/riscv/Kconfig
> > @@ -45,6 +45,8 @@ config RISCV_VIRT
> >   select FW_CFG_DMA
> >   select PLATFORM_BUS
> >   select ACPI
> > +select ATCDMAC300
> > +select RISCV_IOPMP
> >   config SHAKTI_C
> >   bool
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index c7fc97e273..3e23ee3afc 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -53,6 +53,8 @@
> >   #include "hw/display/ramfb.h"
> >   #include "hw/acpi/aml-build.h"
> >   #include "qapi/qapi-visit-common.h"
> > +#include "hw/misc/riscv_iopmp.h"
> > +#include "hw/dma/atcdmac300.h"
> >   /*
> >* The virt machine physical address space used by some of the devices
> > @@ -97,6 +99,9 @@ static const MemMapEntry virt_memmap[] = {
> >   [VIRT_UART0] ={ 0x1000, 0x100 },
> >   [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
> >   [VIRT_FW_CFG] =   { 0x1010,  0x18 },
> > +[VIRT_IOPMP] ={ 0x1020,  0x10 },
> > +[VIRT_IOPMP2] =   { 0x1030,  0x10 },
> > +[VIRT_DMAC] = { 0x1040,  0x10 },
> >   [VIRT_FLASH] ={ 0x2000, 0x400 },
> >   [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
> >   [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
> > @@ -1527,13 +1532,33 @@ static void virt_machine_init(MachineState *machine)
> >   create_platform_bus(s, mmio_irqchip);
> > -serial_mm_init(system_memory, memmap[VIRT_UART0].base,
> > -0, qdev_get_gpio_in(mmio_irqchip, UART0_IRQ), 399193,
> > +serial_mm_init(system_memory, memmap[VIRT_UART0].base + 0x20,
> > +0x2, qdev_get_gpio_in(mmio_irqchip, UART0_IRQ), 38400,
> >   serial_hd(0), DEVICE_LITTLE_ENDIAN);
> 
> It would be good to have some variables/macros to hold these literals like 
> '0x20'
> since they aren't self-explanatory when reading the code.

Sorry that this is my mistake. These two lines should not be modified.

> 
> >   sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base,
> >   qdev_get_gpio_in(mmio_irqchip, RTC_IRQ));
> > +/* DMAC */
> > +DeviceState *dmac_dev = atcdmac300_create("atcdmac300",
> > +memmap[VIRT_DMAC].base, memmap[VIRT_DMAC].size,
> > +qdev_get_gpio_in(DEVICE(mmio_irqchip), DMAC_IRQ));
> > +
> > +if (s->have_iopmp) {
> > +/* IOPMP */
> > +DeviceState *iopmp_dev = iopmp_create(memmap[VIRT_IOPMP].base,
> > +qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
> > +/* DMA with IOPMP */
> > +atcdmac300_connect_iopmp(dmac_dev, &(IOPMP(iopmp_dev)->iopmp_as),
> > +(StreamSink *)&(IOPMP(iopmp_dev)->transaction_info_sink), 0);
> > +if (s->have_iopmp_cascade) {
> > +DeviceState *iopmp_dev2 = 
> > iopmp_create(memmap[VIRT_IOPMP2].base,
> > +qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP2_IRQ));
> > +cascade_iopmp(iopmp_dev, iopmp_dev2);
> > +}
> > +}
> > +
> > +
> 
> Extra blank line.
> 
> Everything else LGTM. Thanks,
> 
> 
> Daniel

I will fix the issue you mentioned.

Thanks,
Ethan Chen

> 
> >   for (i = 0; i < ARRAY_SIZE(s->flash); i++) {
> >   /* Map legacy -drive if=pflash to machine properties */
> >   pflash_cfi01_legacy_drive(s->flash[i],
> > @@ -1628,6 +1653,35 @@ static void virt_set_aclint(Object *obj, bool value, 
> > Error **errp)
> >   s->have_aclint = value;
> >   }
> > +static bool virt_get_iopmp(Object *obj, Error **errp)
> > +{
> > +RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
> > +
> > +return s->have_iopmp;
> > +}
> > +
> > +static void virt_set_iopmp(Object *obj, bool value, Error **errp)
> > +{
> > +RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
> > +
> > +s->have_iopmp = value;
> > +}
> > +
> > +static bool virt_get_iopmp_cascade(Object *obj, Error **errp)
> > +{
> > +RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
> > +
> > +return s->have_iopmp_cascade;
> > +}
> > +
> > +static void virt_set_iopmp_cascade(Object *obj, bool value, Error **errp)
> > +{
> > +RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
> > +
> > +s->have_iopmp_cascade = value;
> > +}
> > +
> > +
> >   bool virt_is_acpi_enabled(RISCVVirtState *s)
> >   {
> >   return s->acpi != ON_OFF_AUTO_OFF;
> > @

[PATCH v3 2/4] Add RISC-V IOPMP support

[Ethan Chen via]

Support specification Version 1.0.0-draft4 rapid-k model.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |   4 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 967 ++
 include/hw/misc/riscv_iopmp.h | 342 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
 5 files changed, 1342 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index cc8a8c1418..953569e682 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -200,4 +200,8 @@ config IOSB
 config XLNX_VERSAL_TRNG
 bool
 
+config RISCV_IOPMP
+bool
+select STREAM
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index 36c20d5637..86b81e1690 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..99d0a554dd
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,967 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+#define TYPE_IOPMP_TRASACTION_INFO_SINK "iopmp_transaction_info_sink"
+
+DECLARE_INSTANCE_CHECKER(Iopmp_StreamSink, IOPMP_TRASACTION_INFO_SINK,
+ TYPE_IOPMP_TRASACTION_INFO_SINK)
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_RISCV_IOPMP
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+
+static void iopmp_decode_napot(target_ulong a, target_ulong *sa,
+   target_ulong *ea)
+{
+/*
+ * ...aaa0   8-byte NAPOT range
+ * ...aa01   16-byte NAPOT range
+ * ...a011   32-byte NAPOT range
+ * ...
+ * aa01...   2^XLEN-byte NAPOT range
+ * a011...   2^(XLEN+1)-byte NAPOT range
+ * 0111...   2^(XLEN+2)-byte NAPOT range
+ *  ...   Reserved
+ */
+
+a = (a << 2) | 0x3;
+*sa = a & (a + 1);
+*ea = a | (a + 1);
+}
+
+static void iopmp_update_rule(IopmpState *s, uint32_t entry_index)
+{
+uint8_t this_cfg = s->regs.entry[entry_index].cfg_reg;
+target_ulong this_addr = s->regs.entry[entry_index].addr_reg;
+target_ulong prev_addr = 0u;
+target_ulong sa = 0u;
+target_ulong ea = 0u;
+
+if (entry_index >= 1u) {
+prev_addr = s->regs.entry[entry_index - 1].addr_reg;
+}
+
+switch (iopmp_get_field(this_cfg, ENTRY_CFG_A)) {
+case IOPMP_AMATCH_OFF:
+sa = 0u;
+ea = -1;
+break;
+
+case IOPMP_AMATCH_TOR:
+sa = (prev_addr) << 2; /* shift up from [xx:0] to [xx+2:2] */
+ea = ((this_addr) << 2) - 1u;
+if (sa > ea) {
+sa = ea = 0u;
+}
+break;
+
+case IOPMP_AMATCH_NA4:
+sa = this_addr << 2; /* shift up from [xx:0] to [xx+2:2] */
+ea = (sa + 4u) - 1u;
+break;
+
+case IOPMP_AMATCH_NAPOT:
+iopmp_decode_napot(this_addr, &sa, &ea);
+break;
+
+default:
+sa = 0u;
+ea = 0u;
+break;
+}
+
+s->entry_addr[entry_index].sa = sa;
+s->entry_addr[entry_index].ea = ea;
+}
+
+static uint64_t iopmp_read(void *opaque, hwaddr addr, unsigned size)
+{
+IopmpState *s = IOPMP(opaque);
+uint32_t rz = 0;
+uint32_t is_stall = 0;
+uint32_t 

[PATCH v3 3/4] hw/dma: Add Andes ATCDMAC300 support

[Ethan Chen via]

Signed-off-by: Ethan Chen 
---
 hw/dma/Kconfig  |   4 +
 hw/dma/atcdmac300.c | 566 
 hw/dma/meson.build  |   1 +
 include/hw/dma/atcdmac300.h | 180 
 4 files changed, 751 insertions(+)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 include/hw/dma/atcdmac300.h

diff --git a/hw/dma/Kconfig b/hw/dma/Kconfig
index 98fbb1bb04..a1d335b52f 100644
--- a/hw/dma/Kconfig
+++ b/hw/dma/Kconfig
@@ -30,3 +30,7 @@ config SIFIVE_PDMA
 config XLNX_CSU_DMA
 bool
 select REGISTER
+
+config ATCDMAC300
+bool
+select STREAM
diff --git a/hw/dma/atcdmac300.c b/hw/dma/atcdmac300.c
new file mode 100644
index 00..bc6012f44e
--- /dev/null
+++ b/hw/dma/atcdmac300.c
@@ -0,0 +1,566 @@
+/*
+ * Andes ATCDMAC300 (Andes Technology DMA Controller)
+ *
+ * Copyright (c) 2022 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see 
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/dma/atcdmac300.h"
+#include "hw/irq.h"
+#include "hw/qdev-properties.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "exec/memattrs.h"
+#include "exec/address-spaces.h"
+#include "hw/stream.h"
+#include "hw/misc/riscv_iopmp_transaction_info.h"
+
+/* #define DEBUG_ANDES_ATCDMAC300 */
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_ANDES_ATCDMAC300
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+static void atcdmac300_dma_int_stat_update(ATCDMAC300State *s, int status,
+   int ch)
+{
+s->IntStatus |= (1 << (status + ch));
+}
+
+static void atcdmac300_dma_reset_chan(ATCDMAC300State *s, int ch)
+{
+if (s) {
+s->chan[ch].ChnCtrl &= ~(1 << CHAN_CTL_ENABLE);
+s->ChEN &= ~(1 << ch);
+}
+}
+
+static void atcdmac300_dma_reset(ATCDMAC300State *s)
+{
+int ch;
+for (ch = 0; ch < ATCDMAC300_MAX_CHAN; ch++) {
+atcdmac300_dma_reset_chan(s, ch);
+}
+}
+
+static uint64_t atcdmac300_read(void *opaque, hwaddr offset, unsigned size)
+{
+ATCDMAC300State *s = opaque;
+int ch = 0;
+uint64_t result = 0;
+
+if (offset >= 0x40) {
+ch = ATCDMAC300_GET_CHAN(offset);
+offset = ATCDMAC300_GET_OFF(offset, ch);
+}
+
+switch (offset) {
+case ATCDMAC300_DMA_CFG:
+result = s->DMACfg;
+break;
+case ATCDMAC300_DMAC_CTRL:
+break;
+case ATCDMAC300_CHN_ABT:
+break;
+case ATCDMAC300_INT_STATUS:
+result = s->IntStatus;
+break;
+case ATCDMAC300_CHAN_ENABLE:
+result = s->ChEN;
+break;
+case ATCDMAC300_CHAN_CTL:
+result = s->chan[ch].ChnCtrl;
+break;
+default:
+LOGGE("%s: Bad offset 0x%" HWADDR_PRIX "\n",
+  __func__, offset);
+break;
+}
+
+LOG("### atcdmac300_read()=0x%lx, val=0x%lx\n", offset, result);
+return result;
+}
+
+static void transaction_info_push(StreamSink *sink, uint8_t *buf,
+bool eop)
+{
+if (sink == NULL) {
+/* Do nothing if address stream is not support*/
+return;
+}
+if (eop) {
+while (stream_push(sink, buf, sizeof(iopmp_transaction_info), true)
+   == 0) {
+;
+}
+} else {
+while (stream_push(sink, buf, sizeof(iopmp_transaction_info), false)
+   == 0) {
+;
+}
+}
+}
+
+static MemTxResult dma_iopmp_read(ATCDMAC300State *s, hwaddr addr, void *buf,
+  hwaddr len,
+  iopmp_transaction_info *transaction)
+{
+MemTxResult result;
+if (s->iopmp_as) {
+if (s->iopmp_address_sink) {
+transaction_info_push(s->iopmp_address_sink,
+(uint8_t *)transaction, false);
+}
+MemTxAttrs dma_attrs = {.requester_id = s->sid};
+result = address_space_rw(s->iopmp_as, addr, dma_attrs,
+buf, len, false);
+if (s->iopmp_address_sink) {
+transaction_info_push(s->iopmp_address_sink,
+(uint8_t *)transaction, true);
+return result;
+}
+}
+cpu_physical_memory_read(addr, buf, len);
+return MEMTX_OK;
+}
+

[PATCH v3 0/4] Support RISC-V IOPMP

[Ethan Chen via]

This series implements IOPMP specification v1.0.0-draft4 rapid-k model.

When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
platform. This DMA devce is connected to the IOPMP and has the functionalities
required by IOPMP, including:
- Support specify source-id (SID)
- Support asynchronous I/O to handle stall transcations

IOPMP takes a transaction which partially match an entry as a partially hit
error. The transaction size is depending on source device, destination device
and bus.

Source device can send transaction information to IOPMP StreamSink. IOPMP will
check partially hit with transaction information. If source device does not send
transaction information, IOPMP will skip partially hit check.

Changes for v3:
  - Remove iopmp_start_addr and iopmp_end_addr in MemTxAttrs
  - IOPMP: Get transaction_info from StreamSink instead of MemTxAttrs
  - ATCDMAC300: Convert ATCDMAC burst to AXI burst
  - ATCDMAC300: Send transaction_info to IOPMP StreamSink

Ethan Chen (4):
  hw/core: Add config stream
  Add RISC-V IOPMP support
  hw/dma: Add Andes ATCDMAC300 support
  hw/riscv/virt: Add IOPMP support

 hw/core/Kconfig   |   3 +
 hw/core/meson.build   |   1 +
 hw/dma/Kconfig|   4 +
 hw/dma/atcdmac300.c   | 566 ++
 hw/dma/meson.build|   1 +
 hw/misc/Kconfig   |   4 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 967 ++
 hw/riscv/Kconfig  |   2 +
 hw/riscv/virt.c   |  72 +-
 include/hw/dma/atcdmac300.h   | 180 
 include/hw/misc/riscv_iopmp.h | 342 +++
 .../hw/misc/riscv_iopmp_transaction_info.h|  28 +
 include/hw/riscv/virt.h   |  10 +-
 14 files changed, 2178 insertions(+), 3 deletions(-)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/dma/atcdmac300.h
 create mode 100644 include/hw/misc/riscv_iopmp.h
 create mode 100644 include/hw/misc/riscv_iopmp_transaction_info.h

-- 
2.34.1

[PATCH v3 4/4] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

- Add 'iopmp=on' option to enable a iopmp device and a dma device
 connect to the iopmp device
- Add 'iopmp_cascade=on' option to enable iopmp cascading.

Signed-off-by: Ethan Chen 
---
 hw/riscv/Kconfig|  2 ++
 hw/riscv/virt.c | 72 +++--
 include/hw/riscv/virt.h | 10 +-
 3 files changed, 81 insertions(+), 3 deletions(-)

diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index b6a5eb4452..c30a104aa4 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -45,6 +45,8 @@ config RISCV_VIRT
 select FW_CFG_DMA
 select PLATFORM_BUS
 select ACPI
+select ATCDMAC300
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index c7fc97e273..3e23ee3afc 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -53,6 +53,8 @@
 #include "hw/display/ramfb.h"
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "hw/dma/atcdmac300.h"
 
 /*
  * The virt machine physical address space used by some of the devices
@@ -97,6 +99,9 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
+[VIRT_IOPMP2] =   { 0x1030,  0x10 },
+[VIRT_DMAC] = { 0x1040,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -1527,13 +1532,33 @@ static void virt_machine_init(MachineState *machine)
 
 create_platform_bus(s, mmio_irqchip);
 
-serial_mm_init(system_memory, memmap[VIRT_UART0].base,
-0, qdev_get_gpio_in(mmio_irqchip, UART0_IRQ), 399193,
+serial_mm_init(system_memory, memmap[VIRT_UART0].base + 0x20,
+0x2, qdev_get_gpio_in(mmio_irqchip, UART0_IRQ), 38400,
 serial_hd(0), DEVICE_LITTLE_ENDIAN);
 
 sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base,
 qdev_get_gpio_in(mmio_irqchip, RTC_IRQ));
 
+/* DMAC */
+DeviceState *dmac_dev = atcdmac300_create("atcdmac300",
+memmap[VIRT_DMAC].base, memmap[VIRT_DMAC].size,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), DMAC_IRQ));
+
+if (s->have_iopmp) {
+/* IOPMP */
+DeviceState *iopmp_dev = iopmp_create(memmap[VIRT_IOPMP].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
+/* DMA with IOPMP */
+atcdmac300_connect_iopmp(dmac_dev, &(IOPMP(iopmp_dev)->iopmp_as),
+(StreamSink *)&(IOPMP(iopmp_dev)->transaction_info_sink), 0);
+if (s->have_iopmp_cascade) {
+DeviceState *iopmp_dev2 = iopmp_create(memmap[VIRT_IOPMP2].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP2_IRQ));
+cascade_iopmp(iopmp_dev, iopmp_dev2);
+}
+}
+
+
 for (i = 0; i < ARRAY_SIZE(s->flash); i++) {
 /* Map legacy -drive if=pflash to machine properties */
 pflash_cfi01_legacy_drive(s->flash[i],
@@ -1628,6 +1653,35 @@ static void virt_set_aclint(Object *obj, bool value, 
Error **errp)
 s->have_aclint = value;
 }
 
+static bool virt_get_iopmp(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp;
+}
+
+static void virt_set_iopmp(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp = value;
+}
+
+static bool virt_get_iopmp_cascade(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp_cascade;
+}
+
+static void virt_set_iopmp_cascade(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp_cascade = value;
+}
+
+
 bool virt_is_acpi_enabled(RISCVVirtState *s)
 {
 return s->acpi != ON_OFF_AUTO_OFF;
@@ -1730,6 +1784,20 @@ static void virt_machine_class_init(ObjectClass *oc, 
void *data)
   NULL, NULL);
 object_class_property_set_description(oc, "acpi",
   "Enable ACPI");
+
+object_class_property_add_bool(oc, "iopmp", virt_get_iopmp,
+   virt_set_iopmp);
+object_class_property_set_description(oc, "iopmp",
+  "Set on/off to enable/disable "
+  "iopmp device");
+
+object_class_property_add_bool(oc, "iopmp-cascade",
+   virt_get_iopmp_cascade,
+   virt_set_iopmp_cascade);
+object_class_property_set_description(oc, "iopmp-cascade",
+  "Set on/off to enable/disable "
+  "iopmp2 device which is cascaded by "
+

[PATCH v3 1/4] hw/core: Add config stream

[Ethan Chen via]

Make other device can use /hw/core/stream.c by select this config.

Signed-off-by: Ethan Chen 
---
 hw/core/Kconfig | 3 +++
 hw/core/meson.build | 1 +
 2 files changed, 4 insertions(+)

diff --git a/hw/core/Kconfig b/hw/core/Kconfig
index 9397503656..628dc3d883 100644
--- a/hw/core/Kconfig
+++ b/hw/core/Kconfig
@@ -27,3 +27,6 @@ config REGISTER
 
 config SPLIT_IRQ
 bool
+
+config STREAM
+bool
\ No newline at end of file
diff --git a/hw/core/meson.build b/hw/core/meson.build
index 67dad04de5..d6ce14d5ce 100644
--- a/hw/core/meson.build
+++ b/hw/core/meson.build
@@ -34,6 +34,7 @@ system_ss.add(when: 'CONFIG_REGISTER', if_true: 
files('register.c'))
 system_ss.add(when: 'CONFIG_SPLIT_IRQ', if_true: files('split-irq.c'))
 system_ss.add(when: 'CONFIG_XILINX_AXI', if_true: files('stream.c'))
 system_ss.add(when: 'CONFIG_PLATFORM_BUS', if_true: files('sysbus-fdt.c'))
+system_ss.add(when: 'CONFIG_STREAM', if_true: files('stream.c'))
 
 system_ss.add(files(
   'cpu-sysemu.c',
-- 
2.34.1

Re: [PATCH v2 1/4] exec/memattrs: Add iopmp source id, start address, end address to MemTxAttrs

[Ethan Chen via]

On Tue, Nov 07, 2023 at 10:53:40AM +, Peter Maydell wrote:
> On Tue, 7 Nov 2023 at 03:02, Ethan Chen  wrote:
> >
> > On Mon, Nov 06, 2023 at 10:34:41AM +, Peter Maydell wrote:
> > > What AXI bus signals? You already get address and size in the
> > > actual memory transaction, they don't need to go in the MemTxAttrs.
> > >
> >
> > A burst contains multiple continuous read or write operations. In current
> > transaction, I can only get the size and address of a single operation. 
> > IOPMP
> > checks not only a single operation but also the burst information. I propose
> > to add those signals to MemTxAttrs.
> 
> QEMU doesn't emulate bus transactions at that level -- we have
> no concept of burst transactions. You should have the IOMMU
> do whatever it would do for a series of simple transactions.
>

I propose to use another method like StreamSink in hw/dma/xilinx_axidma.c to
let DMA send the signals to IOPMP instead of modifying IOMMU.

Thanks,
Ethan Chen

Re: [PATCH v2 1/4] exec/memattrs: Add iopmp source id, start address, end address to MemTxAttrs

[Ethan Chen via]

On Mon, Nov 06, 2023 at 10:34:41AM +, Peter Maydell wrote:
> On Mon, 6 Nov 2023 at 01:57, Ethan Chen  wrote:
> >
> > On Fri, Nov 03, 2023 at 10:34:28AM +, Peter Maydell wrote:
> > > On Fri, 3 Nov 2023 at 03:29, Ethan Chen  wrote:
> > > >
> > > > On Thu, Nov 02, 2023 at 01:53:05PM +, Peter Maydell wrote:
> > > > > On Thu, 2 Nov 2023 at 13:49, Peter Xu  wrote:
> > > > > >
> > > > > > On Thu, Nov 02, 2023 at 05:40:12PM +0800, Ethan Chen wrote:
> > > > > > > Signed-off-by: Ethan Chen 
> > > > > > > ---
> > > > > > >  include/exec/memattrs.h | 6 ++
> > > > > > >  1 file changed, 6 insertions(+)
> > > > > > >
> > > > > > > diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
> > > > > > > index d04170aa27..fc15e5d7d3 100644
> > > > > > > --- a/include/exec/memattrs.h
> > > > > > > +++ b/include/exec/memattrs.h
> > > > > > > @@ -64,6 +64,12 @@ typedef struct MemTxAttrs {
> > > > > > >  unsigned int target_tlb_bit0 : 1;
> > > > > > >  unsigned int target_tlb_bit1 : 1;
> > > > > > >  unsigned int target_tlb_bit2 : 1;
> > > > > > > +
> > > > > > > +/* IOPMP support up to 65535 sources */
> > > > > > > +unsigned int iopmp_sid:16;
> > > > > >
> > > > > > There's MemTxAttrs.requester_id, SID for pci, same length.  Reuse 
> > > > > > it?
> > > > > >
> > > > > > > +/* Transaction infomation for IOPMP */
> > > > > > > +unsigned long long iopmp_start_addr;
> > > > > > > +unsigned long long iopmp_end_addr;
> > > > > >
> > > > > > PS: encoding addresses into memattrs is.. strange, but since I know 
> > > > > > nothing
> > > > > > about iopmp, I'll leave that for other reviewers.
> > > > > >
> > > > > > Currently MemTxAttrs are passed as a whole int on the stack, if it 
> > > > > > keeps
> > > > > > growing we may start to consider a pointer, but need to check the 
> > > > > > side
> > > > > > effects of unexpected fields modified within a call.
> > > > >
> > > > > Yeah, this struct is intended to model the various attributes that
> > > > > get passed around on the bus alongside data in real hardware.
> > > > > I'm pretty sure no real hardware is passing around start and
> > > > > end transaction addresses on its bus with every read and
> > > > > write, which suggests that we should be doing this some other
> > > > > way than adding these fields to the MemTxAttrs struct.
> > > >
> > > > For AXI bus ADDR, LEN, SIZE are signals in read/write address channel.
> > > > IOPMP will check that start address = ADDR,
> > > > and end address = ADDR + LEN * SIZE.
> > >
> > > Yes, but you don't pass the start and end address on the AXI
> > > bus, so they don't go in QEMU's MemTxAttrs either.
> >
> > I will add those AXI bus signals to MemTxAttrs instead of using start and 
> > end
> > address in next revision.
> 
> What AXI bus signals? You already get address and size in the
> actual memory transaction, they don't need to go in the MemTxAttrs.
>

A burst contains multiple continuous read or write operations. In current 
transaction, I can only get the size and address of a single operation. IOPMP 
checks not only a single operation but also the burst information. I propose
to add those signals to MemTxAttrs.

Following AXI signals are needed for IOPMP.
(AW/AR)ADDR: Address of the first beat(operation) of the burst
(AW/AR)LEN: Number of beats inside the burst
(AW/AR)SIZE: Size of each beat
(AW/AR)BURST: Type of the burst

Thanks,
Ethan Chen

Re: [PATCH v2 1/4] exec/memattrs: Add iopmp source id, start address, end address to MemTxAttrs

[Ethan Chen via]

On Fri, Nov 03, 2023 at 10:34:28AM +, Peter Maydell wrote:
> On Fri, 3 Nov 2023 at 03:29, Ethan Chen  wrote:
> >
> > On Thu, Nov 02, 2023 at 01:53:05PM +, Peter Maydell wrote:
> > > On Thu, 2 Nov 2023 at 13:49, Peter Xu  wrote:
> > > >
> > > > On Thu, Nov 02, 2023 at 05:40:12PM +0800, Ethan Chen wrote:
> > > > > Signed-off-by: Ethan Chen 
> > > > > ---
> > > > >  include/exec/memattrs.h | 6 ++
> > > > >  1 file changed, 6 insertions(+)
> > > > >
> > > > > diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
> > > > > index d04170aa27..fc15e5d7d3 100644
> > > > > --- a/include/exec/memattrs.h
> > > > > +++ b/include/exec/memattrs.h
> > > > > @@ -64,6 +64,12 @@ typedef struct MemTxAttrs {
> > > > >  unsigned int target_tlb_bit0 : 1;
> > > > >  unsigned int target_tlb_bit1 : 1;
> > > > >  unsigned int target_tlb_bit2 : 1;
> > > > > +
> > > > > +/* IOPMP support up to 65535 sources */
> > > > > +unsigned int iopmp_sid:16;
> > > >
> > > > There's MemTxAttrs.requester_id, SID for pci, same length.  Reuse it?
> > > >
> > > > > +/* Transaction infomation for IOPMP */
> > > > > +unsigned long long iopmp_start_addr;
> > > > > +unsigned long long iopmp_end_addr;
> > > >
> > > > PS: encoding addresses into memattrs is.. strange, but since I know 
> > > > nothing
> > > > about iopmp, I'll leave that for other reviewers.
> > > >
> > > > Currently MemTxAttrs are passed as a whole int on the stack, if it keeps
> > > > growing we may start to consider a pointer, but need to check the side
> > > > effects of unexpected fields modified within a call.
> > >
> > > Yeah, this struct is intended to model the various attributes that
> > > get passed around on the bus alongside data in real hardware.
> > > I'm pretty sure no real hardware is passing around start and
> > > end transaction addresses on its bus with every read and
> > > write, which suggests that we should be doing this some other
> > > way than adding these fields to the MemTxAttrs struct.
> >
> > For AXI bus ADDR, LEN, SIZE are signals in read/write address channel.
> > IOPMP will check that start address = ADDR,
> > and end address = ADDR + LEN * SIZE.
> 
> Yes, but you don't pass the start and end address on the AXI
> bus, so they don't go in QEMU's MemTxAttrs either.

I will add those AXI bus signals to MemTxAttrs instead of using start and end
address in next revision.

Thanks,
Ethan Chen

Re: [PATCH v2 1/4] exec/memattrs: Add iopmp source id, start address, end address to MemTxAttrs

[Ethan Chen via]

On Thu, Nov 02, 2023 at 01:53:05PM +, Peter Maydell wrote:
> On Thu, 2 Nov 2023 at 13:49, Peter Xu  wrote:
> >
> > On Thu, Nov 02, 2023 at 05:40:12PM +0800, Ethan Chen wrote:
> > > Signed-off-by: Ethan Chen 
> > > ---
> > >  include/exec/memattrs.h | 6 ++
> > >  1 file changed, 6 insertions(+)
> > >
> > > diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
> > > index d04170aa27..fc15e5d7d3 100644
> > > --- a/include/exec/memattrs.h
> > > +++ b/include/exec/memattrs.h
> > > @@ -64,6 +64,12 @@ typedef struct MemTxAttrs {
> > >  unsigned int target_tlb_bit0 : 1;
> > >  unsigned int target_tlb_bit1 : 1;
> > >  unsigned int target_tlb_bit2 : 1;
> > > +
> > > +/* IOPMP support up to 65535 sources */
> > > +unsigned int iopmp_sid:16;
> >
> > There's MemTxAttrs.requester_id, SID for pci, same length.  Reuse it?
> >
> > > +/* Transaction infomation for IOPMP */
> > > +unsigned long long iopmp_start_addr;
> > > +unsigned long long iopmp_end_addr;
> >
> > PS: encoding addresses into memattrs is.. strange, but since I know nothing
> > about iopmp, I'll leave that for other reviewers.
> >
> > Currently MemTxAttrs are passed as a whole int on the stack, if it keeps
> > growing we may start to consider a pointer, but need to check the side
> > effects of unexpected fields modified within a call.
> 
> Yeah, this struct is intended to model the various attributes that
> get passed around on the bus alongside data in real hardware.
> I'm pretty sure no real hardware is passing around start and
> end transaction addresses on its bus with every read and
> write, which suggests that we should be doing this some other
> way than adding these fields to the MemTxAttrs struct.

For AXI bus ADDR, LEN, SIZE are signals in read/write address channel.
IOPMP will check that start address = ADDR, 
and end address = ADDR + LEN * SIZE.

Thanks,
Ethan Chen

Re: [PATCH v2 1/4] exec/memattrs: Add iopmp source id, start address, end address to MemTxAttrs

[Ethan Chen via]

On Thu, Nov 02, 2023 at 09:49:17AM -0400, Peter Xu wrote:
> On Thu, Nov 02, 2023 at 05:40:12PM +0800, Ethan Chen wrote:
> > Signed-off-by: Ethan Chen 
> > ---
> >  include/exec/memattrs.h | 6 ++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
> > index d04170aa27..fc15e5d7d3 100644
> > --- a/include/exec/memattrs.h
> > +++ b/include/exec/memattrs.h
> > @@ -64,6 +64,12 @@ typedef struct MemTxAttrs {
> >  unsigned int target_tlb_bit0 : 1;
> >  unsigned int target_tlb_bit1 : 1;
> >  unsigned int target_tlb_bit2 : 1;
> > +
> > +/* IOPMP support up to 65535 sources */
> > +unsigned int iopmp_sid:16;
> 
> There's MemTxAttrs.requester_id, SID for pci, same length.  Reuse it?

OK, I will reuse it.

> 
> > +/* Transaction infomation for IOPMP */
> > +unsigned long long iopmp_start_addr;
> > +unsigned long long iopmp_end_addr;
> 
> PS: encoding addresses into memattrs is.. strange, but since I know nothing
> about iopmp, I'll leave that for other reviewers.

Current IOMMU translate function only have address need to be translate. 
In IOPMP, It needs the transfer start and end address to check this transfer
is valid or not.

> 
> Currently MemTxAttrs are passed as a whole int on the stack, if it keeps
> growing we may start to consider a pointer, but need to check the side
> effects of unexpected fields modified within a call.

It's good for me to use a pointer.

Thanks,
Ethan Chen

[PATCH v2 1/4] exec/memattrs: Add iopmp source id, start address, end address to MemTxAttrs

[Ethan Chen via]

Signed-off-by: Ethan Chen 
---
 include/exec/memattrs.h | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
index d04170aa27..fc15e5d7d3 100644
--- a/include/exec/memattrs.h
+++ b/include/exec/memattrs.h
@@ -64,6 +64,12 @@ typedef struct MemTxAttrs {
 unsigned int target_tlb_bit0 : 1;
 unsigned int target_tlb_bit1 : 1;
 unsigned int target_tlb_bit2 : 1;
+
+/* IOPMP support up to 65535 sources */
+unsigned int iopmp_sid:16;
+/* Transaction infomation for IOPMP */
+unsigned long long iopmp_start_addr;
+unsigned long long iopmp_end_addr;
 } MemTxAttrs;
 
 /* Bus masters which don't specify any attributes will get this,
-- 
2.34.1

[PATCH v2 2/4] Add RISC-V IOPMP support

[Ethan Chen via]

Support specification Version 1.0.0-draft4 rapid-k model.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |   3 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 902 ++
 include/hw/misc/riscv_iopmp.h | 330 +
 4 files changed, 1236 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index dba41afe67..8c63eb4cef 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -197,4 +197,7 @@ config DJMEMC
 config IOSB
 bool
 
+config RISCV_IOPMP
+bool
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index f60de33f9a..7826ed9b7b 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..da9ade0d5f
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,902 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_RISCV_IOPMP
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+
+static void iopmp_decode_napot(target_ulong a, target_ulong *sa,
+   target_ulong *ea)
+{
+/*
+ * ...aaa0   8-byte NAPOT range
+ * ...aa01   16-byte NAPOT range
+ * ...a011   32-byte NAPOT range
+ * ...
+ * aa01...   2^XLEN-byte NAPOT range
+ * a011...   2^(XLEN+1)-byte NAPOT range
+ * 0111...   2^(XLEN+2)-byte NAPOT range
+ *  ...   Reserved
+ */
+
+a = (a << 2) | 0x3;
+*sa = a & (a + 1);
+*ea = a | (a + 1);
+}
+
+static void iopmp_update_rule(IopmpState *s, uint32_t entry_index)
+{
+uint8_t this_cfg = s->regs.entry[entry_index].cfg_reg;
+target_ulong this_addr = s->regs.entry[entry_index].addr_reg;
+target_ulong prev_addr = 0u;
+target_ulong sa = 0u;
+target_ulong ea = 0u;
+
+if (entry_index >= 1u) {
+prev_addr = s->regs.entry[entry_index - 1].addr_reg;
+}
+
+switch (iopmp_get_field(this_cfg, ENTRY_CFG_A)) {
+case IOPMP_AMATCH_OFF:
+sa = 0u;
+ea = -1;
+break;
+
+case IOPMP_AMATCH_TOR:
+sa = (prev_addr) << 2; /* shift up from [xx:0] to [xx+2:2] */
+ea = ((this_addr) << 2) - 1u;
+if (sa > ea) {
+sa = ea = 0u;
+}
+break;
+
+case IOPMP_AMATCH_NA4:
+sa = this_addr << 2; /* shift up from [xx:0] to [xx+2:2] */
+ea = (sa + 4u) - 1u;
+break;
+
+case IOPMP_AMATCH_NAPOT:
+iopmp_decode_napot(this_addr, &sa, &ea);
+break;
+
+default:
+sa = 0u;
+ea = 0u;
+break;
+}
+
+s->entry_addr[entry_index].sa = sa;
+s->entry_addr[entry_index].ea = ea;
+}
+
+static uint64_t
+iopmp_read(void *opaque, hwaddr addr, unsigned size)
+{
+IopmpState *s = IOPMP(opaque);
+uint32_t rz = 0;
+uint32_t is_stall = 0;
+uint32_t sid;
+switch (addr) {
+case IOPMP_VERSION ... IOPMP_USER_CFG0 + 16 * (IOPMP_MAX_ENTRY_NUM - 1):
+switch (addr) {
+case IOPMP_VERSION:
+rz = VENDER_ANDES << VERSION_VENDOR |
+ SPECVER_1_0_0_DRAFT4 << VERSION_SPECVER;
+break;
+case IOPMP_IMP:
+rz = IMPID_1_0_0_DRAFT4_0;
+break;
+case IOPMP_H

[PATCH v2 0/4] Support RISC-V IOPMP

[Ethan Chen via]

This series implements IOPMP specification v1.0.0-draft4 rapid-k model.

When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt
platform. This DMA devce is connected to the IOPMP and has the functionalities
required by IOPMP, including:
- Support specify source-id (SID)
- Support asynchronous I/O to handle stall transcations

IOPMP takes a transaction which partially match an entry as a partially hit
error. The transaction size is depending on source device, destination device
and bus.

As v1 disccussion, new iommu translate_size() function is not suitable because
the size may be modified in previos stage.

In addtion to SID, start address and end address are also added to MemTxAttrs
in v2. IOPMP matches transaction to an entry with those attributes.


Changes for v2:

  - Add iopmp_start_addr and iopmp_end_addr to MemTxAttrs.
  - Remove translate_size().
  - IOPMP: Get transaction info in attrs_to_index instead of using
translate_size().
  - IOPMP: Fix some partially hit transactions are not detected.
  - ATCDMAC300: Fix write stall is not resumed correctly.
  - ATCDMAC300: Fix some partially hit errors are not detected.

Ethan Chen (4):
  exec/memattrs: Add iopmp source id, start address, end address to
MemTxAttrs
  Add RISC-V IOPMP support
  hw/dma: Add Andes ATCDMAC300 support
  hw/riscv/virt: Add IOPMP support

 hw/dma/Kconfig|   3 +
 hw/dma/atcdmac300.c   | 460 +
 hw/dma/meson.build|   1 +
 hw/misc/Kconfig   |   3 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 902 ++
 hw/riscv/Kconfig  |   2 +
 hw/riscv/virt.c   |  68 +++
 include/exec/memattrs.h   |   6 +
 include/hw/dma/atcdmac300.h   | 171 +++
 include/hw/misc/riscv_iopmp.h | 330 +
 include/hw/riscv/virt.h   |  10 +-
 12 files changed, 1956 insertions(+), 1 deletion(-)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/dma/atcdmac300.h
 create mode 100644 include/hw/misc/riscv_iopmp.h

-- 
2.34.1

[PATCH v2 3/4] hw/dma: Add Andes ATCDMAC300 support

[Ethan Chen via]

Signed-off-by: Ethan Chen 
---
 hw/dma/Kconfig  |   3 +
 hw/dma/atcdmac300.c | 460 
 hw/dma/meson.build  |   1 +
 include/hw/dma/atcdmac300.h | 171 ++
 4 files changed, 635 insertions(+)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 include/hw/dma/atcdmac300.h

diff --git a/hw/dma/Kconfig b/hw/dma/Kconfig
index 98fbb1bb04..b3f4edc873 100644
--- a/hw/dma/Kconfig
+++ b/hw/dma/Kconfig
@@ -30,3 +30,6 @@ config SIFIVE_PDMA
 config XLNX_CSU_DMA
 bool
 select REGISTER
+
+config ATCDMAC300
+bool
diff --git a/hw/dma/atcdmac300.c b/hw/dma/atcdmac300.c
new file mode 100644
index 00..bfa8d179eb
--- /dev/null
+++ b/hw/dma/atcdmac300.c
@@ -0,0 +1,460 @@
+/*
+ * Andes ATCDMAC300 (Andes Technology DMA Controller)
+ *
+ * Copyright (c) 2022 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see 
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/dma/atcdmac300.h"
+#include "hw/irq.h"
+#include "hw/qdev-properties.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "exec/memattrs.h"
+#include "exec/address-spaces.h"
+
+/* #define DEBUG_ANDES_ATCDMAC300 */
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_ANDES_ATCDMAC300
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+static void atcdmac300_dma_int_stat_update(ATCDMAC300State *s, int status,
+   int ch)
+{
+s->IntStatus |= (1 << (status + ch));
+}
+
+static void atcdmac300_dma_reset_chan(ATCDMAC300State *s, int ch)
+{
+if (s) {
+s->chan[ch].ChnCtrl &= ~(1 << CHAN_CTL_ENABLE);
+s->ChEN &= ~(1 << ch);
+}
+}
+
+static void atcdmac300_dma_reset(ATCDMAC300State *s)
+{
+int ch;
+for (ch = 0; ch < ATCDMAC300_MAX_CHAN; ch++) {
+atcdmac300_dma_reset_chan(s, ch);
+}
+}
+
+static uint64_t atcdmac300_read(void *opaque, hwaddr offset, unsigned size)
+{
+ATCDMAC300State *s = opaque;
+int ch = 0;
+uint64_t result = 0;
+
+if (offset >= 0x40) {
+ch = ATCDMAC300_GET_CHAN(offset);
+offset = ATCDMAC300_GET_OFF(offset, ch);
+}
+
+switch (offset) {
+case ATCDMAC300_DMA_CFG:
+result = s->DMACfg;
+break;
+case ATCDMAC300_DMAC_CTRL:
+break;
+case ATCDMAC300_CHN_ABT:
+break;
+case ATCDMAC300_INT_STATUS:
+result = s->IntStatus;
+break;
+case ATCDMAC300_CHAN_ENABLE:
+result = s->ChEN;
+break;
+case ATCDMAC300_CHAN_CTL:
+result = s->chan[ch].ChnCtrl;
+break;
+default:
+LOGGE("%s: Bad offset 0x%" HWADDR_PRIX "\n",
+  __func__, offset);
+break;
+}
+
+LOG("### atcdmac300_read()=0x%lx, val=0x%lx\n", offset, result);
+return result;
+}
+
+static MemTxResult dma_iopmp_read(ATCDMAC300State *s, hwaddr addr, void *buf,
+  hwaddr len)
+{
+if (s->iopmp_as) {
+MemTxAttrs dma_attrs = {.iopmp_sid = s->sid,
+.iopmp_start_addr = addr,
+.iopmp_end_addr = addr + len - 1
+   };
+return address_space_rw(s->iopmp_as, addr, dma_attrs,
+buf, len, false);
+}
+cpu_physical_memory_read(addr, buf, len);
+return MEMTX_OK;
+}
+
+static MemTxResult dma_iopmp_write(ATCDMAC300State *s, hwaddr addr, void *buf,
+   hwaddr len)
+{
+if (s->iopmp_as) {
+MemTxAttrs dma_attrs = {.iopmp_sid = s->sid,
+.iopmp_start_addr = addr,
+.iopmp_end_addr = addr + len - 1,
+   };
+return address_space_rw(s->iopmp_as, addr, dma_attrs,
+buf, len, true);
+}
+cpu_physical_memory_write(addr, buf, len);
+return MEMTX_OK;
+}
+
+static void atcdmac300_co_run(void *opaque)
+{
+ATCDMAC300State *s = opaque;
+int64_t remain_size, burst_remain_size, copy_size, copy_size_dst;
+uint64_t src_addr, dst_addr, src_width, dst_width, burst_size,
+ src_addr_ctl, dst_addr_ctl, int_tc_mask, int_err_mask,
+ int_abort_mask;
+uint64_t buf[ATCDMAC300_MAX_BURST

[PATCH v2 4/4] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

- Add 'iopmp=on' option to enable a iopmp device and a dma device
 connect to the iopmp device
- Add 'iopmp_cascade=on' option to enable iopmp cascading.

Signed-off-by: Ethan Chen 
---
 hw/riscv/Kconfig|  2 ++
 hw/riscv/virt.c | 68 +
 include/hw/riscv/virt.h | 10 +-
 3 files changed, 79 insertions(+), 1 deletion(-)

diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index b6a5eb4452..c30a104aa4 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -45,6 +45,8 @@ config RISCV_VIRT
 select FW_CFG_DMA
 select PLATFORM_BUS
 select ACPI
+select ATCDMAC300
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 9de578c756..320aed3b29 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -53,6 +53,9 @@
 #include "hw/display/ramfb.h"
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "hw/dma/atcdmac300.h"
+
 
 /*
  * The virt machine physical address space used by some of the devices
@@ -97,6 +100,9 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
+[VIRT_IOPMP2] =   { 0x1030,  0x10 },
+[VIRT_DMAC] = { 0x1040,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -1534,6 +1540,25 @@ static void virt_machine_init(MachineState *machine)
 sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base,
 qdev_get_gpio_in(mmio_irqchip, RTC_IRQ));
 
+/* DMAC */
+DeviceState *dmac_dev = atcdmac300_create("atcdmac300",
+memmap[VIRT_DMAC].base, memmap[VIRT_DMAC].size,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), DMAC_IRQ));
+
+if (s->have_iopmp) {
+/* IOPMP */
+DeviceState *iopmp_dev = iopmp_create(memmap[VIRT_IOPMP].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
+/* DMA with IOPMP */
+atcdmac300_connect_iopmp_as(dmac_dev, &(IOPMP(iopmp_dev)->iopmp_as), 
0);
+if (s->have_iopmp_cascade) {
+DeviceState *iopmp_dev2 = iopmp_create(memmap[VIRT_IOPMP2].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP2_IRQ));
+cascade_iopmp(iopmp_dev, iopmp_dev2);
+}
+}
+
+
 for (i = 0; i < ARRAY_SIZE(s->flash); i++) {
 /* Map legacy -drive if=pflash to machine properties */
 pflash_cfi01_legacy_drive(s->flash[i],
@@ -1628,6 +1653,35 @@ static void virt_set_aclint(Object *obj, bool value, 
Error **errp)
 s->have_aclint = value;
 }
 
+static bool virt_get_iopmp(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp;
+}
+
+static void virt_set_iopmp(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp = value;
+}
+
+static bool virt_get_iopmp_cascade(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp_cascade;
+}
+
+static void virt_set_iopmp_cascade(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp_cascade = value;
+}
+
+
 bool virt_is_acpi_enabled(RISCVVirtState *s)
 {
 return s->acpi != ON_OFF_AUTO_OFF;
@@ -1730,6 +1784,20 @@ static void virt_machine_class_init(ObjectClass *oc, 
void *data)
   NULL, NULL);
 object_class_property_set_description(oc, "acpi",
   "Enable ACPI");
+
+object_class_property_add_bool(oc, "iopmp", virt_get_iopmp,
+   virt_set_iopmp);
+object_class_property_set_description(oc, "iopmp",
+  "Set on/off to enable/disable "
+  "iopmp device");
+
+object_class_property_add_bool(oc, "iopmp-cascade",
+   virt_get_iopmp_cascade,
+   virt_set_iopmp_cascade);
+object_class_property_set_description(oc, "iopmp-cascade",
+  "Set on/off to enable/disable "
+  "iopmp2 device which is cascaded by "
+  "iopmp1 device");
 }
 
 static const TypeInfo virt_machine_typeinfo = {
diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h
index e5c474b26e..5fa2944d29 100644
--- a/include/hw/riscv/virt.h
+++ b/include/hw/riscv/virt.h
@@ -54,6 +54,8 @@ struct RISCVVirtState {
 
 int fdt_size;
 bool have_aclint;
+bool have_iopmp;
+bool have_iopmp_cascade;

Re: [PATCH 2/6] system/physmem: IOMMU: Invoke the translate_size function if it is implemented

[Ethan Chen via]

On Mon, Oct 30, 2023 at 11:02:30AM -0400, Peter Xu wrote:
> On Mon, Oct 30, 2023 at 02:00:54PM +0800, Ethan Chen wrote:
> > On Fri, Oct 27, 2023 at 12:13:50PM -0400, Peter Xu wrote:
> > > Add cc list.
> > > 
> > > On Fri, Oct 27, 2023 at 12:02:24PM -0400, Peter Xu wrote:
> > > > On Fri, Oct 27, 2023 at 11:28:36AM +0800, Ethan Chen wrote:
> > > > > On Thu, Oct 26, 2023 at 10:20:41AM -0400, Peter Xu wrote:
> > > > > > Could you elaborate why is that important?  In what use case?
> > > > > I was not involved in the formulation of the IOPMP specification, but 
> > > > > I'll try
> > > > > to explain my perspective. IOPMP use the same the idea as PMP. "The 
> > > > > matching 
> > > > > PMP entry must match all bytes of an access, or the access fails."
> > > > > 
> > > > > > 
> > > > > > Consider IOVA mapped for address range iova=[0, 4K] only, here we 
> > > > > > have a
> > > > > > DMA request with range=[0, 8K].  Now my understanding is what you 
> > > > > > want to
> > > > > > achieve is don't trigger the DMA to [0, 4K] and fail the whole [0, 
> > > > > > 8K]
> > > > > > request.
> > > > > > 
> > > > > > Can we just fail at the latter DMA [4K, 8K] when it happens?  After 
> > > > > > all,
> > > > > > IIUC a device can split the 0-8K DMA into two smaller DMAs, then 
> > > > > > the 1st
> > > > > > chunk can succeed then if it falls in 0-4K.  Some further 
> > > > > > explanation of
> > > > > > the failure use case could be helpful.
> > > > > 
> > > > > IOPMP can only detect partially hit in an access. DMA device will 
> > > > > split a 
> > > > > large DMA transfer to small DMA transfers base on target and DMA 
> > > > > transfer 
> > > > > width, so partially hit error only happens when an access cross the 
> > > > > boundary.
> > > > > But to ensure that an access is only within one entry is still 
> > > > > important. 
> > > > > For example, an entry may mean permission of a device memory region. 
> > > > > We do 
> > > > > not want to see one DMA transfer can access mutilple devices, 
> > > > > although DMA 
> > > > > have permissions from multiple entries.
> > > > 
> > > > I was expecting a DMA request can be fulfilled successfully as long as 
> > > > the
> > > > DMA translations are valid for the whole range of the request, even if 
> > > > the
> > > > requested range may include two separate translated targets or more, 
> > > > each
> > > > point to different places (either RAM, or other devicie's MMIO regions).
> > 
> > IOPMP is used to check DMA translation is vaild or not. In IOPMP 
> > specification
> > , a translation access more than one entry is not invalid.
> > Though it is not recommand, user can create an IOPMP entry contains mutiple
> > places to make this kind translations valid.
> > 
> > > > 
> > > > AFAIK currently QEMU memory model will automatically split that large
> > > > request into two or more smaller requests, and fulfill them separately 
> > > > by
> > > > two/more IOMMU translations, with its memory access dispatched to the
> > > > specific memory regions.
> > 
> > Because of requests may be split, I need a method to take the original 
> > request
> > information to IOPMP.
> 
> I'm not sure whether translate() is the "original request" either.  The
> problem is QEMU can split the request for various reasons already, afaict.
> 
> For example, address_space_translate_internal() has this:
> 
> if (memory_region_is_ram(mr)) {
> diff = int128_sub(section->size, int128_make64(addr));
> *plen = int128_get64(int128_min(diff, int128_make64(*plen)));
> }
> 
> Which can already shrink the request size from the caller before reaching
> translate().  So the length passed into translate() can already be
> modified.
> 
> Another thing is, we have two other common call sites for translate():
> 
> memory_region_iommu_replay
> address_space_translate_for_iotlb
> 
> I'm not sure whether you've looked into them and think they don't need to
> be trapped: at least memory_region_iommu_replay() looks all fine in this
> regard because it always translate in min page size granule.  But I think
> the restriction should apply to all translate()s.
> 
> translate_size() is weird on its own. If the only purpose is to pass the
> length into translate(), another option is to add that parameter into
> current translate(), allowing the implementation to ignore it.  I think
> that'll be better, but even if so, I'm not 100% sure it'll always do what
> you wanted as discussed above.

It seems that there are too many things that have not been considered in my 
current method. I am doing the revision that no new translation function but 
adding start address and end address to MemTxAttrs. 

Since attrs_to_index() only return one interger, IOPMP attrs_to_index() will 
copy the address range to its device state and then handle the translate(). 

Thanks,
Ethan Chen

Re: [PATCH 2/6] system/physmem: IOMMU: Invoke the translate_size function if it is implemented

[Ethan Chen via]

On Fri, Oct 27, 2023 at 12:13:50PM -0400, Peter Xu wrote:
> Add cc list.
> 
> On Fri, Oct 27, 2023 at 12:02:24PM -0400, Peter Xu wrote:
> > On Fri, Oct 27, 2023 at 11:28:36AM +0800, Ethan Chen wrote:
> > > On Thu, Oct 26, 2023 at 10:20:41AM -0400, Peter Xu wrote:
> > > > Could you elaborate why is that important?  In what use case?
> > > I was not involved in the formulation of the IOPMP specification, but 
> > > I'll try
> > > to explain my perspective. IOPMP use the same the idea as PMP. "The 
> > > matching 
> > > PMP entry must match all bytes of an access, or the access fails."
> > > 
> > > > 
> > > > Consider IOVA mapped for address range iova=[0, 4K] only, here we have a
> > > > DMA request with range=[0, 8K].  Now my understanding is what you want 
> > > > to
> > > > achieve is don't trigger the DMA to [0, 4K] and fail the whole [0, 8K]
> > > > request.
> > > > 
> > > > Can we just fail at the latter DMA [4K, 8K] when it happens?  After all,
> > > > IIUC a device can split the 0-8K DMA into two smaller DMAs, then the 1st
> > > > chunk can succeed then if it falls in 0-4K.  Some further explanation of
> > > > the failure use case could be helpful.
> > > 
> > > IOPMP can only detect partially hit in an access. DMA device will split a 
> > > large DMA transfer to small DMA transfers base on target and DMA transfer 
> > > width, so partially hit error only happens when an access cross the 
> > > boundary.
> > > But to ensure that an access is only within one entry is still important. 
> > > For example, an entry may mean permission of a device memory region. We 
> > > do 
> > > not want to see one DMA transfer can access mutilple devices, although 
> > > DMA 
> > > have permissions from multiple entries.
> > 
> > I was expecting a DMA request can be fulfilled successfully as long as the
> > DMA translations are valid for the whole range of the request, even if the
> > requested range may include two separate translated targets or more, each
> > point to different places (either RAM, or other devicie's MMIO regions).

IOPMP is used to check DMA translation is vaild or not. In IOPMP specification
, a translation access more than one entry is not invalid.
Though it is not recommand, user can create an IOPMP entry contains mutiple
places to make this kind translations valid.

> > 
> > AFAIK currently QEMU memory model will automatically split that large
> > request into two or more smaller requests, and fulfill them separately by
> > two/more IOMMU translations, with its memory access dispatched to the
> > specific memory regions.

Because of requests may be split, I need a method to take the original request
information to IOPMP.

> > 
> > The example you provided doesn't seem to be RISCV specific.  Do you mean it
> > is a generic requirement from PCI/PCIe POV, or is it only a restriction of
> > IOPMP?  If it's a valid PCI restriction, does it mean that all the rest
> > IOMMU implementations in QEMU currently are broken?
> > 

It only a restriction of IOPMP.

Thanks,
Ethan Chen

Re: [PATCH 2/6] system/physmem: IOMMU: Invoke the translate_size function if it is implemented

[Ethan Chen via]

On Thu, Oct 26, 2023 at 10:20:41AM -0400, Peter Xu wrote:
> Could you elaborate why is that important?  In what use case?
I was not involved in the formulation of the IOPMP specification, but I'll try
to explain my perspective. IOPMP use the same the idea as PMP. "The matching 
PMP entry must match all bytes of an access, or the access fails."

> 
> Consider IOVA mapped for address range iova=[0, 4K] only, here we have a
> DMA request with range=[0, 8K].  Now my understanding is what you want to
> achieve is don't trigger the DMA to [0, 4K] and fail the whole [0, 8K]
> request.
> 
> Can we just fail at the latter DMA [4K, 8K] when it happens?  After all,
> IIUC a device can split the 0-8K DMA into two smaller DMAs, then the 1st
> chunk can succeed then if it falls in 0-4K.  Some further explanation of
> the failure use case could be helpful.

IOPMP can only detect partially hit in an access. DMA device will split a 
large DMA transfer to small DMA transfers base on target and DMA transfer 
width, so partially hit error only happens when an access cross the boundary.
But to ensure that an access is only within one entry is still important. 
For example, an entry may mean permission of a device memory region. We do 
not want to see one DMA transfer can access mutilple devices, although DMA 
have permissions from multiple entries.

Thanks,
Ethan Chen

Re: [PATCH 0/6] Support RISC-V IOPMP

[Ethan Chen via]

I found that after add size information it is still not enough for IOPMP to 
reject partially hit error. Access is separated in flatview_read_continue and
lost the start address information. I will fix it in next version. 

Address start, address end will be added to MemTxAttr, translate_size will 
beremoved and translate_attr which is have full MemTxAttr information will be 
added in next version patch.

Re: [PATCH 1/6] exec/memory: Introduce the translate_size function within the IOMMU class

[Ethan Chen via]

On Wed, Oct 25, 2023 at 04:56:22PM +0200, David Hildenbrand wrote:
> On 25.10.23 07:14, Ethan Chen wrote:
> > IOMMU have size information during translation.
> > 
> 
> Can you add some more information why we would want this and how the backend
> can do "better" things with the size at hand?
>
With size information, IOMMU can reject a memory access which is patially in 
valid region.

Currently translation function limit memory access size with a mask, so the 
valid part of access will success. My target is to detect partially hit and 
reject whole access. Translation function cannot detect partially hit because 
it lacks size information.
> Note that I was not CCed on the cover letter.
> 
> > Signed-off-by: Ethan Chen 
> > ---
> >   include/exec/memory.h | 19 +++
> >   1 file changed, 19 insertions(+)
> > 
> > diff --git a/include/exec/memory.h b/include/exec/memory.h
> > index 9087d02769..5520b7c8c0 100644
> > --- a/include/exec/memory.h
> > +++ b/include/exec/memory.h
> > @@ -396,6 +396,25 @@ struct IOMMUMemoryRegionClass {
> >*/
> >   IOMMUTLBEntry (*translate)(IOMMUMemoryRegion *iommu, hwaddr addr,
> >  IOMMUAccessFlags flag, int iommu_idx);
> > +/**
> > + * @translate_size:
> > + *
> > + * Return a TLB entry that contains a given address and size.
> > + *
> > + * @iommu: the IOMMUMemoryRegion
> > + *
> > + * @hwaddr: address to be translated within the memory region
> > + *
> > + * @size: size to indicate the scope of the entire transaction
> > + *
> > + * @flag: requested access permission
> > + *
> > + * @iommu_idx: IOMMU index for the translation
> > + */
> > +IOMMUTLBEntry (*translate_size)(IOMMUMemoryRegion *iommu, hwaddr addr,
> > +hwaddr size, IOMMUAccessFlags flag,
> > +int iommu_idx);
> > +
> >   /**
> >* @get_min_page_size:
> >*
> 
> -- 
> Cheers,
> 
> David / dhildenb
> 
Thanks,
Ethan Chen

Re: [PATCH 2/6] system/physmem: IOMMU: Invoke the translate_size function if it is implemented

[Ethan Chen via]

On Wed, Oct 25, 2023 at 11:14:42AM -0400, Peter Xu wrote:
> On Wed, Oct 25, 2023 at 01:14:26PM +0800, Ethan Chen wrote:
> > Signed-off-by: Ethan Chen 
> > ---
> >  system/physmem.c | 9 +++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> > 
> > diff --git a/system/physmem.c b/system/physmem.c
> > index fc2b0fee01..53b6ab735c 100644
> > --- a/system/physmem.c
> > +++ b/system/physmem.c
> > @@ -432,8 +432,13 @@ static MemoryRegionSection 
> > address_space_translate_iommu(IOMMUMemoryRegion *iomm
> >  iommu_idx = imrc->attrs_to_index(iommu_mr, attrs);
> >  }
> >  
> > -iotlb = imrc->translate(iommu_mr, addr, is_write ?
> > -IOMMU_WO : IOMMU_RO, iommu_idx);
> > +if (imrc->translate_size) {
> > +iotlb = imrc->translate_size(iommu_mr, addr, *plen_out, 
> > is_write ?
> > + IOMMU_WO : IOMMU_RO, iommu_idx);
> > +} else {
> > +iotlb = imrc->translate(iommu_mr, addr, is_write ?
> > +IOMMU_WO : IOMMU_RO, iommu_idx);
> > +}
> 
> Currently the translation size is encoded in iotlb.addr_mask.  Can riscv do
> the same?
Riscv do the same, so translation size may be reduced by iotlb.addr_mask.
>
> For example, lookup addr in match_entry_md() ranges, report size back into
> iotlb.addr_mask, rather than enforcing *plen_out range always resides in
> one translation only.
>
> IMHO it's actually legal if *plen_out covers more than one IOMMU
> translations.  QEMU memory core should have taken care of that by
> separately translate the ranges and apply RW on top.  With current proposal
> of translate_size() I think it'll fail instead, which is not wanted.
>
My target is to support IOPMP partially hit error. IOPMP checks whole memory 
access region is in the same entry. If not, reject the access instead of modify
the access size.

Because most of IOPMP permisson checking features can be implemented by 
current IOMMU class, so I add this function in IOMMU class. There may be 
other more suitable ways to support partially hit error.
> Thanks,
> 
> -- 
> Peter Xu
> 
Thanks,
Ethan Chen

[PATCH 6/6] hw/riscv/virt: Add IOPMP support

[Ethan Chen via]

- Add 'iopmp=on' option to enable iopmp
- Add 'iopmp_cascade=on' option to enable iopmp cascading.

Signed-off-by: Ethan Chen 
---
 hw/riscv/Kconfig|  2 ++
 hw/riscv/virt.c | 72 +++--
 include/hw/riscv/virt.h | 10 +-
 3 files changed, 81 insertions(+), 3 deletions(-)

diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index b6a5eb4452..c30a104aa4 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -45,6 +45,8 @@ config RISCV_VIRT
 select FW_CFG_DMA
 select PLATFORM_BUS
 select ACPI
+select ATCDMAC300
+select RISCV_IOPMP
 
 config SHAKTI_C
 bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 9de578c756..d3d4320c73 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -53,6 +53,9 @@
 #include "hw/display/ramfb.h"
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "hw/dma/atcdmac300.h"
+
 
 /*
  * The virt machine physical address space used by some of the devices
@@ -97,6 +100,9 @@ static const MemMapEntry virt_memmap[] = {
 [VIRT_UART0] ={ 0x1000, 0x100 },
 [VIRT_VIRTIO] =   { 0x10001000,0x1000 },
 [VIRT_FW_CFG] =   { 0x1010,  0x18 },
+[VIRT_IOPMP] ={ 0x1020,  0x10 },
+[VIRT_IOPMP2] =   { 0x1030,  0x10 },
+[VIRT_DMAC] = { 0x1040,  0x10 },
 [VIRT_FLASH] ={ 0x2000, 0x400 },
 [VIRT_IMSIC_M] =  { 0x2400, VIRT_IMSIC_MAX_SIZE },
 [VIRT_IMSIC_S] =  { 0x2800, VIRT_IMSIC_MAX_SIZE },
@@ -1527,13 +1533,32 @@ static void virt_machine_init(MachineState *machine)
 
 create_platform_bus(s, mmio_irqchip);
 
-serial_mm_init(system_memory, memmap[VIRT_UART0].base,
-0, qdev_get_gpio_in(mmio_irqchip, UART0_IRQ), 399193,
+serial_mm_init(system_memory, memmap[VIRT_UART0].base + 0x20,
+0x2, qdev_get_gpio_in(DEVICE(mmio_irqchip), UART0_IRQ), 38400,
 serial_hd(0), DEVICE_LITTLE_ENDIAN);
 
 sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base,
 qdev_get_gpio_in(mmio_irqchip, RTC_IRQ));
 
+/* DMAC */
+DeviceState *dmac_dev = atcdmac300_create("atcdmac300",
+memmap[VIRT_DMAC].base, memmap[VIRT_DMAC].size,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), DMAC_IRQ));
+
+if (s->have_iopmp) {
+/* IOPMP */
+DeviceState *iopmp_dev = iopmp_create(memmap[VIRT_IOPMP].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP_IRQ));
+/* DMA with IOPMP */
+atcdmac300_connect_iopmp_as(dmac_dev, &(IOPMP(iopmp_dev)->iopmp_as), 
0);
+if (s->have_iopmp_cascade) {
+DeviceState *iopmp_dev2 = iopmp_create(memmap[VIRT_IOPMP2].base,
+qdev_get_gpio_in(DEVICE(mmio_irqchip), IOPMP2_IRQ));
+cascade_iopmp(iopmp_dev, iopmp_dev2);
+}
+}
+
+
 for (i = 0; i < ARRAY_SIZE(s->flash); i++) {
 /* Map legacy -drive if=pflash to machine properties */
 pflash_cfi01_legacy_drive(s->flash[i],
@@ -1628,6 +1653,35 @@ static void virt_set_aclint(Object *obj, bool value, 
Error **errp)
 s->have_aclint = value;
 }
 
+static bool virt_get_iopmp(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp;
+}
+
+static void virt_set_iopmp(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp = value;
+}
+
+static bool virt_get_iopmp_cascade(Object *obj, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+return s->have_iopmp_cascade;
+}
+
+static void virt_set_iopmp_cascade(Object *obj, bool value, Error **errp)
+{
+RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+s->have_iopmp_cascade = value;
+}
+
+
 bool virt_is_acpi_enabled(RISCVVirtState *s)
 {
 return s->acpi != ON_OFF_AUTO_OFF;
@@ -1730,6 +1784,20 @@ static void virt_machine_class_init(ObjectClass *oc, 
void *data)
   NULL, NULL);
 object_class_property_set_description(oc, "acpi",
   "Enable ACPI");
+
+object_class_property_add_bool(oc, "iopmp", virt_get_iopmp,
+   virt_set_iopmp);
+object_class_property_set_description(oc, "iopmp",
+  "Set on/off to enable/disable "
+  "iopmp device");
+
+object_class_property_add_bool(oc, "iopmp-cascade",
+   virt_get_iopmp_cascade,
+   virt_set_iopmp_cascade);
+object_class_property_set_description(oc, "iopmp-cascade",
+  "Set on/off to enable/disable "
+  "iopmp2 device which is cascaded by "
+  "iopmp1 device");
 }
 
 static const TypeInfo virt_machine_typeinfo =

[PATCH 3/6] exec/memattrs: Add iopmp source id to MemTxAttrs

[Ethan Chen via]

Signed-off-by: Ethan Chen 
---
 include/exec/memattrs.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
index d04170aa27..e27b4fab00 100644
--- a/include/exec/memattrs.h
+++ b/include/exec/memattrs.h
@@ -64,6 +64,9 @@ typedef struct MemTxAttrs {
 unsigned int target_tlb_bit0 : 1;
 unsigned int target_tlb_bit1 : 1;
 unsigned int target_tlb_bit2 : 1;
+
+/* IOPMP support up to 65535 sources */
+unsigned int iopmp_sid:16;
 } MemTxAttrs;
 
 /* Bus masters which don't specify any attributes will get this,
-- 
2.34.1

[PATCH 2/6] system/physmem: IOMMU: Invoke the translate_size function if it is implemented

[Ethan Chen via]

Signed-off-by: Ethan Chen 
---
 system/physmem.c | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/system/physmem.c b/system/physmem.c
index fc2b0fee01..53b6ab735c 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -432,8 +432,13 @@ static MemoryRegionSection 
address_space_translate_iommu(IOMMUMemoryRegion *iomm
 iommu_idx = imrc->attrs_to_index(iommu_mr, attrs);
 }
 
-iotlb = imrc->translate(iommu_mr, addr, is_write ?
-IOMMU_WO : IOMMU_RO, iommu_idx);
+if (imrc->translate_size) {
+iotlb = imrc->translate_size(iommu_mr, addr, *plen_out, is_write ?
+ IOMMU_WO : IOMMU_RO, iommu_idx);
+} else {
+iotlb = imrc->translate(iommu_mr, addr, is_write ?
+IOMMU_WO : IOMMU_RO, iommu_idx);
+}
 
 if (!(iotlb.perm & (1 << is_write))) {
 goto unassigned;
-- 
2.34.1

[PATCH 1/6] exec/memory: Introduce the translate_size function within the IOMMU class

[Ethan Chen via]

IOMMU have size information during translation.

Signed-off-by: Ethan Chen 
---
 include/exec/memory.h | 19 +++
 1 file changed, 19 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 9087d02769..5520b7c8c0 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -396,6 +396,25 @@ struct IOMMUMemoryRegionClass {
  */
 IOMMUTLBEntry (*translate)(IOMMUMemoryRegion *iommu, hwaddr addr,
IOMMUAccessFlags flag, int iommu_idx);
+/**
+ * @translate_size:
+ *
+ * Return a TLB entry that contains a given address and size.
+ *
+ * @iommu: the IOMMUMemoryRegion
+ *
+ * @hwaddr: address to be translated within the memory region
+ *
+ * @size: size to indicate the scope of the entire transaction
+ *
+ * @flag: requested access permission
+ *
+ * @iommu_idx: IOMMU index for the translation
+ */
+IOMMUTLBEntry (*translate_size)(IOMMUMemoryRegion *iommu, hwaddr addr,
+hwaddr size, IOMMUAccessFlags flag,
+int iommu_idx);
+
 /**
  * @get_min_page_size:
  *
-- 
2.34.1

[PATCH 5/6] hw/dma: Add Andes ATCDMAC300 support

[Ethan Chen via]

Signed-off-by: Ethan Chen 
---
 hw/dma/Kconfig  |   3 +
 hw/dma/atcdmac300.c | 435 
 hw/dma/meson.build  |   1 +
 include/hw/dma/atcdmac300.h | 171 ++
 4 files changed, 610 insertions(+)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 include/hw/dma/atcdmac300.h

diff --git a/hw/dma/Kconfig b/hw/dma/Kconfig
index 98fbb1bb04..b3f4edc873 100644
--- a/hw/dma/Kconfig
+++ b/hw/dma/Kconfig
@@ -30,3 +30,6 @@ config SIFIVE_PDMA
 config XLNX_CSU_DMA
 bool
 select REGISTER
+
+config ATCDMAC300
+bool
diff --git a/hw/dma/atcdmac300.c b/hw/dma/atcdmac300.c
new file mode 100644
index 00..4cc37060d6
--- /dev/null
+++ b/hw/dma/atcdmac300.c
@@ -0,0 +1,435 @@
+/*
+ * Andes ATCDMAC300 (Andes Technology DMA Controller)
+ *
+ * Copyright (c) 2022 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see 
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/dma/atcdmac300.h"
+#include "hw/irq.h"
+#include "hw/qdev-properties.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "exec/memattrs.h"
+#include "exec/address-spaces.h"
+
+/* #define DEBUG_ANDES_ATCDMAC300 */
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_ANDES_ATCDMAC300
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+static void atcdmac300_dma_int_stat_update(ATCDMAC300State *s, int status,
+   int ch)
+{
+s->IntStatus |= (1 << (status + ch));
+}
+
+static void atcdmac300_dma_reset_chan(ATCDMAC300State *s, int ch)
+{
+if (s) {
+s->chan[ch].ChnCtrl &= ~(1 << CHAN_CTL_ENABLE);
+s->ChEN &= ~(1 << ch);
+}
+}
+
+static void atcdmac300_dma_reset(ATCDMAC300State *s)
+{
+int ch;
+for (ch = 0; ch < ATCDMAC300_MAX_CHAN; ch++) {
+atcdmac300_dma_reset_chan(s, ch);
+}
+}
+
+static uint64_t atcdmac300_read(void *opaque, hwaddr offset, unsigned size)
+{
+ATCDMAC300State *s = opaque;
+int ch = 0;
+uint64_t result = 0;
+
+if (offset >= 0x40) {
+ch = ATCDMAC300_GET_CHAN(offset);
+offset = ATCDMAC300_GET_OFF(offset, ch);
+}
+
+switch (offset) {
+case ATCDMAC300_DMA_CFG:
+result = s->DMACfg;
+break;
+case ATCDMAC300_DMAC_CTRL:
+break;
+case ATCDMAC300_CHN_ABT:
+break;
+case ATCDMAC300_INT_STATUS:
+result = s->IntStatus;
+break;
+case ATCDMAC300_CHAN_ENABLE:
+result = s->ChEN;
+break;
+case ATCDMAC300_CHAN_CTL:
+result = s->chan[ch].ChnCtrl;
+break;
+default:
+LOGGE("%s: Bad offset 0x%" HWADDR_PRIX "\n",
+  __func__, offset);
+break;
+}
+
+LOG("### atcdmac300_read()=0x%lx, val=0x%lx\n", offset, result);
+return result;
+}
+
+static MemTxResult dma_iopmp_read(ATCDMAC300State *s, hwaddr addr, void *buf,
+  hwaddr len)
+{
+if (s->iopmp_as) {
+MemTxAttrs dma_attrs = {.iopmp_sid = s->sid};
+return address_space_rw(s->iopmp_as, addr, dma_attrs,
+buf, len, false);
+}
+cpu_physical_memory_read(addr, buf, len);
+return MEMTX_OK;
+}
+
+static MemTxResult dma_iopmp_write(ATCDMAC300State *s, hwaddr addr, void *buf,
+   hwaddr len)
+{
+if (s->iopmp_as) {
+MemTxAttrs dma_attrs = {.iopmp_sid = s->sid};
+return address_space_rw(s->iopmp_as, addr, dma_attrs,
+buf, len, true);
+}
+cpu_physical_memory_write(addr, buf, len);
+return MEMTX_OK;
+}
+
+static void atcdmac300_co_run(void *opaque)
+{
+ATCDMAC300State *s = opaque;
+int64_t remain_size, burst_remain_size, copy_size, copy_size_dst;
+uint64_t src_addr, dst_addr, src_width, dst_width, burst_size,
+ src_addr_ctl, dst_addr_ctl, int_tc_mask, int_err_mask,
+ int_abort_mask;
+uint64_t buf[ATCDMAC300_MAX_BURST_SIZE];
+int result = 0;
+
+while (1) {
+for (int ch = 0; ch < ATCDMAC300_MAX_CHAN; ch++) {
+if (((s->chan[ch].ChnCtrl >> CHAN_CTL_ENABLE) & 0x1) == 0x1) {
+src_width = (s->chan[ch].ChnCtrl >> CHAN_CTL_SRC_WIDTH) &
+CHAN_CTL_SRC_WIDTH_MASK;
+ 

[PATCH 0/6] Support RISC-V IOPMP

[Ethan Chen via]

This series implements IOPMP specification v1.0.0-draft4 rapid-k model:
https://github.com/riscv-non-isa/iopmp-spec/blob/main/riscv_iopmp_specification.pdf

When IOPMP is enabled, a DMA device ATCDMAC300 is added to RISC-V virt platform.
This DMA devce is connected to the IOPMP and has the functionalities required
by IOPMP, including:
- Support specify source-id (SID)
- Support asynchronous I/O to handle stall transcations

Ethan Chen (6):
  exec/memory: Introduce the translate_size function within the IOMMU
class
  system/physmem: IOMMU: Invoke the translate_size function if it is
implemented
  exec/memattrs: Add iopmp source id to MemTxAttrs
  Add RISC-V IOPMP support
  hw/dma: Add Andes ATCDMAC300 support
  hw/riscv/virt: Add IOPMP support

 hw/dma/Kconfig|   3 +
 hw/dma/atcdmac300.c   | 435 +
 hw/dma/meson.build|   1 +
 hw/misc/Kconfig   |   3 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 881 ++
 hw/riscv/Kconfig  |   2 +
 hw/riscv/virt.c   |  72 ++-
 include/exec/memattrs.h   |   3 +
 include/exec/memory.h |  19 +
 include/hw/dma/atcdmac300.h   | 171 +++
 include/hw/misc/riscv_iopmp.h | 322 +
 include/hw/riscv/virt.h   |  10 +-
 system/physmem.c  |   9 +-
 14 files changed, 1927 insertions(+), 5 deletions(-)
 create mode 100644 hw/dma/atcdmac300.c
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/dma/atcdmac300.h
 create mode 100644 include/hw/misc/riscv_iopmp.h

-- 
2.34.1

[PATCH 4/6] Add RISC-V IOPMP support

[Ethan Chen via]

Support specification Version 1.0.0-draft4 rapid-k model.

Signed-off-by: Ethan Chen 
---
 hw/misc/Kconfig   |   3 +
 hw/misc/meson.build   |   1 +
 hw/misc/riscv_iopmp.c | 881 ++
 include/hw/misc/riscv_iopmp.h | 322 +
 4 files changed, 1207 insertions(+)
 create mode 100644 hw/misc/riscv_iopmp.c
 create mode 100644 include/hw/misc/riscv_iopmp.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index dba41afe67..8c63eb4cef 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -197,4 +197,7 @@ config DJMEMC
 config IOSB
 bool
 
+config RISCV_IOPMP
+bool
+
 source macio/Kconfig
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index f60de33f9a..7826ed9b7b 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -35,6 +35,7 @@ system_ss.add(when: 'CONFIG_SIFIVE_E_PRCI', if_true: 
files('sifive_e_prci.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_E_AON', if_true: files('sifive_e_aon.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_OTP', if_true: files('sifive_u_otp.c'))
 system_ss.add(when: 'CONFIG_SIFIVE_U_PRCI', if_true: files('sifive_u_prci.c'))
+specific_ss.add(when: 'CONFIG_RISCV_IOPMP', if_true: files('riscv_iopmp.c'))
 
 subdir('macio')
 
diff --git a/hw/misc/riscv_iopmp.c b/hw/misc/riscv_iopmp.c
new file mode 100644
index 00..5f9535501a
--- /dev/null
+++ b/hw/misc/riscv_iopmp.c
@@ -0,0 +1,881 @@
+/*
+ * QEMU RISC-V IOPMP (Input Output Physical Memory Protection)
+ *
+ * Copyright (c) 2023 Andes Tech. Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "trace.h"
+#include "exec/exec-all.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/misc/riscv_iopmp.h"
+#include "memory.h"
+#include "hw/irq.h"
+
+#define TYPE_IOPMP_IOMMU_MEMORY_REGION "iopmp-iommu-memory-region"
+
+#define LOGGE(x...) qemu_log_mask(LOG_GUEST_ERROR, x)
+#define xLOG(x...)
+#define yLOG(x...) qemu_log(x)
+#ifdef DEBUG_RISCV_IOPMP
+  #define LOG(x...) yLOG(x)
+#else
+  #define LOG(x...) xLOG(x)
+#endif
+
+#define MEMTX_IOPMP_STALL (1 << 3)
+
+
+static void iopmp_decode_napot(target_ulong a, target_ulong *sa,
+   target_ulong *ea)
+{
+/*
+ * ...aaa0   8-byte NAPOT range
+ * ...aa01   16-byte NAPOT range
+ * ...a011   32-byte NAPOT range
+ * ...
+ * aa01...   2^XLEN-byte NAPOT range
+ * a011...   2^(XLEN+1)-byte NAPOT range
+ * 0111...   2^(XLEN+2)-byte NAPOT range
+ *  ...   Reserved
+ */
+
+a = (a << 2) | 0x3;
+*sa = a & (a + 1);
+*ea = a | (a + 1);
+}
+
+static void iopmp_update_rule(IopmpState *s, uint32_t entry_index)
+{
+uint8_t this_cfg = s->regs.entry[entry_index].cfg_reg;
+target_ulong this_addr = s->regs.entry[entry_index].addr_reg;
+target_ulong prev_addr = 0u;
+target_ulong sa = 0u;
+target_ulong ea = 0u;
+
+if (entry_index >= 1u) {
+prev_addr = s->regs.entry[entry_index - 1].addr_reg;
+}
+
+switch (iopmp_get_field(this_cfg, ENTRY_CFG_A)) {
+case IOPMP_AMATCH_OFF:
+sa = 0u;
+ea = -1;
+break;
+
+case IOPMP_AMATCH_TOR:
+sa = (prev_addr) << 2; /* shift up from [xx:0] to [xx+2:2] */
+ea = ((this_addr) << 2) - 1u;
+if (sa > ea) {
+sa = ea = 0u;
+}
+break;
+
+case IOPMP_AMATCH_NA4:
+sa = this_addr << 2; /* shift up from [xx:0] to [xx+2:2] */
+ea = (sa + 4u) - 1u;
+break;
+
+case IOPMP_AMATCH_NAPOT:
+iopmp_decode_napot(this_addr, &sa, &ea);
+break;
+
+default:
+sa = 0u;
+ea = 0u;
+break;
+}
+
+s->entry_addr[entry_index].sa = sa;
+s->entry_addr[entry_index].ea = ea;
+}
+
+static uint64_t
+iopmp_read(void *opaque, hwaddr addr, unsigned size)
+{
+IopmpState *s = IOPMP(opaque);
+uint32_t rz = 0;
+uint32_t is_stall = 0;
+uint32_t sid;
+switch (addr) {
+case IOPMP_VERSION ... IOPMP_USER_CFG0 + 16 * (IOPMP_MAX_ENTRY_NUM - 1):
+switch (addr) {
+case IOPMP_VERSION:
+rz = VENDER_ANDES << VERSION_VENDOR |
+ SPECVER_1_0_0_DRAFT4 << VERSION_SPECVER;
+break;
+case IOPMP_IMP:
+rz = IMPID_1_0_0_DRAFT4_0;
+break;
+case IOPMP_H