[PATCH v2 1/1] target/riscv/kvm.c: Fix the hart bit setting of AIA
In AIA spec, each hart (or each hart within a group) has a unique hart
number to locate the memory pages of interrupt files in the address
space. The number of bits required to represent any hart number is equal
to ceil(log2(hmax + 1)), where hmax is the largest hart number among
groups.
However, if the largest hart number among groups is a power of 2, QEMU
will pass an inaccurate hart-index-bit setting to Linux. For example, when
the guest OS has 4 harts, only ceil(log2(3 + 1)) = 2 bits are sufficient
to represent 4 harts, but we passes 3 to Linux. The code needs to be
updated to ensure accurate hart-index-bit settings.
Additionally, a Linux patch[1] is necessary to correctly recover the hart
index when the guest OS has only 1 hart, where the hart-index-bit is 0.
[1]
https://lore.kernel.org/lkml/20240415064905.25184-1-yongxuan.w...@sifive.com/t/
Signed-off-by: Yong-Xuan Wang
---
Changelog
v2:
- update commit message
---
target/riscv/kvm/kvm-cpu.c | 9 -
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index 473416649fda..235e2cdaca1a 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -1777,7 +1777,14 @@ void kvm_riscv_aia_create(MachineState *machine,
uint64_t group_shift,
}
}
-hart_bits = find_last_bit(_hart_per_socket, BITS_PER_LONG) + 1;
+
+if (max_hart_per_socket > 1) {
+max_hart_per_socket--;
+hart_bits = find_last_bit(_hart_per_socket, BITS_PER_LONG) + 1;
+} else {
+hart_bits = 0;
+}
+
ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
KVM_DEV_RISCV_AIA_CONFIG_HART_BITS,
_bits, true, NULL);
--
2.17.1
Re: [PATCH 1/1] target/riscv/kvm.c: Fix the hart bit setting of AIA
Hi Alistair,
On Mon, Apr 29, 2024 at 11:25 AM Alistair Francis wrote:
>
> On Mon, Apr 15, 2024 at 4:53 PM Yong-Xuan Wang
> wrote:
> >
> > The hart bit setting is different with Linux AIA driver[1] when the number
> > of hart is power of 2. For example, when the guest has 4 harts, the
> > estimated result of AIA driver is 2, whereas we pass 3 to RISC-V/KVM. Since
> > only 2 bits are needed to represent 4 harts, update the formula to get the
> > accurate result.
>
> I don't really follow this.
>
> Do you mind re-wording it to talk about what the specification says?
> Not what Linux does.
>
Sure!
> >
> > [1]
> > https://lore.kernel.org/all/20240307140307.646078-1-apa...@ventanamicro.com/
> >
> > Signed-off-by: Yong-Xuan Wang
> > ---
> > target/riscv/kvm/kvm-cpu.c | 9 -
> > 1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
> > index 6a6c6cae80f1..388c4ddaa145 100644
> > --- a/target/riscv/kvm/kvm-cpu.c
> > +++ b/target/riscv/kvm/kvm-cpu.c
> > @@ -1642,7 +1642,14 @@ void kvm_riscv_aia_create(MachineState *machine,
> > uint64_t group_shift,
> > }
> > }
> >
> > -hart_bits = find_last_bit(_hart_per_socket, BITS_PER_LONG) + 1;
> > +
> > +if (max_hart_per_socket > 1) {
> > +max_hart_per_socket--;
>
> Assuming there are an even number of cores (which there usually are)
> won't this always result in a
>
> > +hart_bits = find_last_bit(_hart_per_socket, BITS_PER_LONG) + 1;
>
> 1 being returned by find_last_bit()?
>
find_last_bit() returns the position of the leftmost set bit. The
output will not be 1 when the given input is an odd number.
Regards,
Yong-Xuan
> Alistair
>
> > +} else {
> > +hart_bits = 0;
> > +}
> > +
> > ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
> > KVM_DEV_RISCV_AIA_CONFIG_HART_BITS,
> > _bits, true, NULL);
> > --
> > 2.17.1
> >
> >
[PATCH 1/1] target/riscv/kvm.c: Fix the hart bit setting of AIA
The hart bit setting is different with Linux AIA driver[1] when the number
of hart is power of 2. For example, when the guest has 4 harts, the
estimated result of AIA driver is 2, whereas we pass 3 to RISC-V/KVM. Since
only 2 bits are needed to represent 4 harts, update the formula to get the
accurate result.
[1] https://lore.kernel.org/all/20240307140307.646078-1-apa...@ventanamicro.com/
Signed-off-by: Yong-Xuan Wang
---
target/riscv/kvm/kvm-cpu.c | 9 -
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index 6a6c6cae80f1..388c4ddaa145 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -1642,7 +1642,14 @@ void kvm_riscv_aia_create(MachineState *machine,
uint64_t group_shift,
}
}
-hart_bits = find_last_bit(_hart_per_socket, BITS_PER_LONG) + 1;
+
+if (max_hart_per_socket > 1) {
+max_hart_per_socket--;
+hart_bits = find_last_bit(_hart_per_socket, BITS_PER_LONG) + 1;
+} else {
+hart_bits = 0;
+}
+
ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
KVM_DEV_RISCV_AIA_CONFIG_HART_BITS,
_bits, true, NULL);
--
2.17.1
[PATCH v2 1/1] target/riscv/kvm: fix timebase-frequency when using KVM acceleration
The timebase-frequency of guest OS should be the same with host
machine. The timebase-frequency value in DTS should be got from
hypervisor when using KVM acceleration.
Reviewed-by: Andrew Jones
Signed-off-by: Yong-Xuan Wang
---
Changelog
v2:
- update the function definition
- restructure if-else statement
---
hw/riscv/virt.c | 2 ++
target/riscv/kvm/kvm-cpu.c | 9 +
target/riscv/kvm/kvm_riscv.h | 1 +
3 files changed, 12 insertions(+)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index a094af97c32a..533b17799581 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -711,6 +711,8 @@ static void create_fdt_sockets(RISCVVirtState *s, const
MemMapEntry *memmap,
qemu_fdt_add_subnode(ms->fdt, "/cpus");
qemu_fdt_setprop_cell(ms->fdt, "/cpus", "timebase-frequency",
+ kvm_enabled() ?
+ kvm_riscv_get_timebase_frequency(first_cpu) :
RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ);
qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#size-cells", 0x0);
qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#address-cells", 0x1);
diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index c7afdb1e81b7..bbb115eaa867 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -739,6 +739,15 @@ static void kvm_riscv_put_regs_timer(CPUState *cs)
env->kvm_timer_dirty = false;
}
+uint64_t kvm_riscv_get_timebase_frequency(CPUState *cs)
+{
+uint64_t reg;
+
+KVM_RISCV_GET_TIMER(cs, frequency, reg);
+
+return reg;
+}
+
static int kvm_riscv_get_regs_vector(CPUState *cs)
{
RISCVCPU *cpu = RISCV_CPU(cs);
diff --git a/target/riscv/kvm/kvm_riscv.h b/target/riscv/kvm/kvm_riscv.h
index 4bd98fddc776..58518988681d 100644
--- a/target/riscv/kvm/kvm_riscv.h
+++ b/target/riscv/kvm/kvm_riscv.h
@@ -28,5 +28,6 @@ void kvm_riscv_aia_create(MachineState *machine, uint64_t
group_shift,
void riscv_kvm_aplic_request(void *opaque, int irq, int level);
int kvm_riscv_sync_mpstate_to_kvm(RISCVCPU *cpu, int state);
void riscv_kvm_cpu_finalize_features(RISCVCPU *cpu, Error **errp);
+uint64_t kvm_riscv_get_timebase_frequency(CPUState *cs);
#endif
--
2.17.1
[PATCH 1/1] target/riscv/kvm: fix timebase-frequency when using KVM acceleration
The timebase-frequency of guest OS should be the same with host
machine. The timebase-frequency value in DTS should be got from
hypervisor when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
---
hw/riscv/virt.c | 11 +--
target/riscv/kvm/kvm-cpu.c | 9 +
target/riscv/kvm/kvm_riscv.h | 13 +
3 files changed, 31 insertions(+), 2 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index a094af97c32a..a7ed7fa13010 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -710,8 +710,15 @@ static void create_fdt_sockets(RISCVVirtState *s, const
MemMapEntry *memmap,
int socket_count = riscv_socket_count(ms);
qemu_fdt_add_subnode(ms->fdt, "/cpus");
-qemu_fdt_setprop_cell(ms->fdt, "/cpus", "timebase-frequency",
- RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ);
+
+if (kvm_enabled()) {
+qemu_fdt_setprop_cell(ms->fdt, "/cpus", "timebase-frequency",
+ kvm_riscv_get_timebase_frequency(first_cpu));
+} else {
+qemu_fdt_setprop_cell(ms->fdt, "/cpus", "timebase-frequency",
+ RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ);
+}
+
qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#size-cells", 0x0);
qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#address-cells", 0x1);
qemu_fdt_add_subnode(ms->fdt, "/cpus/cpu-map");
diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index c7afdb1e81b7..bbb115eaa867 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -739,6 +739,15 @@ static void kvm_riscv_put_regs_timer(CPUState *cs)
env->kvm_timer_dirty = false;
}
+uint64_t kvm_riscv_get_timebase_frequency(CPUState *cs)
+{
+uint64_t reg;
+
+KVM_RISCV_GET_TIMER(cs, frequency, reg);
+
+return reg;
+}
+
static int kvm_riscv_get_regs_vector(CPUState *cs)
{
RISCVCPU *cpu = RISCV_CPU(cs);
diff --git a/target/riscv/kvm/kvm_riscv.h b/target/riscv/kvm/kvm_riscv.h
index 4bd98fddc776..130a4bde0480 100644
--- a/target/riscv/kvm/kvm_riscv.h
+++ b/target/riscv/kvm/kvm_riscv.h
@@ -29,4 +29,17 @@ void riscv_kvm_aplic_request(void *opaque, int irq, int
level);
int kvm_riscv_sync_mpstate_to_kvm(RISCVCPU *cpu, int state);
void riscv_kvm_cpu_finalize_features(RISCVCPU *cpu, Error **errp);
+#ifdef CONFIG_KVM
+
+uint64_t kvm_riscv_get_timebase_frequency(CPUState *cs);
+
+#else
+
+static inline uint64_t kvm_riscv_get_timebase_frequency(CPUState *cs)
+{
+g_assert_not_reached();
+}
+
+#endif
+
#endif
--
2.17.1
[PATCH 1/1] target/riscv/kvm.c: remove group setting of KVM AIA if the machine only has 1 socket
The emulated AIA within the Linux kernel restores the HART index
of the IMSICs according to the configured AIA settings. During
this process, the group setting is used only when the machine
partitions harts into groups. It's unnecessary to set the group
configuration if the machine has only one socket, as its address
space might not contain the group shift.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm/kvm-cpu.c | 31 +--
1 file changed, 17 insertions(+), 14 deletions(-)
diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index 62a1e51f0a2e..6494597157b8 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -1387,21 +1387,24 @@ void kvm_riscv_aia_create(MachineState *machine,
uint64_t group_shift,
exit(1);
}
-socket_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
-ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
-KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS,
-_bits, true, NULL);
-if (ret < 0) {
-error_report("KVM AIA: failed to set group_bits");
-exit(1);
-}
-ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
-KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT,
-_shift, true, NULL);
-if (ret < 0) {
-error_report("KVM AIA: failed to set group_shift");
-exit(1);
+if (socket_count > 1) {
+socket_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS,
+_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set group_bits");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT,
+_shift, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set group_shift");
+exit(1);
+}
}
guest_bits = guest_num == 0 ? 0 :
--
2.17.1
[PATCH 1/1] hw/riscv/virt.c: fix the interrupts-extended property format of PLIC
The interrupts-extended property of PLIC only has 2 * hart number
fields when KVM enabled, copy 4 * hart number fields to fdt will
expose some uninitialized value.
In this patch, I also refactor the code about the setting of
interrupts-extended property of PLIC for improved readability.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 47 +++
1 file changed, 27 insertions(+), 20 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index d2eac2415619..e42baf82cab6 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -460,24 +460,6 @@ static void create_fdt_socket_plic(RISCVVirtState *s,
"sifive,plic-1.0.0", "riscv,plic0"
};
-if (kvm_enabled()) {
-plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
-} else {
-plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
-}
-
-for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
-if (kvm_enabled()) {
-plic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-plic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
-} else {
-plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
-plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
-plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
-plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
-}
-}
-
plic_phandles[socket] = (*phandle)++;
plic_addr = memmap[VIRT_PLIC].base + (memmap[VIRT_PLIC].size * socket);
plic_name = g_strdup_printf("/soc/plic@%lx", plic_addr);
@@ -490,8 +472,33 @@ static void create_fdt_socket_plic(RISCVVirtState *s,
(char **)_compat,
ARRAY_SIZE(plic_compat));
qemu_fdt_setprop(ms->fdt, plic_name, "interrupt-controller", NULL, 0);
-qemu_fdt_setprop(ms->fdt, plic_name, "interrupts-extended",
-plic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 4);
+
+if (kvm_enabled()) {
+plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+
+for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
+plic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+plic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
+}
+
+qemu_fdt_setprop(ms->fdt, plic_name, "interrupts-extended",
+ plic_cells,
+ s->soc[socket].num_harts * sizeof(uint32_t) * 2);
+ } else {
+plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
+
+for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
+plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
+plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
+plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
+plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
+}
+
+qemu_fdt_setprop(ms->fdt, plic_name, "interrupts-extended",
+ plic_cells,
+ s->soc[socket].num_harts * sizeof(uint32_t) * 4);
+}
+
qemu_fdt_setprop_cells(ms->fdt, plic_name, "reg",
0x0, plic_addr, 0x0, memmap[VIRT_PLIC].size);
qemu_fdt_setprop_cell(ms->fdt, plic_name, "riscv,ndev",
--
2.17.1
[PATCH v7 2/5] target/riscv: check the in-kernel irqchip support
We check the in-kernel irqchip support when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
target/riscv/kvm.c | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 9d8a8982f9..005e054604 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -914,7 +914,15 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
int kvm_arch_irqchip_create(KVMState *s)
{
-return 0;
+if (kvm_kernel_irqchip_split()) {
+error_report("-machine kernel_irqchip=split is not supported on
RISC-V.");
+exit(1);
+}
+
+/*
+ * We can create the VAIA using the newer device control API.
+ */
+return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
}
int kvm_arch_process_async_events(CPUState *cs)
--
2.17.1
[PATCH v7 0/5] Add RISC-V KVM AIA Support
This series adds support for KVM AIA in RISC-V architecture. In order to test these patches, we require Linux with KVM AIA support which can be found in the riscv_kvm_aia_hwaccel_v1 branch at https://github.com/avpatel/linux.git --- v7: - fix compiler warning in PATCH3 - rename the "kvm-aia" property to "riscv-aia" and add it as a RISC-V KVM accelerator property. also move this setting from PATCH5 to PATCH3 in the code. v6: - fix alignment - add hart index to the error message of ISMIC address setting in PATCH3 v5: - remove the linux-header update patch since the riscv-to-apply.next QEMU has synced up to Linux 6.5-rc1 headers. - create the APLIC and IMSIC FDT helper functions in PATCH1 - add the irqfd support in PATCH3 - fix the comments and refine the code v4: - update the linux header by the scripts/update-linux-headers.sh in PATCH1 - remove the checking for "aplic_m" before creating S-mode APLIC device in PATCH2 - add more setting when we initialize the KVM AIA chip in PATCH4 - fix msi message delivery and the APLIC devices emulation in PATCH5 - fix the AIA devices mapping with NUMA enabled in PATCH6 - add "kvm-aia" parameter to sepecify the KVM AIA mode in PATCH6 v3: - fix typo - tag the linux-header patch as placeholder v2: - rebase to riscv-to-apply.next - update the linux header by the scripts/update-linux-headers.sh Yong-Xuan Wang (5): target/riscv: support the AIA device emulation with KVM enabled target/riscv: check the in-kernel irqchip support target/riscv: Create an KVM AIA irqchip target/riscv: update APLIC and IMSIC to support KVM AIA target/riscv: select KVM AIA in riscv virt machine hw/intc/riscv_aplic.c| 56 -- hw/intc/riscv_imsic.c| 25 ++- hw/riscv/virt.c | 372 --- target/riscv/kvm.c | 196 - target/riscv/kvm_riscv.h | 4 + 5 files changed, 454 insertions(+), 199 deletions(-) -- 2.17.1
[PATCH v7 3/5] target/riscv: Create an KVM AIA irqchip
We create a vAIA chip by using the KVM_DEV_TYPE_RISCV_AIA and then set up
the chip with the KVM_DEV_RISCV_AIA_GRP_* APIs.
We also extend KVM accelerator to specify the KVM AIA mode. The "riscv-aia"
parameter is passed along with --accel in QEMU command-line.
1) "riscv-aia=emul": IMSIC is emulated by hypervisor
2) "riscv-aia=hwaccel": use hardware guest IMSIC
3) "riscv-aia=auto": use the hardware guest IMSICs whenever available
otherwise we fallback to software emulation.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
target/riscv/kvm.c | 186 +++
target/riscv/kvm_riscv.h | 4 +
2 files changed, 190 insertions(+)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 005e054604..0c17e2027a 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -36,6 +36,7 @@
#include "exec/address-spaces.h"
#include "hw/boards.h"
#include "hw/irq.h"
+#include "hw/intc/riscv_imsic.h"
#include "qemu/log.h"
#include "hw/loader.h"
#include "kvm_riscv.h"
@@ -43,6 +44,7 @@
#include "chardev/char-fe.h"
#include "migration/migration.h"
#include "sysemu/runstate.h"
+#include "hw/riscv/numa.h"
static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,
uint64_t idx)
@@ -1023,6 +1025,190 @@ bool kvm_arch_cpu_check_are_resettable(void)
return true;
}
+static int aia_mode;
+
+static const char *kvm_aia_mode_str(uint64_t mode)
+{
+switch (mode) {
+case KVM_DEV_RISCV_AIA_MODE_EMUL:
+return "emul";
+case KVM_DEV_RISCV_AIA_MODE_HWACCEL:
+return "hwaccel";
+case KVM_DEV_RISCV_AIA_MODE_AUTO:
+default:
+return "auto";
+};
+}
+
+static char *riscv_get_kvm_aia(Object *obj, Error **errp)
+{
+return g_strdup(kvm_aia_mode_str(aia_mode));
+}
+
+static void riscv_set_kvm_aia(Object *obj, const char *val, Error **errp)
+{
+if (!strcmp(val, "emul")) {
+aia_mode = KVM_DEV_RISCV_AIA_MODE_EMUL;
+} else if (!strcmp(val, "hwaccel")) {
+aia_mode = KVM_DEV_RISCV_AIA_MODE_HWACCEL;
+} else if (!strcmp(val, "auto")) {
+aia_mode = KVM_DEV_RISCV_AIA_MODE_AUTO;
+} else {
+error_setg(errp, "Invalid KVM AIA mode");
+error_append_hint(errp, "Valid values are emul, hwaccel, and auto.\n");
+}
+}
+
void kvm_arch_accel_class_init(ObjectClass *oc)
{
+object_class_property_add_str(oc, "riscv-aia", riscv_get_kvm_aia,
+ riscv_set_kvm_aia);
+object_class_property_set_description(oc, "riscv-aia",
+ "Set KVM AIA mode. Valid values are "
+ "emul, hwaccel, and auto. Default "
+ "is auto.");
+object_property_set_default_str(object_class_property_find(oc,
"riscv-aia"),
+"auto");
+}
+
+void kvm_riscv_aia_create(MachineState *machine, uint64_t group_shift,
+ uint64_t aia_irq_num, uint64_t aia_msi_num,
+ uint64_t aplic_base, uint64_t imsic_base,
+ uint64_t guest_num)
+{
+int ret, i;
+int aia_fd = -1;
+uint64_t default_aia_mode;
+uint64_t socket_count = riscv_socket_count(machine);
+uint64_t max_hart_per_socket = 0;
+uint64_t socket, base_hart, hart_count, socket_imsic_base, imsic_addr;
+uint64_t socket_bits, hart_bits, guest_bits;
+
+aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
+
+if (aia_fd < 0) {
+error_report("Unable to create in-kernel irqchip");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_aia_mode, false, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to get current KVM AIA mode");
+exit(1);
+}
+qemu_log("KVM AIA: default mode is %s\n",
+ kvm_aia_mode_str(default_aia_mode));
+
+if (default_aia_mode != aia_mode) {
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_mode, true, NULL);
+if (ret < 0)
+warn_report("KVM AIA: failed to set KVM AIA mode");
+else
+qemu_log("KVM AIA: set current mode to %s\n",
+ kvm_aia_mode_str(aia_mode));
+}
+
+re
[PATCH v7 4/5] target/riscv: update APLIC and IMSIC to support KVM AIA
KVM AIA can't emulate APLIC only. When "aia=aplic" parameter is passed,
APLIC devices is emulated by QEMU. For "aia=aplic-imsic", remove the
mmio operations of APLIC when using KVM AIA and send wired interrupt
signal via KVM_IRQ_LINE API.
After KVM AIA enabled, MSI messages are delivered by KVM_SIGNAL_MSI API
when the IMSICs receive mmio write requests.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
hw/intc/riscv_aplic.c | 56 ++-
hw/intc/riscv_imsic.c | 25 +++
2 files changed, 61 insertions(+), 20 deletions(-)
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index 4bdc6a5d1a..592c3ce768 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -31,6 +31,7 @@
#include "hw/irq.h"
#include "target/riscv/cpu.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define APLIC_MAX_IDC (1UL << 14)
@@ -148,6 +149,15 @@
#define APLIC_IDC_CLAIMI 0x1c
+/*
+ * KVM AIA only supports APLIC MSI, fallback to QEMU emulation if we want to
use
+ * APLIC Wired.
+ */
+static bool is_kvm_aia(bool msimode)
+{
+return kvm_irqchip_in_kernel() && msimode;
+}
+
static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
uint32_t word)
{
@@ -471,6 +481,11 @@ static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState
*aplic, uint32_t idc)
return topi;
}
+static void riscv_kvm_aplic_request(void *opaque, int irq, int level)
+{
+kvm_set_irq(kvm_state, irq, !!level);
+}
+
static void riscv_aplic_request(void *opaque, int irq, int level)
{
bool update = false;
@@ -801,29 +816,35 @@ static void riscv_aplic_realize(DeviceState *dev, Error
**errp)
uint32_t i;
RISCVAPLICState *aplic = RISCV_APLIC(dev);
-aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
-aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
-aplic->state = g_new0(uint32_t, aplic->num_irqs);
-aplic->target = g_new0(uint32_t, aplic->num_irqs);
-if (!aplic->msimode) {
-for (i = 0; i < aplic->num_irqs; i++) {
-aplic->target[i] = 1;
+if (!is_kvm_aia(aplic->msimode)) {
+aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
+aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
+aplic->state = g_new0(uint32_t, aplic->num_irqs);
+aplic->target = g_new0(uint32_t, aplic->num_irqs);
+if (!aplic->msimode) {
+for (i = 0; i < aplic->num_irqs; i++) {
+aplic->target[i] = 1;
+}
}
-}
-aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
-aplic->iforce = g_new0(uint32_t, aplic->num_harts);
-aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
+aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
+aplic->iforce = g_new0(uint32_t, aplic->num_harts);
+aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
-memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops, aplic,
- TYPE_RISCV_APLIC, aplic->aperture_size);
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops,
+ aplic, TYPE_RISCV_APLIC, aplic->aperture_size);
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/*
* Only root APLICs have hardware IRQ lines. All non-root APLICs
* have IRQ lines delegated by their parent APLIC.
*/
if (!aplic->parent) {
-qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+if (is_kvm_aia(aplic->msimode)) {
+qdev_init_gpio_in(dev, riscv_kvm_aplic_request, aplic->num_irqs);
+} else {
+qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+}
}
/* Create output IRQ lines for non-MSI mode */
@@ -958,7 +979,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
qdev_prop_set_bit(dev, "mmode", mmode);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!is_kvm_aia(msimode)) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
if (parent) {
riscv_aplic_add_child(parent, dev);
diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
index fea3385b51..760dbddcf7 100644
--- a/hw/intc/riscv_imsic.c
+++ b/hw/intc/riscv_imsic.c
@@ -32,6 +32,7 @@
#include "target/riscv/cpu.h"
#include "target/riscv/cpu_bits.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/v
[PATCH v7 1/5] target/riscv: support the AIA device emulation with KVM enabled
In this patch, we create the APLIC and IMSIC FDT helper functions and
remove M mode AIA devices when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
hw/riscv/virt.c | 290 +++-
1 file changed, 137 insertions(+), 153 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index d90286dc46..f595380be1 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -516,79 +516,28 @@ static uint32_t imsic_num_bits(uint32_t count)
return ret;
}
-static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
- uint32_t *phandle, uint32_t *intc_phandles,
- uint32_t *msi_m_phandle, uint32_t *msi_s_phandle)
+static void create_fdt_one_imsic(RISCVVirtState *s, hwaddr base_addr,
+ uint32_t *intc_phandles, uint32_t msi_phandle,
+ bool m_mode, uint32_t imsic_guest_bits)
{
int cpu, socket;
char *imsic_name;
MachineState *ms = MACHINE(s);
int socket_count = riscv_socket_count(ms);
-uint32_t imsic_max_hart_per_socket, imsic_guest_bits;
+uint32_t imsic_max_hart_per_socket;
uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size;
-*msi_m_phandle = (*phandle)++;
-*msi_s_phandle = (*phandle)++;
imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
imsic_regs = g_new0(uint32_t, socket_count * 4);
-/* M-level IMSIC node */
for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
+imsic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT);
}
-imsic_max_hart_per_socket = 0;
-for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_M].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
-imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
-imsic_regs[socket * 4 + 0] = 0;
-imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
-imsic_regs[socket * 4 + 2] = 0;
-imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
-if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
-imsic_max_hart_per_socket = s->soc[socket].num_harts;
-}
-}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_M].base);
-qemu_fdt_add_subnode(ms->fdt, imsic_name);
-qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
-"riscv,imsics");
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
-FDT_IMSIC_INT_CELLS);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
-qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
-socket_count * sizeof(uint32_t) * 4);
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
-VIRT_IRQCHIP_NUM_MSIS);
-if (socket_count > 1) {
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits",
-imsic_num_bits(imsic_max_hart_per_socket));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits",
-imsic_num_bits(socket_count));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift",
-IMSIC_MMIO_GROUP_MIN_SHIFT);
-}
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle);
-
-g_free(imsic_name);
-/* S-level IMSIC node */
-for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
-imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
-}
-imsic_guest_bits = imsic_num_bits(s->aia_guests + 1);
imsic_max_hart_per_socket = 0;
for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_S].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
+imsic_addr = base_addr + socket * VIRT_IMSIC_GROUP_MAX_SIZE;
imsic_size = IMSIC_HART_SIZE(imsic_guest_bits) *
s->soc[socket].num_harts;
imsic_regs[socket * 4 + 0] = 0;
@@ -599,119 +548,151 @@ static void create_fdt_imsic(RISCVVirtState *s, const
MemMapEntry *memmap,
imsic_max_hart_per_socket = s->soc[socket].num_harts;
}
}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned
[PATCH v7 5/5] target/riscv: select KVM AIA in riscv virt machine
Select KVM AIA when the host kernel has in-kernel AIA chip support.
Since KVM AIA only has one APLIC instance, we map the QEMU APLIC
devices to KVM APLIC.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
hw/riscv/virt.c | 94 +
1 file changed, 63 insertions(+), 31 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index f595380be1..4af73ac1bb 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
#include "hw/riscv/virt.h"
#include "hw/riscv/boot.h"
#include "hw/riscv/numa.h"
+#include "kvm_riscv.h"
#include "hw/intc/riscv_aclint.h"
#include "hw/intc/riscv_aplic.h"
#include "hw/intc/riscv_imsic.h"
@@ -75,6 +76,12 @@
#error "Can't accomodate all IMSIC groups in address space"
#endif
+/* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */
+static bool virt_use_kvm_aia(RISCVVirtState *s)
+{
+return kvm_irqchip_in_kernel() && s->aia_type == VIRT_AIA_TYPE_APLIC_IMSIC;
+}
+
static const MemMapEntry virt_memmap[] = {
[VIRT_DEBUG] ={0x0, 0x100 },
[VIRT_MROM] = { 0x1000,0xf000 },
@@ -609,16 +616,16 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int
socket,
uint32_t *intc_phandles,
uint32_t aplic_phandle,
uint32_t aplic_child_phandle,
- bool m_mode)
+ bool m_mode, int num_harts)
{
int cpu;
char *aplic_name;
uint32_t *aplic_cells;
MachineState *ms = MACHINE(s);
-aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+aplic_cells = g_new0(uint32_t, num_harts * 2);
-for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
+for (cpu = 0; cpu < num_harts; cpu++) {
aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
aplic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT);
}
@@ -632,8 +639,7 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int
socket,
if (s->aia_type == VIRT_AIA_TYPE_APLIC) {
qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended",
- aplic_cells,
- s->soc[socket].num_harts * sizeof(uint32_t) * 2);
+ aplic_cells, num_harts * sizeof(uint32_t) * 2);
} else {
qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", msi_phandle);
}
@@ -664,7 +670,8 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
uint32_t msi_s_phandle,
uint32_t *phandle,
uint32_t *intc_phandles,
-uint32_t *aplic_phandles)
+uint32_t *aplic_phandles,
+int num_harts)
{
char *aplic_name;
unsigned long aplic_addr;
@@ -681,7 +688,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_M].size,
msi_m_phandle, intc_phandles,
aplic_m_phandle, aplic_s_phandle,
- true);
+ true, num_harts);
}
/* S-level APLIC node */
@@ -690,7 +697,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_S].size,
msi_s_phandle, intc_phandles,
aplic_s_phandle, 0,
- false);
+ false, num_harts);
aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr);
@@ -774,34 +781,51 @@ static void create_fdt_sockets(RISCVVirtState *s, const
MemMapEntry *memmap,
*msi_pcie_phandle = msi_s_phandle;
}
-phandle_pos = ms->smp.cpus;
-for (socket = (socket_count - 1); socket >= 0; socket--) {
-phandle_pos -= s->soc[socket].num_harts;
-
-if (s->aia_type == VIRT_AIA_TYPE_NONE) {
-create_fdt_socket_plic(s, memmap, socket, phandle,
-_phandles[phandle_pos], xplic_phandles);
-} else {
-create_fdt_socket_aplic(s, memmap, socket,
-msi_m_phandle, msi_s_phandle, phandle,
-_phandles[phandle_pos], xplic_phandles);
+/* KVM AIA only has one APLIC instance */
+if (virt_use_kvm_aia(s)) {
+create_fdt_socket_aplic(s, memmap, 0,
+msi_m_phandle, msi_s_phandle, phandle,
+_phandles[0], xplic_phandles,
+ms->
[PATCH v6 4/5] target/riscv: update APLIC and IMSIC to support KVM AIA
KVM AIA can't emulate APLIC only. When "aia=aplic" parameter is passed,
APLIC devices is emulated by QEMU. For "aia=aplic-imsic", remove the
mmio operations of APLIC when using KVM AIA and send wired interrupt
signal via KVM_IRQ_LINE API.
After KVM AIA enabled, MSI messages are delivered by KVM_SIGNAL_MSI API
when the IMSICs receive mmio write requests.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
hw/intc/riscv_aplic.c | 56 ++-
hw/intc/riscv_imsic.c | 25 +++
2 files changed, 61 insertions(+), 20 deletions(-)
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index 4bdc6a5d1a..592c3ce768 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -31,6 +31,7 @@
#include "hw/irq.h"
#include "target/riscv/cpu.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define APLIC_MAX_IDC (1UL << 14)
@@ -148,6 +149,15 @@
#define APLIC_IDC_CLAIMI 0x1c
+/*
+ * KVM AIA only supports APLIC MSI, fallback to QEMU emulation if we want to
use
+ * APLIC Wired.
+ */
+static bool is_kvm_aia(bool msimode)
+{
+return kvm_irqchip_in_kernel() && msimode;
+}
+
static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
uint32_t word)
{
@@ -471,6 +481,11 @@ static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState
*aplic, uint32_t idc)
return topi;
}
+static void riscv_kvm_aplic_request(void *opaque, int irq, int level)
+{
+kvm_set_irq(kvm_state, irq, !!level);
+}
+
static void riscv_aplic_request(void *opaque, int irq, int level)
{
bool update = false;
@@ -801,29 +816,35 @@ static void riscv_aplic_realize(DeviceState *dev, Error
**errp)
uint32_t i;
RISCVAPLICState *aplic = RISCV_APLIC(dev);
-aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
-aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
-aplic->state = g_new0(uint32_t, aplic->num_irqs);
-aplic->target = g_new0(uint32_t, aplic->num_irqs);
-if (!aplic->msimode) {
-for (i = 0; i < aplic->num_irqs; i++) {
-aplic->target[i] = 1;
+if (!is_kvm_aia(aplic->msimode)) {
+aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
+aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
+aplic->state = g_new0(uint32_t, aplic->num_irqs);
+aplic->target = g_new0(uint32_t, aplic->num_irqs);
+if (!aplic->msimode) {
+for (i = 0; i < aplic->num_irqs; i++) {
+aplic->target[i] = 1;
+}
}
-}
-aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
-aplic->iforce = g_new0(uint32_t, aplic->num_harts);
-aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
+aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
+aplic->iforce = g_new0(uint32_t, aplic->num_harts);
+aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
-memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops, aplic,
- TYPE_RISCV_APLIC, aplic->aperture_size);
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops,
+ aplic, TYPE_RISCV_APLIC, aplic->aperture_size);
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/*
* Only root APLICs have hardware IRQ lines. All non-root APLICs
* have IRQ lines delegated by their parent APLIC.
*/
if (!aplic->parent) {
-qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+if (is_kvm_aia(aplic->msimode)) {
+qdev_init_gpio_in(dev, riscv_kvm_aplic_request, aplic->num_irqs);
+} else {
+qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+}
}
/* Create output IRQ lines for non-MSI mode */
@@ -958,7 +979,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
qdev_prop_set_bit(dev, "mmode", mmode);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!is_kvm_aia(msimode)) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
if (parent) {
riscv_aplic_add_child(parent, dev);
diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
index fea3385b51..760dbddcf7 100644
--- a/hw/intc/riscv_imsic.c
+++ b/hw/intc/riscv_imsic.c
@@ -32,6 +32,7 @@
#include "target/riscv/cpu.h"
#include "target/riscv/cpu_bits.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/v
[PATCH v6 1/5] target/riscv: support the AIA device emulation with KVM enabled
In this patch, we create the APLIC and IMSIC FDT helper functions and
remove M mode AIA devices when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
hw/riscv/virt.c | 290 +++-
1 file changed, 137 insertions(+), 153 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index d90286dc46..f595380be1 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -516,79 +516,28 @@ static uint32_t imsic_num_bits(uint32_t count)
return ret;
}
-static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
- uint32_t *phandle, uint32_t *intc_phandles,
- uint32_t *msi_m_phandle, uint32_t *msi_s_phandle)
+static void create_fdt_one_imsic(RISCVVirtState *s, hwaddr base_addr,
+ uint32_t *intc_phandles, uint32_t msi_phandle,
+ bool m_mode, uint32_t imsic_guest_bits)
{
int cpu, socket;
char *imsic_name;
MachineState *ms = MACHINE(s);
int socket_count = riscv_socket_count(ms);
-uint32_t imsic_max_hart_per_socket, imsic_guest_bits;
+uint32_t imsic_max_hart_per_socket;
uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size;
-*msi_m_phandle = (*phandle)++;
-*msi_s_phandle = (*phandle)++;
imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
imsic_regs = g_new0(uint32_t, socket_count * 4);
-/* M-level IMSIC node */
for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
+imsic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT);
}
-imsic_max_hart_per_socket = 0;
-for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_M].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
-imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
-imsic_regs[socket * 4 + 0] = 0;
-imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
-imsic_regs[socket * 4 + 2] = 0;
-imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
-if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
-imsic_max_hart_per_socket = s->soc[socket].num_harts;
-}
-}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_M].base);
-qemu_fdt_add_subnode(ms->fdt, imsic_name);
-qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
-"riscv,imsics");
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
-FDT_IMSIC_INT_CELLS);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
-qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
-socket_count * sizeof(uint32_t) * 4);
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
-VIRT_IRQCHIP_NUM_MSIS);
-if (socket_count > 1) {
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits",
-imsic_num_bits(imsic_max_hart_per_socket));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits",
-imsic_num_bits(socket_count));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift",
-IMSIC_MMIO_GROUP_MIN_SHIFT);
-}
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle);
-
-g_free(imsic_name);
-/* S-level IMSIC node */
-for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
-imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
-}
-imsic_guest_bits = imsic_num_bits(s->aia_guests + 1);
imsic_max_hart_per_socket = 0;
for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_S].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
+imsic_addr = base_addr + socket * VIRT_IMSIC_GROUP_MAX_SIZE;
imsic_size = IMSIC_HART_SIZE(imsic_guest_bits) *
s->soc[socket].num_harts;
imsic_regs[socket * 4 + 0] = 0;
@@ -599,119 +548,151 @@ static void create_fdt_imsic(RISCVVirtState *s, const
MemMapEntry *memmap,
imsic_max_hart_per_socket = s->soc[socket].num_harts;
}
}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned
[PATCH v6 5/5] target/riscv: select KVM AIA in riscv virt machine
Select KVM AIA when the host kernel has in-kernel AIA chip support.
Since KVM AIA only has one APLIC instance, we map the QEMU APLIC
devices to KVM APLIC.
We also extend virt machine to specify the KVM AIA mode. The "kvm-aia"
parameter is passed along with machine name in QEMU command-line.
1) "kvm-aia=emul": IMSIC is emulated by hypervisor
2) "kvm-aia=hwaccel": use hardware guest IMSIC
3) "kvm-aia=auto": use the hardware guest IMSICs whenever available
otherwise we fallback to software emulation.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
hw/riscv/virt.c | 132 ++--
include/hw/riscv/virt.h | 1 +
2 files changed, 102 insertions(+), 31 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index f595380be1..6367597dfa 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
#include "hw/riscv/virt.h"
#include "hw/riscv/boot.h"
#include "hw/riscv/numa.h"
+#include "kvm_riscv.h"
#include "hw/intc/riscv_aclint.h"
#include "hw/intc/riscv_aplic.h"
#include "hw/intc/riscv_imsic.h"
@@ -75,6 +76,12 @@
#error "Can't accomodate all IMSIC groups in address space"
#endif
+/* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */
+static bool virt_use_kvm_aia(RISCVVirtState *s)
+{
+return kvm_irqchip_in_kernel() && s->aia_type == VIRT_AIA_TYPE_APLIC_IMSIC;
+}
+
static const MemMapEntry virt_memmap[] = {
[VIRT_DEBUG] ={0x0, 0x100 },
[VIRT_MROM] = { 0x1000,0xf000 },
@@ -609,16 +616,16 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int
socket,
uint32_t *intc_phandles,
uint32_t aplic_phandle,
uint32_t aplic_child_phandle,
- bool m_mode)
+ bool m_mode, int num_harts)
{
int cpu;
char *aplic_name;
uint32_t *aplic_cells;
MachineState *ms = MACHINE(s);
-aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+aplic_cells = g_new0(uint32_t, num_harts * 2);
-for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
+for (cpu = 0; cpu < num_harts; cpu++) {
aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
aplic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT);
}
@@ -632,8 +639,7 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int
socket,
if (s->aia_type == VIRT_AIA_TYPE_APLIC) {
qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended",
- aplic_cells,
- s->soc[socket].num_harts * sizeof(uint32_t) * 2);
+ aplic_cells, num_harts * sizeof(uint32_t) * 2);
} else {
qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", msi_phandle);
}
@@ -664,7 +670,8 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
uint32_t msi_s_phandle,
uint32_t *phandle,
uint32_t *intc_phandles,
-uint32_t *aplic_phandles)
+uint32_t *aplic_phandles,
+int num_harts)
{
char *aplic_name;
unsigned long aplic_addr;
@@ -681,7 +688,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_M].size,
msi_m_phandle, intc_phandles,
aplic_m_phandle, aplic_s_phandle,
- true);
+ true, num_harts);
}
/* S-level APLIC node */
@@ -690,7 +697,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_S].size,
msi_s_phandle, intc_phandles,
aplic_s_phandle, 0,
- false);
+ false, num_harts);
aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr);
@@ -774,34 +781,51 @@ static void create_fdt_sockets(RISCVVirtState *s, const
MemMapEntry *memmap,
*msi_pcie_phandle = msi_s_phandle;
}
-phandle_pos = ms->smp.cpus;
-for (socket = (socket_count - 1); socket >= 0; socket--) {
-phandle_pos -= s->soc[socket].num_harts;
-
-if (s->aia_type == VIRT_AIA_TYPE_NONE) {
-create_fdt_socket_plic(s, memmap, socket, phandle,
-_phandles[phandle_pos], xplic_phandles);
-} else {
-cre
[PATCH v6 2/5] target/riscv: check the in-kernel irqchip support
We check the in-kernel irqchip support when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
target/riscv/kvm.c | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 9d8a8982f9..005e054604 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -914,7 +914,15 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
int kvm_arch_irqchip_create(KVMState *s)
{
-return 0;
+if (kvm_kernel_irqchip_split()) {
+error_report("-machine kernel_irqchip=split is not supported on
RISC-V.");
+exit(1);
+}
+
+/*
+ * We can create the VAIA using the newer device control API.
+ */
+return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
}
int kvm_arch_process_async_events(CPUState *cs)
--
2.17.1
[PATCH v6 3/5] target/riscv: Create an KVM AIA irqchip
We create a vAIA chip by using the KVM_DEV_TYPE_RISCV_AIA and then set up
the chip with the KVM_DEV_RISCV_AIA_GRP_* APIs.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
target/riscv/kvm.c | 160 +++
target/riscv/kvm_riscv.h | 6 ++
2 files changed, 166 insertions(+)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 005e054604..9bc92cedff 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -36,6 +36,7 @@
#include "exec/address-spaces.h"
#include "hw/boards.h"
#include "hw/irq.h"
+#include "hw/intc/riscv_imsic.h"
#include "qemu/log.h"
#include "hw/loader.h"
#include "kvm_riscv.h"
@@ -43,6 +44,7 @@
#include "chardev/char-fe.h"
#include "migration/migration.h"
#include "sysemu/runstate.h"
+#include "hw/riscv/numa.h"
static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,
uint64_t idx)
@@ -1026,3 +1028,161 @@ bool kvm_arch_cpu_check_are_resettable(void)
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
+
+char *kvm_aia_mode_str(uint64_t aia_mode)
+{
+const char *val;
+
+switch (aia_mode) {
+case KVM_DEV_RISCV_AIA_MODE_EMUL:
+return "emul";
+case KVM_DEV_RISCV_AIA_MODE_HWACCEL:
+return "hwaccel";
+case KVM_DEV_RISCV_AIA_MODE_AUTO:
+default:
+return "auto";
+};
+}
+
+void kvm_riscv_aia_create(MachineState *machine,
+ uint64_t aia_mode, uint64_t group_shift,
+ uint64_t aia_irq_num, uint64_t aia_msi_num,
+ uint64_t aplic_base, uint64_t imsic_base,
+ uint64_t guest_num)
+{
+int ret, i;
+int aia_fd = -1;
+uint64_t default_aia_mode;
+uint64_t socket_count = riscv_socket_count(machine);
+uint64_t max_hart_per_socket = 0;
+uint64_t socket, base_hart, hart_count, socket_imsic_base, imsic_addr;
+uint64_t socket_bits, hart_bits, guest_bits;
+
+aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
+
+if (aia_fd < 0) {
+error_report("Unable to create in-kernel irqchip");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_aia_mode, false, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to get current KVM AIA mode");
+exit(1);
+}
+qemu_log("KVM AIA: default mode is %s\n",
+ kvm_aia_mode_str(default_aia_mode));
+
+if (default_aia_mode != aia_mode) {
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_mode, true, NULL);
+if (ret < 0)
+warn_report("KVM AIA: failed to set KVM AIA mode");
+else
+qemu_log("KVM AIA: set current mode to %s\n",
+ kvm_aia_mode_str(aia_mode));
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_SRCS,
+_irq_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set number of input irq lines");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_IDS,
+_msi_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set number of msi");
+exit(1);
+}
+
+socket_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS,
+_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set group_bits");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT,
+_shift, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set group_shift");
+exit(1);
+}
+
+guest_bits = guest_num == 0 ? 0 :
+ find_last_bit(_num, BITS_PER_LONG) + 1;
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GUEST_BITS,
+_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set guest_bits");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RI
[PATCH v6 0/5] Add RISC-V KVM AIA Support
This series adds support for KVM AIA in RISC-V architecture. In order to test these patches, we require Linux with KVM AIA support which can be found in the riscv_kvm_aia_hwaccel_v1 branch at https://github.com/avpatel/linux.git --- v6: - fix alignment - add hart index to the error message of ISMIC address setting in PATCH3 v5: - remove the linux-header update patch since the riscv-to-apply.next QEMU has synced up to Linux 6.5-rc1 headers. - create the APLIC and IMSIC FDT helper functions in PATCH1 - add the irqfd support in PATCH3 - fix the comments and refine the code v4: - update the linux header by the scripts/update-linux-headers.sh in PATCH1 - remove the checking for "aplic_m" before creating S-mode APLIC device in PATCH2 - add more setting when we initialize the KVM AIA chip in PATCH4 - fix msi message delivery and the APLIC devices emulation in PATCH5 - fix the AIA devices mapping with NUMA enabled in PATCH6 - add "kvm-aia" parameter to sepecify the KVM AIA mode in PATCH6 v3: - fix typo - tag the linux-header patch as placeholder v2: - rebase to riscv-to-apply.next - update the linux header by the scripts/update-linux-headers.sh Yong-Xuan Wang (5): target/riscv: support the AIA device emulation with KVM enabled target/riscv: check the in-kernel irqchip support target/riscv: Create an KVM AIA irqchip target/riscv: update APLIC and IMSIC to support KVM AIA target/riscv: select KVM AIA in riscv virt machine hw/intc/riscv_aplic.c| 56 -- hw/intc/riscv_imsic.c| 25 ++- hw/riscv/virt.c | 410 ++- include/hw/riscv/virt.h | 1 + target/riscv/kvm.c | 170 +++- target/riscv/kvm_riscv.h | 6 + 6 files changed, 469 insertions(+), 199 deletions(-) -- 2.17.1
[PATCH v5 3/5] target/riscv: Create an KVM AIA irqchip
We create a vAIA chip by using the KVM_DEV_TYPE_RISCV_AIA and then set up
the chip with the KVM_DEV_RISCV_AIA_GRP_* APIs.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
---
target/riscv/kvm.c | 160 +++
target/riscv/kvm_riscv.h | 6 ++
2 files changed, 166 insertions(+)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 005e054604..64156c15ec 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -36,6 +36,7 @@
#include "exec/address-spaces.h"
#include "hw/boards.h"
#include "hw/irq.h"
+#include "hw/intc/riscv_imsic.h"
#include "qemu/log.h"
#include "hw/loader.h"
#include "kvm_riscv.h"
@@ -43,6 +44,7 @@
#include "chardev/char-fe.h"
#include "migration/migration.h"
#include "sysemu/runstate.h"
+#include "hw/riscv/numa.h"
static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,
uint64_t idx)
@@ -1026,3 +1028,161 @@ bool kvm_arch_cpu_check_are_resettable(void)
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
+
+char *kvm_aia_mode_str(uint64_t aia_mode)
+{
+const char *val;
+
+switch (aia_mode) {
+case KVM_DEV_RISCV_AIA_MODE_EMUL:
+return "emul";
+case KVM_DEV_RISCV_AIA_MODE_HWACCEL:
+return "hwaccel";
+case KVM_DEV_RISCV_AIA_MODE_AUTO:
+default:
+return "auto";
+};
+}
+
+void kvm_riscv_aia_create(MachineState *machine,
+ uint64_t aia_mode, uint64_t group_shift,
+ uint64_t aia_irq_num, uint64_t aia_msi_num,
+ uint64_t aplic_base, uint64_t imsic_base,
+ uint64_t guest_num)
+{
+int ret, i;
+int aia_fd = -1;
+uint64_t default_aia_mode;
+uint64_t socket_count = riscv_socket_count(machine);
+uint64_t max_hart_per_socket = 0;
+uint64_t socket, base_hart, hart_count, socket_imsic_base, imsic_addr;
+uint64_t socket_bits, hart_bits, guest_bits;
+
+aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
+
+if (aia_fd < 0) {
+error_report("Unable to create in-kernel irqchip");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_aia_mode, false, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to get current KVM AIA mode");
+exit(1);
+}
+qemu_log("KVM AIA: default mode is %s\n",
+ kvm_aia_mode_str(default_aia_mode));
+
+if (default_aia_mode != aia_mode) {
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_mode, true, NULL);
+if (ret < 0)
+warn_report("KVM AIA: failed to set KVM AIA mode");
+else
+qemu_log("KVM AIA: set current mode to %s\n",
+ kvm_aia_mode_str(aia_mode));
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_SRCS,
+_irq_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set number of input irq lines");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_IDS,
+_msi_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set number of msi");
+exit(1);
+}
+
+socket_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS,
+_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set group_bits");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT,
+_shift, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set group_shift");
+exit(1);
+}
+
+guest_bits = guest_num == 0 ? 0 :
+ find_last_bit(_num, BITS_PER_LONG) + 1;
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GUEST_BITS,
+_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: failed to set guest_bits");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+
[PATCH v5 1/5] target/riscv: support the AIA device emulation with KVM enabled
In this patch, we create the APLIC and IMSIC FDT helper functions and
remove M mode AIA devices when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
---
hw/riscv/virt.c | 264 ++--
1 file changed, 123 insertions(+), 141 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index d90286dc46..26b3aff28e 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -516,79 +516,28 @@ static uint32_t imsic_num_bits(uint32_t count)
return ret;
}
-static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
- uint32_t *phandle, uint32_t *intc_phandles,
- uint32_t *msi_m_phandle, uint32_t *msi_s_phandle)
+static void create_fdt_one_imsic(RISCVVirtState *s, hwaddr base_addr,
+ uint32_t *intc_phandles, uint32_t msi_phandle,
+ bool m_mode, uint32_t imsic_guest_bits)
{
int cpu, socket;
char *imsic_name;
MachineState *ms = MACHINE(s);
int socket_count = riscv_socket_count(ms);
-uint32_t imsic_max_hart_per_socket, imsic_guest_bits;
+uint32_t imsic_max_hart_per_socket;
uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size;
-*msi_m_phandle = (*phandle)++;
-*msi_s_phandle = (*phandle)++;
imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
imsic_regs = g_new0(uint32_t, socket_count * 4);
-/* M-level IMSIC node */
for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
-}
-imsic_max_hart_per_socket = 0;
-for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_M].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
-imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
-imsic_regs[socket * 4 + 0] = 0;
-imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
-imsic_regs[socket * 4 + 2] = 0;
-imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
-if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
-imsic_max_hart_per_socket = s->soc[socket].num_harts;
-}
-}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_M].base);
-qemu_fdt_add_subnode(ms->fdt, imsic_name);
-qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
-"riscv,imsics");
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
-FDT_IMSIC_INT_CELLS);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
-qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
-socket_count * sizeof(uint32_t) * 4);
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
-VIRT_IRQCHIP_NUM_MSIS);
-if (socket_count > 1) {
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits",
-imsic_num_bits(imsic_max_hart_per_socket));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits",
-imsic_num_bits(socket_count));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift",
-IMSIC_MMIO_GROUP_MIN_SHIFT);
+imsic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT);
}
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle);
-g_free(imsic_name);
-
-/* S-level IMSIC node */
-for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
-imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
-}
-imsic_guest_bits = imsic_num_bits(s->aia_guests + 1);
imsic_max_hart_per_socket = 0;
for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_S].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
+imsic_addr = base_addr + socket * VIRT_IMSIC_GROUP_MAX_SIZE;
imsic_size = IMSIC_HART_SIZE(imsic_guest_bits) *
s->soc[socket].num_harts;
imsic_regs[socket * 4 + 0] = 0;
@@ -599,119 +548,149 @@ static void create_fdt_imsic(RISCVVirtState *s, const
MemMapEntry *memmap,
imsic_max_hart_per_socket = s->soc[socket].num_harts;
}
}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_S]
[PATCH v5 4/5] target/riscv: update APLIC and IMSIC to support KVM AIA
KVM AIA can't emulate APLIC only. When "aia=aplic" parameter is passed,
APLIC devices is emulated by QEMU. For "aia=aplic-imsic", remove the
mmio operations of APLIC when using KVM AIA and send wired interrupt
signal via KVM_IRQ_LINE API.
After KVM AIA enabled, MSI messages are delivered by KVM_SIGNAL_MSI API
when the IMSICs receive mmio write requests.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
Reviewed-by: Andrew Jones
---
hw/intc/riscv_aplic.c | 56 ++-
hw/intc/riscv_imsic.c | 25 +++
2 files changed, 61 insertions(+), 20 deletions(-)
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index 4bdc6a5d1a..eab5db053b 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -31,6 +31,7 @@
#include "hw/irq.h"
#include "target/riscv/cpu.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define APLIC_MAX_IDC (1UL << 14)
@@ -148,6 +149,15 @@
#define APLIC_IDC_CLAIMI 0x1c
+/*
+ * KVM AIA only supports APLIC MSI, fallback to QEMU emulation if we want to
use
+ * APLIC Wired.
+ */
+static bool is_kvm_aia(bool msimode)
+{
+return kvm_irqchip_in_kernel() && msimode;
+}
+
static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
uint32_t word)
{
@@ -471,6 +481,11 @@ static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState
*aplic, uint32_t idc)
return topi;
}
+static void riscv_kvm_aplic_request(void *opaque, int irq, int level)
+{
+kvm_set_irq(kvm_state, irq, !!level);
+}
+
static void riscv_aplic_request(void *opaque, int irq, int level)
{
bool update = false;
@@ -801,29 +816,35 @@ static void riscv_aplic_realize(DeviceState *dev, Error
**errp)
uint32_t i;
RISCVAPLICState *aplic = RISCV_APLIC(dev);
-aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
-aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
-aplic->state = g_new0(uint32_t, aplic->num_irqs);
-aplic->target = g_new0(uint32_t, aplic->num_irqs);
-if (!aplic->msimode) {
-for (i = 0; i < aplic->num_irqs; i++) {
-aplic->target[i] = 1;
+if (!is_kvm_aia(aplic->msimode)) {
+aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
+aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
+aplic->state = g_new0(uint32_t, aplic->num_irqs);
+aplic->target = g_new0(uint32_t, aplic->num_irqs);
+if (!aplic->msimode) {
+for (i = 0; i < aplic->num_irqs; i++) {
+aplic->target[i] = 1;
+}
}
-}
-aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
-aplic->iforce = g_new0(uint32_t, aplic->num_harts);
-aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
+aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
+aplic->iforce = g_new0(uint32_t, aplic->num_harts);
+aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
-memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops, aplic,
- TYPE_RISCV_APLIC, aplic->aperture_size);
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops,
+ aplic, TYPE_RISCV_APLIC, aplic->aperture_size);
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/*
* Only root APLICs have hardware IRQ lines. All non-root APLICs
* have IRQ lines delegated by their parent APLIC.
*/
if (!aplic->parent) {
-qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+if (is_kvm_aia(aplic->msimode)) {
+qdev_init_gpio_in(dev, riscv_kvm_aplic_request, aplic->num_irqs);
+} else {
+qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+}
}
/* Create output IRQ lines for non-MSI mode */
@@ -958,7 +979,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
qdev_prop_set_bit(dev, "mmode", mmode);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!is_kvm_aia(msimode)) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
if (parent) {
riscv_aplic_add_child(parent, dev);
diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
index fea3385b51..760dbddcf7 100644
--- a/hw/intc/riscv_imsic.c
+++ b/hw/intc/riscv_imsic.c
@@ -32,6 +32,7 @@
#include "target/riscv/cpu.h"
#include "target/riscv/cpu_bits.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/v
[PATCH v5 2/5] target/riscv: check the in-kernel irqchip support
We check the in-kernel irqchip support when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
---
target/riscv/kvm.c | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 9d8a8982f9..005e054604 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -914,7 +914,15 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
int kvm_arch_irqchip_create(KVMState *s)
{
-return 0;
+if (kvm_kernel_irqchip_split()) {
+error_report("-machine kernel_irqchip=split is not supported on
RISC-V.");
+exit(1);
+}
+
+/*
+ * We can create the VAIA using the newer device control API.
+ */
+return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
}
int kvm_arch_process_async_events(CPUState *cs)
--
2.17.1
[PATCH v5 0/5] Add RISC-V KVM AIA Support
This series adds support for KVM AIA in RISC-V architecture. In order to test these patches, we require Linux with KVM AIA support which can be found in the riscv_kvm_aia_hwaccel_v1 branch at https://github.com/avpatel/linux.git --- v5: - remove the linux-header update patch since the riscv-to-apply.next QEMU has synced up to Linux 6.5-rc1 headers. - create the APLIC and IMSIC FDT helper functions in PATCH1 - add the irqfd support in PATCH3 - fix the comments and refine the code v4: - update the linux header by the scripts/update-linux-headers.sh in PATCH1 - remove the checking for "aplic_m" before creating S-mode APLIC device in PATCH2 - add more setting when we initialize the KVM AIA chip in PATCH4 - fix msi message delivery and the APLIC devices emulation in PATCH5 - fix the AIA devices mapping with NUMA enabled in PATCH6 - add "kvm-aia" parameter to sepecify the KVM AIA mode in PATCH6 v3: - fix typo - tag the linux-header patch as placeholder v2: - rebase to riscv-to-apply.next - update the linux header by the scripts/update-linux-headers.sh Yong-Xuan Wang (5): target/riscv: support the AIA device emulation with KVM enabled target/riscv: check the in-kernel irqchip support target/riscv: Create an KVM AIA irqchip target/riscv: update APLIC and IMSIC to support KVM AIA target/riscv: select KVM AIA in riscv virt machine hw/intc/riscv_aplic.c| 56 -- hw/intc/riscv_imsic.c| 25 ++- hw/riscv/virt.c | 381 ++- include/hw/riscv/virt.h | 1 + target/riscv/kvm.c | 170 - target/riscv/kvm_riscv.h | 6 + 6 files changed, 452 insertions(+), 187 deletions(-) -- 2.17.1
[PATCH v5 5/5] target/riscv: select KVM AIA in riscv virt machine
Select KVM AIA when the host kernel has in-kernel AIA chip support.
Since KVM AIA only has one APLIC instance, we map the QEMU APLIC
devices to KVM APLIC.
We also extend virt machine to specify the KVM AIA mode. The "kvm-aia"
parameter is passed along with machine name in QEMU command-line.
1) "kvm-aia=emul": IMSIC is emulated by hypervisor
2) "kvm-aia=hwaccel": use hardware guest IMSIC
3) "kvm-aia=auto": use the hardware guest IMSICs whenever available
otherwise we fallback to software emulation.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
---
hw/riscv/virt.c | 125 ++--
include/hw/riscv/virt.h | 1 +
2 files changed, 97 insertions(+), 29 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 26b3aff28e..b74142d978 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
#include "hw/riscv/virt.h"
#include "hw/riscv/boot.h"
#include "hw/riscv/numa.h"
+#include "kvm_riscv.h"
#include "hw/intc/riscv_aclint.h"
#include "hw/intc/riscv_aplic.h"
#include "hw/intc/riscv_imsic.h"
@@ -75,6 +76,12 @@
#error "Can't accomodate all IMSIC groups in address space"
#endif
+/* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */
+static bool virt_use_kvm_aia(RISCVVirtState *s)
+{
+return kvm_irqchip_in_kernel() && s->aia_type == VIRT_AIA_TYPE_APLIC_IMSIC;
+}
+
static const MemMapEntry virt_memmap[] = {
[VIRT_DEBUG] ={0x0, 0x100 },
[VIRT_MROM] = { 0x1000,0xf000 },
@@ -609,16 +616,16 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int
socket,
uint32_t *intc_phandles,
uint32_t aplic_phandle,
uint32_t aplic_child_phandle,
- bool m_mode)
+ bool m_mode, int num_harts)
{
int cpu;
char *aplic_name;
uint32_t *aplic_cells;
MachineState *ms = MACHINE(s);
-aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+aplic_cells = g_new0(uint32_t, num_harts * 2);
-for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
+for (cpu = 0; cpu < num_harts; cpu++) {
aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
aplic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT);
}
@@ -632,7 +639,7 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int
socket,
if (s->aia_type == VIRT_AIA_TYPE_APLIC) {
qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended",
-aplic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 2);
+aplic_cells, num_harts * sizeof(uint32_t) * 2);
} else {
qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", msi_phandle);
}
@@ -662,7 +669,8 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
uint32_t msi_s_phandle,
uint32_t *phandle,
uint32_t *intc_phandles,
-uint32_t *aplic_phandles)
+uint32_t *aplic_phandles,
+int num_harts)
{
char *aplic_name;
unsigned long aplic_addr;
@@ -679,7 +687,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_M].size,
msi_m_phandle, intc_phandles,
aplic_m_phandle, aplic_s_phandle,
- true);
+ true, num_harts);
}
/* S-level APLIC node */
@@ -688,7 +696,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_S].size,
msi_s_phandle, intc_phandles,
aplic_s_phandle, 0,
- false);
+ false, num_harts);
aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr);
@@ -772,34 +780,48 @@ static void create_fdt_sockets(RISCVVirtState *s, const
MemMapEntry *memmap,
*msi_pcie_phandle = msi_s_phandle;
}
-phandle_pos = ms->smp.cpus;
-for (socket = (socket_count - 1); socket >= 0; socket--) {
-phandle_pos -= s->soc[socket].num_harts;
+/* KVM AIA only has one APLIC instance */
+if (virt_use_kvm_aia(s)) {
+create_fdt_socket_aplic(s, memmap, 0,
+msi_m_phandle, msi_s_phandle, phandle,
+_phandles[0], xplic_phandles, ms->smp.cpus);
+} else {
+phandle_pos = ms->smp.cpus;
+
Re: [PATCH v4 0/6] Add RISC-V KVM AIA Support
Hi Andrew,
I'll add it into patch v5. Thank you!
Regards,
Yong-Xuan
On Wed, Jul 5, 2023 at 4:14 PM Andrew Jones wrote:
>
> On Wed, Jun 21, 2023 at 02:54:50PM +, Yong-Xuan Wang wrote:
> > This series adds support for KVM AIA in RISC-V architecture.
> >
> > In order to test these patches, we require Linux with KVM AIA support which
> > can
> > be found in the riscv_kvm_aia_hwaccel_v1 branch at
> > https://github.com/avpatel/linux.git
>
> Is it possible to add irqfd support? Maybe even as simply as the diff
> below? If not, can you explain what still needs to be done in order
> to do so?
>
> Thanks,
> drew
>
>
> diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
> index 00fdb60fc6ab..b38d9bb5506a 100644
> --- a/hw/intc/riscv_imsic.c
> +++ b/hw/intc/riscv_imsic.c
> @@ -371,6 +371,7 @@ static void riscv_imsic_realize(DeviceState *dev, Error
> **errp)
> }
>
> msi_nonbroken = true;
> +kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
> }
>
> static Property riscv_imsic_properties[] = {
>
[PATCH v4 4/6] target/riscv: Create an KVM AIA irqchip
implement a function to create an KVM AIA chip
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm.c | 163 +++
target/riscv/kvm_riscv.h | 6 ++
2 files changed, 169 insertions(+)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index eb469e8ca5..3dd8467031 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -34,6 +34,7 @@
#include "exec/address-spaces.h"
#include "hw/boards.h"
#include "hw/irq.h"
+#include "hw/intc/riscv_imsic.h"
#include "qemu/log.h"
#include "hw/loader.h"
#include "kvm_riscv.h"
@@ -41,6 +42,7 @@
#include "chardev/char-fe.h"
#include "migration/migration.h"
#include "sysemu/runstate.h"
+#include "hw/riscv/numa.h"
static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,
uint64_t idx)
@@ -548,3 +550,164 @@ bool kvm_arch_cpu_check_are_resettable(void)
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
+
+char *kvm_aia_mode_str(uint64_t aia_mode)
+{
+const char *val;
+
+switch (aia_mode) {
+case KVM_DEV_RISCV_AIA_MODE_EMUL:
+val = "emul";
+break;
+case KVM_DEV_RISCV_AIA_MODE_HWACCEL:
+val = "hwaccel";
+break;
+case KVM_DEV_RISCV_AIA_MODE_AUTO:
+default:
+val = "auto";
+break;
+};
+
+return g_strdup(val);
+}
+
+void kvm_riscv_aia_create(MachineState *machine,
+ uint64_t aia_mode, uint64_t group_shift,
+ uint64_t aia_irq_num, uint64_t aia_msi_num,
+ uint64_t aplic_base, uint64_t imsic_base,
+ uint64_t guest_num)
+{
+int ret, i;
+int aia_fd = -1;
+uint64_t default_aia_mode;
+uint64_t socket_count = riscv_socket_count(machine);
+uint64_t max_hart_per_socket = 0;
+uint64_t socket, base_hart, hart_count, socket_imsic_base, imsic_addr;
+uint64_t socket_bits, hart_bits, guest_bits;
+
+aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
+
+if (aia_fd < 0) {
+error_report("Unable to create in-kernel irqchip");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_aia_mode, false, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to get current KVM AIA mode");
+exit(1);
+}
+qemu_log("KVM AIA: default mode is %s\n",
+ kvm_aia_mode_str(default_aia_mode));
+
+if (default_aia_mode != aia_mode) {
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_mode, true, NULL);
+if (ret < 0)
+warn_report("KVM AIA: fail to set KVM AIA mode");
+else
+qemu_log("KVM AIA: set current mode to %s\n",
+ kvm_aia_mode_str(aia_mode));
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_SRCS,
+_irq_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number of input irq lines");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_IDS,
+_msi_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number of msi");
+exit(1);
+}
+
+socket_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS,
+_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set group_bits");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT,
+_shift, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set group_shift");
+exit(1);
+}
+
+guest_bits = guest_num == 0 ? 0 :
+ find_last_bit(_num, BITS_PER_LONG) + 1;
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_GUEST_BITS,
+_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set guest_bits");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+KVM_DEV_RISCV_AIA_ADDR_APLIC,
+
[PATCH v4 1/6] update-linux-headers: sync-up header with Linux for KVM AIA support placeholder
Sync-up Linux header to get latest KVM RISC-V headers having AIA support.
Note: This is a placeholder commit and could be replaced when all referenced
Linux patchsets are mainlined.
The linux-headers changes are from 2 different patchsets.
[1]
https://lore.kernel.org/lkml/20230404153452.2405681-1-apa...@ventanamicro.com/
[2] https://www.spinics.net/lists/kernel/msg4791872.html
Currently, patchset 1 is already merged into mainline kernel in v6.4-rc1 and
patchset 2 is not.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
linux-headers/asm-riscv/kvm.h | 123 +-
linux-headers/linux/kvm.h | 2 +
2 files changed, 124 insertions(+), 1 deletion(-)
diff --git a/linux-headers/asm-riscv/kvm.h b/linux-headers/asm-riscv/kvm.h
index 92af6f3f05..a16ca62419 100644
--- a/linux-headers/asm-riscv/kvm.h
+++ b/linux-headers/asm-riscv/kvm.h
@@ -12,8 +12,10 @@
#ifndef __ASSEMBLY__
#include
+#include
#include
+#define __KVM_HAVE_IRQ_LINE
#define __KVM_HAVE_READONLY_MEM
#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
@@ -64,7 +66,7 @@ struct kvm_riscv_core {
#define KVM_RISCV_MODE_S 1
#define KVM_RISCV_MODE_U 0
-/* CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+/* General CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
struct kvm_riscv_csr {
unsigned long sstatus;
unsigned long sie;
@@ -78,6 +80,17 @@ struct kvm_riscv_csr {
unsigned long scounteren;
};
+/* AIA CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_aia_csr {
+ unsigned long siselect;
+ unsigned long iprio1;
+ unsigned long iprio2;
+ unsigned long sieh;
+ unsigned long siph;
+ unsigned long iprio1h;
+ unsigned long iprio2h;
+};
+
/* TIMER registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
struct kvm_riscv_timer {
__u64 frequency;
@@ -105,9 +118,28 @@ enum KVM_RISCV_ISA_EXT_ID {
KVM_RISCV_ISA_EXT_SVINVAL,
KVM_RISCV_ISA_EXT_ZIHINTPAUSE,
KVM_RISCV_ISA_EXT_ZICBOM,
+ KVM_RISCV_ISA_EXT_ZBB,
+ KVM_RISCV_ISA_EXT_SSAIA,
KVM_RISCV_ISA_EXT_MAX,
};
+/*
+ * SBI extension IDs specific to KVM. This is not the same as the SBI
+ * extension IDs defined by the RISC-V SBI specification.
+ */
+enum KVM_RISCV_SBI_EXT_ID {
+ KVM_RISCV_SBI_EXT_V01 = 0,
+ KVM_RISCV_SBI_EXT_TIME,
+ KVM_RISCV_SBI_EXT_IPI,
+ KVM_RISCV_SBI_EXT_RFENCE,
+ KVM_RISCV_SBI_EXT_SRST,
+ KVM_RISCV_SBI_EXT_HSM,
+ KVM_RISCV_SBI_EXT_PMU,
+ KVM_RISCV_SBI_EXT_EXPERIMENTAL,
+ KVM_RISCV_SBI_EXT_VENDOR,
+ KVM_RISCV_SBI_EXT_MAX,
+};
+
/* Possible states for kvm_riscv_timer */
#define KVM_RISCV_TIMER_STATE_OFF 0
#define KVM_RISCV_TIMER_STATE_ON 1
@@ -118,6 +150,8 @@ enum KVM_RISCV_ISA_EXT_ID {
/* If you need to interpret the index values, here is the key: */
#define KVM_REG_RISCV_TYPE_MASK0xFF00
#define KVM_REG_RISCV_TYPE_SHIFT 24
+#define KVM_REG_RISCV_SUBTYPE_MASK 0x00FF
+#define KVM_REG_RISCV_SUBTYPE_SHIFT16
/* Config registers are mapped as type 1 */
#define KVM_REG_RISCV_CONFIG (0x01 << KVM_REG_RISCV_TYPE_SHIFT)
@@ -131,8 +165,12 @@ enum KVM_RISCV_ISA_EXT_ID {
/* Control and status registers are mapped as type 3 */
#define KVM_REG_RISCV_CSR (0x03 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_CSR_GENERAL (0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_CSR_AIA (0x1 << KVM_REG_RISCV_SUBTYPE_SHIFT)
#define KVM_REG_RISCV_CSR_REG(name)\
(offsetof(struct kvm_riscv_csr, name) / sizeof(unsigned long))
+#define KVM_REG_RISCV_CSR_AIA_REG(name)\
+ (offsetof(struct kvm_riscv_aia_csr, name) / sizeof(unsigned long))
/* Timer registers are mapped as type 4 */
#define KVM_REG_RISCV_TIMER(0x04 << KVM_REG_RISCV_TYPE_SHIFT)
@@ -152,6 +190,89 @@ enum KVM_RISCV_ISA_EXT_ID {
/* ISA Extension registers are mapped as type 7 */
#define KVM_REG_RISCV_ISA_EXT (0x07 << KVM_REG_RISCV_TYPE_SHIFT)
+/* SBI extension registers are mapped as type 8 */
+#define KVM_REG_RISCV_SBI_EXT (0x08 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_SINGLE (0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_MULTI_EN (0x1 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_MULTI_DIS(0x2 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_MULTI_REG(__ext_id) \
+ ((__ext_id) / __BITS_PER_LONG)
+#define KVM_REG_RISCV_SBI_MULTI_MASK(__ext_id) \
+ (1UL << ((__ext_id) % __BITS_PER_LONG))
+#define KVM_REG_RISCV_SBI_MULTI_REG_LAST \
+ KVM_REG_RISCV_SBI_MULTI_REG(KVM_RISCV_SBI_EXT_MAX - 1)
+
+/* Device Control API: RISC-V AIA */
+#define KVM_DEV_RISCV_APLIC_ALIGN 0x1000
+#define KVM_DEV_RISCV_APLI
[PATCH v4 5/6] target/riscv: update APLIC and IMSIC to support KVM AIA
KVM AIA can't emulate APLIC only. When "aia=aplic" parameter is passed,
APLIC devices is emulated by QEMU. For "aia=aplic-imsic", remove the
mmio operations of APLIC when using KVM AIA and send wired interrupt
signal via KVM_IRQ_LINE API.
After KVM AIA enabled, MSI messages are delivered by KVM_SIGNAL_MSI API
when the IMSICs receive mmio write requests.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/intc/riscv_aplic.c | 58 +++
hw/intc/riscv_imsic.c | 26 +++
2 files changed, 63 insertions(+), 21 deletions(-)
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index afc5b54dbb..b1949636b6 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -31,6 +31,7 @@
#include "hw/irq.h"
#include "target/riscv/cpu.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define APLIC_MAX_IDC (1UL << 14)
@@ -148,6 +149,15 @@
#define APLIC_IDC_CLAIMI 0x1c
+/*
+ * KVM AIA only supports APLIC.m, fallback to QEMU emulation if we want to use
+ * APLIC.w.
+ */
+static bool is_kvm_aia(bool msimode)
+{
+return kvm_irqchip_in_kernel() && msimode;
+}
+
static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
uint32_t word)
{
@@ -471,6 +481,13 @@ static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState
*aplic, uint32_t idc)
return topi;
}
+static void riscv_kvm_aplic_request(void *opaque, int irq, int level)
+{
+kvm_set_irq(kvm_state, irq, !!level);
+
+return;
+}
+
static void riscv_aplic_request(void *opaque, int irq, int level)
{
bool update = false;
@@ -801,29 +818,35 @@ static void riscv_aplic_realize(DeviceState *dev, Error
**errp)
uint32_t i;
RISCVAPLICState *aplic = RISCV_APLIC(dev);
-aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
-aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
-aplic->state = g_new0(uint32_t, aplic->num_irqs);
-aplic->target = g_new0(uint32_t, aplic->num_irqs);
-if (!aplic->msimode) {
-for (i = 0; i < aplic->num_irqs; i++) {
-aplic->target[i] = 1;
+if (!is_kvm_aia(aplic->msimode)) {
+aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
+aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
+aplic->state = g_new0(uint32_t, aplic->num_irqs);
+aplic->target = g_new0(uint32_t, aplic->num_irqs);
+if (!aplic->msimode) {
+for (i = 0; i < aplic->num_irqs; i++) {
+aplic->target[i] = 1;
+}
}
-}
-aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
-aplic->iforce = g_new0(uint32_t, aplic->num_harts);
-aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
+aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
+aplic->iforce = g_new0(uint32_t, aplic->num_harts);
+aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
-memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops, aplic,
- TYPE_RISCV_APLIC, aplic->aperture_size);
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops,
+ aplic, TYPE_RISCV_APLIC, aplic->aperture_size);
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/*
* Only root APLICs have hardware IRQ lines. All non-root APLICs
* have IRQ lines delegated by their parent APLIC.
*/
if (!aplic->parent) {
-qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+if (is_kvm_aia(aplic->msimode)) {
+qdev_init_gpio_in(dev, riscv_kvm_aplic_request, aplic->num_irqs);
+} else {
+qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
+}
}
/* Create output IRQ lines for non-MSI mode */
@@ -958,7 +981,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
qdev_prop_set_bit(dev, "mmode", mmode);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!is_kvm_aia(msimode)) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
if (parent) {
riscv_aplic_add_child(parent, dev);
diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
index fea3385b51..00fdb60fc6 100644
--- a/hw/intc/riscv_imsic.c
+++ b/hw/intc/riscv_imsic.c
@@ -32,6 +32,7 @@
#include "target/riscv/cpu.h"
#include "target/riscv/cpu_bits.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define IMSIC_MMIO_PAGE_L
[PATCH v4 6/6] target/riscv: select KVM AIA in riscv virt machine
Select KVM AIA when the host kernel has in-kernel AIA chip support.
Since KVM AIA only has one APLIC instance, we map the QEMU APLIC
devices to KVM APLIC.
We also extend virt machine to specify the KVM AIA mode. The "kvm-aia"
parameter is passed along with machine name in QEMU command-line.
1) "kvm-aia=emul": IMSIC is emulated by hypervisor
2) "kvm-aia=hwaccel": use hardware guest IMSIC
3) "kvm-aia=auto": use the hardware guest IMSICs whenever available
otherwise we fallback to software emulation.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 92 ++---
include/hw/riscv/virt.h | 1 +
2 files changed, 79 insertions(+), 14 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 4a1d29a741..efa176a184 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
#include "hw/riscv/virt.h"
#include "hw/riscv/boot.h"
#include "hw/riscv/numa.h"
+#include "kvm_riscv.h"
#include "hw/intc/riscv_aclint.h"
#include "hw/intc/riscv_aplic.h"
#include "hw/intc/riscv_imsic.h"
@@ -74,6 +75,12 @@
#error "Can't accomodate all IMSIC groups in address space"
#endif
+/* KVM AIA only supports APLIC.m. APLIC.w is always emulated by QEMU. */
+static bool virt_use_kvm_aia(RISCVVirtState *s)
+{
+return kvm_irqchip_in_kernel() && s->aia_type == VIRT_AIA_TYPE_APLIC_IMSIC;
+}
+
static const MemMapEntry virt_memmap[] = {
[VIRT_DEBUG] ={0x0, 0x100 },
[VIRT_MROM] = { 0x1000,0xf000 },
@@ -642,7 +649,8 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
uint32_t msi_s_phandle,
uint32_t *phandle,
uint32_t *intc_phandles,
-uint32_t *aplic_phandles)
+uint32_t *aplic_phandles,
+int num_harts)
{
int cpu;
char *aplic_name;
@@ -653,11 +661,11 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
aplic_m_phandle = (*phandle)++;
aplic_s_phandle = (*phandle)++;
-aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+aplic_cells = g_new0(uint32_t, num_harts * 2);
if (!kvm_enabled()) {
/* M-level APLIC node */
-for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
+for (cpu = 0; cpu < num_harts; cpu++) {
aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
aplic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
}
@@ -691,7 +699,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
}
/* S-level APLIC node */
-for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
+for (cpu = 0; cpu < num_harts; cpu++) {
aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
aplic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
}
@@ -798,17 +806,25 @@ static void create_fdt_sockets(RISCVVirtState *s, const
MemMapEntry *memmap,
*msi_pcie_phandle = msi_s_phandle;
}
-phandle_pos = ms->smp.cpus;
-for (socket = (socket_count - 1); socket >= 0; socket--) {
-phandle_pos -= s->soc[socket].num_harts;
+/* KVM AIA only has one APLIC instance */
+if (virt_use_kvm_aia(s)) {
+create_fdt_socket_aplic(s, memmap, 0,
+msi_m_phandle, msi_s_phandle, phandle,
+_phandles[0], xplic_phandles, ms->smp.cpus);
+} else {
+phandle_pos = ms->smp.cpus;
+for (socket = (socket_count - 1); socket >= 0; socket--) {
+phandle_pos -= s->soc[socket].num_harts;
-if (s->aia_type == VIRT_AIA_TYPE_NONE) {
-create_fdt_socket_plic(s, memmap, socket, phandle,
-_phandles[phandle_pos], xplic_phandles);
-} else {
-create_fdt_socket_aplic(s, memmap, socket,
-msi_m_phandle, msi_s_phandle, phandle,
-_phandles[phandle_pos], xplic_phandles);
+if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+create_fdt_socket_plic(s, memmap, socket, phandle,
+_phandles[phandle_pos], xplic_phandles);
+} else {
+create_fdt_socket_aplic(s, memmap, socket,
+msi_m_phandle, msi_s_phandle, phandle,
+_phandles[phandle_pos], xplic_phandles,
+s->soc[socket].num_harts);
+}
}
}
@@ -819,6 +835,9 @@ static void create_fdt_sockets(RISCVVirtState *s, const
MemMapEntry *memmap,
*irq_mmio_phandle = xplic_phandles[socket];
*irq_virtio_phandle = xplic_phandles[socket];
*irq_pcie_phandle = xplic_phandles[socket];
+
+
[PATCH v4 2/6] target/riscv: support the AIA device emulation with KVM enabled
Remove M mode AIA devices when using KVM acceleration
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 207 +---
1 file changed, 108 insertions(+), 99 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 245c7b97b2..4a1d29a741 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -531,52 +531,54 @@ static void create_fdt_imsic(RISCVVirtState *s, const
MemMapEntry *memmap,
imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
imsic_regs = g_new0(uint32_t, socket_count * 4);
-/* M-level IMSIC node */
-for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
-imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
-}
-imsic_max_hart_per_socket = 0;
-for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_M].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
-imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
-imsic_regs[socket * 4 + 0] = 0;
-imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
-imsic_regs[socket * 4 + 2] = 0;
-imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
-if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
-imsic_max_hart_per_socket = s->soc[socket].num_harts;
+if (!kvm_enabled()) {
+/* M-level IMSIC node */
+for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
}
-}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_M].base);
-qemu_fdt_add_subnode(ms->fdt, imsic_name);
-qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
-"riscv,imsics");
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
-FDT_IMSIC_INT_CELLS);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
-qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
-socket_count * sizeof(uint32_t) * 4);
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
-VIRT_IRQCHIP_NUM_MSIS);
-if (socket_count > 1) {
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits",
-imsic_num_bits(imsic_max_hart_per_socket));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits",
-imsic_num_bits(socket_count));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift",
-IMSIC_MMIO_GROUP_MIN_SHIFT);
-}
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle);
+imsic_max_hart_per_socket = 0;
+for (socket = 0; socket < socket_count; socket++) {
+imsic_addr = memmap[VIRT_IMSIC_M].base +
+ socket * VIRT_IMSIC_GROUP_MAX_SIZE;
+imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
+imsic_regs[socket * 4 + 0] = 0;
+imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
+imsic_regs[socket * 4 + 2] = 0;
+imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
+if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
+imsic_max_hart_per_socket = s->soc[socket].num_harts;
+}
+}
+imsic_name = g_strdup_printf("/soc/imsics@%lx",
+(unsigned long)memmap[VIRT_IMSIC_M].base);
+qemu_fdt_add_subnode(ms->fdt, imsic_name);
+qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
+"riscv,imsics");
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
+FDT_IMSIC_INT_CELLS);
+qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
+NULL, 0);
+qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
+NULL, 0);
+qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
+imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
+qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
+socket_count * sizeof(uint32_t) * 4);
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
+VIRT_IRQCHIP_NUM_MSIS);
+if (socket_count > 1) {
+qemu_fdt_setprop_cel
[PATCH v4 3/6] target/riscv: check the in-kernel irqchip support
We check the in-kernel irqchip support when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
Reviewed-by: Daniel Henrique Barboza
---
target/riscv/kvm.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 0f932a5b96..eb469e8ca5 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -433,7 +433,18 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
int kvm_arch_irqchip_create(KVMState *s)
{
-return 0;
+if (kvm_kernel_irqchip_split()) {
+error_report("-machine kernel_irqchip=split is not supported "
+ "on RISC-V.");
+exit(1);
+}
+
+/*
+ * If we can create the VAIA using the newer device control API, we
+ * let the device do this when it initializes itself, otherwise we
+ * fall back to the old API
+ */
+return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
}
int kvm_arch_process_async_events(CPUState *cs)
--
2.17.1
[PATCH v4 0/6] Add RISC-V KVM AIA Support
This series adds support for KVM AIA in RISC-V architecture. In order to test these patches, we require Linux with KVM AIA support which can be found in the riscv_kvm_aia_hwaccel_v1 branch at https://github.com/avpatel/linux.git --- v4: - update the linux header by the scripts/update-linux-headers.sh in PATCH1 - remove the checking for "aplic_m" before creating S-mode APLIC device in PATCH2 - add more setting when we initialize the KVM AIA chip in PATCH4 - fix msi message delivery and the APLIC devices emulation in PATCH5 - fix the AIA devices mapping with NUMA enabled in PATCH6 - add "kvm-aia" parameter to sepecify the KVM AIA mode in PATCH6 v3: - fix typo - tag the linux-header patch as placeholder v2: - rebase to riscv-to-apply.next - update the linux header by the scripts/update-linux-headers.sh Yong-Xuan Wang (6): update-linux-headers: sync-up header with Linux for KVM AIA support placeholder target/riscv: support the AIA device emulation with KVM enabled target/riscv: check the in-kernel irqchip support target/riscv: Create an KVM AIA irqchip target/riscv: update APLIC and IMSIC to support KVM AIA target/riscv: select KVM AIA in riscv virt machine hw/intc/riscv_aplic.c | 58 +-- hw/intc/riscv_imsic.c | 26 ++- hw/riscv/virt.c | 297 +- include/hw/riscv/virt.h | 1 + linux-headers/asm-riscv/kvm.h | 123 +- linux-headers/linux/kvm.h | 2 + target/riscv/kvm.c| 176 +++- target/riscv/kvm_riscv.h | 6 + 8 files changed, 554 insertions(+), 135 deletions(-) -- 2.17.1
Re: [PATCH v3 2/6] target/riscv: support the AIA device emulation with KVM enabled
Hi Daniel,
I think this checking can be removed too. Would you send a patch to
fix it? Or I can remove it in this patch.
Regards,
Yong-Xuan
On Tue, Jun 6, 2023 at 2:45 AM Daniel Henrique Barboza
wrote:
>
>
>
> On 5/26/23 03:25, Yong-Xuan Wang wrote:
> > Remove M mode AIA devices when using KVM acceleration
> >
> > Signed-off-by: Yong-Xuan Wang
> > Reviewed-by: Jim Shu
> > ---
> > hw/riscv/virt.c | 199 +---
> > 1 file changed, 105 insertions(+), 94 deletions(-)
> >
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index 4e3efbee16..18b94888ab 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -531,52 +531,54 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> > imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
> > imsic_regs = g_new0(uint32_t, socket_count * 4);
> >
> > -/* M-level IMSIC node */
> > -for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> > -imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
> > -imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
> > -}
> > -imsic_max_hart_per_socket = 0;
> > -for (socket = 0; socket < socket_count; socket++) {
> > -imsic_addr = memmap[VIRT_IMSIC_M].base +
> > - socket * VIRT_IMSIC_GROUP_MAX_SIZE;
> > -imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
> > -imsic_regs[socket * 4 + 0] = 0;
> > -imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
> > -imsic_regs[socket * 4 + 2] = 0;
> > -imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
> > -if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
> > -imsic_max_hart_per_socket = s->soc[socket].num_harts;
> > +if (!kvm_enabled()) {
> > +/* M-level IMSIC node */
> > +for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> > +imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
> > +imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
> > }
> > -}
> > -imsic_name = g_strdup_printf("/soc/imsics@%lx",
> > -(unsigned long)memmap[VIRT_IMSIC_M].base);
> > -qemu_fdt_add_subnode(ms->fdt, imsic_name);
> > -qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
> > -"riscv,imsics");
> > -qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
> > -FDT_IMSIC_INT_CELLS);
> > -qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
> > -NULL, 0);
> > -qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
> > -NULL, 0);
> > -qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
> > -imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
> > -qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
> > -socket_count * sizeof(uint32_t) * 4);
> > -qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
> > -VIRT_IRQCHIP_NUM_MSIS);
> > -if (socket_count > 1) {
> > -qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits",
> > -imsic_num_bits(imsic_max_hart_per_socket));
> > -qemu_fdt_setprop_cell(ms->fdt, imsic_name,
> > "riscv,group-index-bits",
> > -imsic_num_bits(socket_count));
> > -qemu_fdt_setprop_cell(ms->fdt, imsic_name,
> > "riscv,group-index-shift",
> > -IMSIC_MMIO_GROUP_MIN_SHIFT);
> > -}
> > -qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle);
> > +imsic_max_hart_per_socket = 0;
> > +for (socket = 0; socket < socket_count; socket++) {
> > +imsic_addr = memmap[VIRT_IMSIC_M].base +
> > + socket * VIRT_IMSIC_GROUP_MAX_SIZE;
> > +imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
> > +imsic_regs[socket * 4 + 0] = 0;
> > +imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
> > +imsic_regs[socket * 4 + 2] = 0;
> > +imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
> > +if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
> > +imsic_max_hart_per_socket = s->soc[socket].num_harts;
> > +}
> > +}
> > +
Re: [PATCH v3 4/6] target/riscv: Create an KVM AIA irqchip
Hi Daniel,
Thanks for your suggestions! I'll fix it in patch v4.
Regards,
Yong-Xuan
On Tue, Jun 6, 2023 at 9:45 PM Daniel Henrique Barboza
wrote:
>
>
>
> On 5/26/23 03:25, Yong-Xuan Wang wrote:
> > implement a function to create an KVM AIA chip
> >
> > Signed-off-by: Yong-Xuan Wang
> > Reviewed-by: Jim Shu
> > ---
> > target/riscv/kvm.c | 83
> > target/riscv/kvm_riscv.h | 3 ++
> > 2 files changed, 86 insertions(+)
> >
> > diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
> > index eb469e8ca5..ead121154f 100644
> > --- a/target/riscv/kvm.c
> > +++ b/target/riscv/kvm.c
> > @@ -34,6 +34,7 @@
> > #include "exec/address-spaces.h"
> > #include "hw/boards.h"
> > #include "hw/irq.h"
> > +#include "hw/intc/riscv_imsic.h"
> > #include "qemu/log.h"
> > #include "hw/loader.h"
> > #include "kvm_riscv.h"
> > @@ -548,3 +549,85 @@ bool kvm_arch_cpu_check_are_resettable(void)
> > void kvm_arch_accel_class_init(ObjectClass *oc)
> > {
> > }
> > +
> > +void kvm_riscv_aia_create(DeviceState *aplic_s, bool msimode, int socket,
> > + uint64_t aia_irq_num, uint64_t hart_count,
> > + uint64_t aplic_base, uint64_t imsic_base)
> > +{
> > +int ret;
> > +int aia_fd = -1;
> > +uint64_t aia_mode;
> > +uint64_t aia_nr_ids;
> > +uint64_t aia_hart_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
> > +
> > +if (!msimode) {
> > +error_report("Currently KVM AIA only supports aplic_imsic mode");
> > +exit(1);
> > +}
> > +
> > +aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
> > +
> > +if (aia_fd < 0) {
> > +error_report("Unable to create in-kernel irqchip");
> > +exit(1);
> > +}
> > +
> > +ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
> > +KVM_DEV_RISCV_AIA_CONFIG_MODE,
> > +_mode, false, NULL);
> > +
> > +ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
> > +KVM_DEV_RISCV_AIA_CONFIG_IDS,
> > +_nr_ids, false, NULL);
> > +
> > +ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
> > +KVM_DEV_RISCV_AIA_CONFIG_SRCS,
> > +_irq_num, true, NULL);
> > +if (ret < 0) {
> > +error_report("KVM AIA: fail to set number input irq lines");
> > +exit(1);
> > +}
>
> I see that you didn't check 'ret' for the first 2 calls of
> kvm_device_access().
> Is it intentional?
>
> Since you're setting customized error messages for every step I think it's
> worth
> also handling the case where we fail to set aia_mode and aia_nr_ids.
>
>
> Thanks,
>
>
> Daniel
>
>
> > +
> > +ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
> > +KVM_DEV_RISCV_AIA_CONFIG_HART_BITS,
> > +_hart_bits, true, NULL);
> > +if (ret < 0) {
> > +error_report("KVM AIA: fail to set number of harts");
> > +exit(1);
> > +}
> > +
> > +ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
> > +KVM_DEV_RISCV_AIA_ADDR_APLIC,
> > +_base, true, NULL);
> > +if (ret < 0) {
> > +error_report("KVM AIA: fail to set the base address of APLIC");
> > +exit(1);
> > +}
> > +
> > +for (int i = 0; i < hart_count; i++) {
> > +uint64_t imsic_addr = imsic_base + i * IMSIC_HART_SIZE(0);
> > +ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
> > +KVM_DEV_RISCV_AIA_ADDR_IMSIC(i),
> > +_addr, true, NULL);
> > +if (ret < 0) {
> > +error_report("KVM AIA: fail to set the base address of
> > IMSICs");
> > +exit(1);
> > +}
> > +}
> > +
> > +if (kvm_has_gsi_routing()) {
> > +for (uint64_t idx = 0; idx < aia_irq_num + 1; ++idx) {
> > +kvm_irqchip_add_irq_route(kvm_state, idx, socket, idx);
> > +}
> > +kvm_g
[PATCH v3 4/6] target/riscv: Create an KVM AIA irqchip
implement a function to create an KVM AIA chip
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm.c | 83
target/riscv/kvm_riscv.h | 3 ++
2 files changed, 86 insertions(+)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index eb469e8ca5..ead121154f 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -34,6 +34,7 @@
#include "exec/address-spaces.h"
#include "hw/boards.h"
#include "hw/irq.h"
+#include "hw/intc/riscv_imsic.h"
#include "qemu/log.h"
#include "hw/loader.h"
#include "kvm_riscv.h"
@@ -548,3 +549,85 @@ bool kvm_arch_cpu_check_are_resettable(void)
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
+
+void kvm_riscv_aia_create(DeviceState *aplic_s, bool msimode, int socket,
+ uint64_t aia_irq_num, uint64_t hart_count,
+ uint64_t aplic_base, uint64_t imsic_base)
+{
+int ret;
+int aia_fd = -1;
+uint64_t aia_mode;
+uint64_t aia_nr_ids;
+uint64_t aia_hart_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
+
+if (!msimode) {
+error_report("Currently KVM AIA only supports aplic_imsic mode");
+exit(1);
+}
+
+aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
+
+if (aia_fd < 0) {
+error_report("Unable to create in-kernel irqchip");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_mode, false, NULL);
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_IDS,
+_nr_ids, false, NULL);
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_SRCS,
+_irq_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number input irq lines");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_HART_BITS,
+_hart_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number of harts");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+KVM_DEV_RISCV_AIA_ADDR_APLIC,
+_base, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set the base address of APLIC");
+exit(1);
+}
+
+for (int i = 0; i < hart_count; i++) {
+uint64_t imsic_addr = imsic_base + i * IMSIC_HART_SIZE(0);
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+KVM_DEV_RISCV_AIA_ADDR_IMSIC(i),
+_addr, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set the base address of IMSICs");
+exit(1);
+}
+}
+
+if (kvm_has_gsi_routing()) {
+for (uint64_t idx = 0; idx < aia_irq_num + 1; ++idx) {
+kvm_irqchip_add_irq_route(kvm_state, idx, socket, idx);
+}
+kvm_gsi_routing_allowed = true;
+kvm_irqchip_commit_routes(kvm_state);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CTRL,
+KVM_DEV_RISCV_AIA_CTRL_INIT,
+NULL, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: initialized fail");
+exit(1);
+}
+}
diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h
index 606968a4b7..6067adff51 100644
--- a/target/riscv/kvm_riscv.h
+++ b/target/riscv/kvm_riscv.h
@@ -21,6 +21,9 @@
void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
+void kvm_riscv_aia_create(DeviceState *aplic_s, bool msimode, int socket,
+ uint64_t aia_irq_num, uint64_t hart_count,
+ uint64_t aplic_base, uint64_t imsic_base);
#define KVM_DEV_RISCV_AIA_GRP_CONFIG0
#define KVM_DEV_RISCV_AIA_CONFIG_MODE 0
--
2.17.1
[PATCH v3 2/6] target/riscv: support the AIA device emulation with KVM enabled
Remove M mode AIA devices when using KVM acceleration
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 199 +---
1 file changed, 105 insertions(+), 94 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 4e3efbee16..18b94888ab 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -531,52 +531,54 @@ static void create_fdt_imsic(RISCVVirtState *s, const
MemMapEntry *memmap,
imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
imsic_regs = g_new0(uint32_t, socket_count * 4);
-/* M-level IMSIC node */
-for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
-imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
-}
-imsic_max_hart_per_socket = 0;
-for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_M].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
-imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
-imsic_regs[socket * 4 + 0] = 0;
-imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
-imsic_regs[socket * 4 + 2] = 0;
-imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
-if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
-imsic_max_hart_per_socket = s->soc[socket].num_harts;
+if (!kvm_enabled()) {
+/* M-level IMSIC node */
+for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
}
-}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_M].base);
-qemu_fdt_add_subnode(ms->fdt, imsic_name);
-qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
-"riscv,imsics");
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
-FDT_IMSIC_INT_CELLS);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
-qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
-socket_count * sizeof(uint32_t) * 4);
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
-VIRT_IRQCHIP_NUM_MSIS);
-if (socket_count > 1) {
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits",
-imsic_num_bits(imsic_max_hart_per_socket));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits",
-imsic_num_bits(socket_count));
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift",
-IMSIC_MMIO_GROUP_MIN_SHIFT);
-}
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle);
+imsic_max_hart_per_socket = 0;
+for (socket = 0; socket < socket_count; socket++) {
+imsic_addr = memmap[VIRT_IMSIC_M].base +
+ socket * VIRT_IMSIC_GROUP_MAX_SIZE;
+imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
+imsic_regs[socket * 4 + 0] = 0;
+imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
+imsic_regs[socket * 4 + 2] = 0;
+imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
+if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
+imsic_max_hart_per_socket = s->soc[socket].num_harts;
+}
+}
+imsic_name = g_strdup_printf("/soc/imsics@%lx",
+(unsigned long)memmap[VIRT_IMSIC_M].base);
+qemu_fdt_add_subnode(ms->fdt, imsic_name);
+qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
+"riscv,imsics");
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
+FDT_IMSIC_INT_CELLS);
+qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
+NULL, 0);
+qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
+NULL, 0);
+qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
+imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
+qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
+socket_count * sizeof(uint32_t) * 4);
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
+VIRT_IRQCHIP_NUM_MSIS);
+if (socket_count > 1) {
+qemu_fdt_setprop_cel
[PATCH v3 3/6] target/riscv: check the in-kernel irqchip support
We check the in-kernel irqchip support when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 0f932a5b96..eb469e8ca5 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -433,7 +433,18 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
int kvm_arch_irqchip_create(KVMState *s)
{
-return 0;
+if (kvm_kernel_irqchip_split()) {
+error_report("-machine kernel_irqchip=split is not supported "
+ "on RISC-V.");
+exit(1);
+}
+
+/*
+ * If we can create the VAIA using the newer device control API, we
+ * let the device do this when it initializes itself, otherwise we
+ * fall back to the old API
+ */
+return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
}
int kvm_arch_process_async_events(CPUState *cs)
--
2.17.1
[PATCH v3 5/6] target/riscv: update APLIC and IMSIC to support KVM AIA
- Do not set the mmio operations of APLIC and IMSIC when using KVM AIA
- Send interrupt signal to KVM AIA via KVM_IRQ_LINE API
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/intc/riscv_aplic.c | 19 +++
hw/intc/riscv_imsic.c | 16 +++-
2 files changed, 26 insertions(+), 9 deletions(-)
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index afc5b54dbb..adf5427f22 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -31,6 +31,7 @@
#include "hw/irq.h"
#include "target/riscv/cpu.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define APLIC_MAX_IDC (1UL << 14)
@@ -479,6 +480,11 @@ static void riscv_aplic_request(void *opaque, int irq, int
level)
assert((0 < irq) && (irq < aplic->num_irqs));
+if (kvm_irqchip_in_kernel()) {
+kvm_set_irq(kvm_state, irq, !!level);
+return;
+}
+
sourcecfg = aplic->sourcecfg[irq];
if (sourcecfg & APLIC_SOURCECFG_D) {
childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
@@ -814,9 +820,11 @@ static void riscv_aplic_realize(DeviceState *dev, Error
**errp)
aplic->iforce = g_new0(uint32_t, aplic->num_harts);
aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
-memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops, aplic,
- TYPE_RISCV_APLIC, aplic->aperture_size);
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+if (!kvm_irqchip_in_kernel()) {
+memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops,
+ aplic, TYPE_RISCV_APLIC, aplic->aperture_size);
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/*
* Only root APLICs have hardware IRQ lines. All non-root APLICs
@@ -958,7 +966,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
qdev_prop_set_bit(dev, "mmode", mmode);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!kvm_irqchip_in_kernel()) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
if (parent) {
riscv_aplic_add_child(parent, dev);
diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
index fea3385b51..8bfa480f7c 100644
--- a/hw/intc/riscv_imsic.c
+++ b/hw/intc/riscv_imsic.c
@@ -32,6 +32,7 @@
#include "target/riscv/cpu.h"
#include "target/riscv/cpu_bits.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define IMSIC_MMIO_PAGE_LE 0x00
@@ -325,10 +326,12 @@ static void riscv_imsic_realize(DeviceState *dev, Error
**errp)
imsic->eithreshold = g_new0(uint32_t, imsic->num_pages);
imsic->eistate = g_new0(uint32_t, imsic->num_eistate);
-memory_region_init_io(>mmio, OBJECT(dev), _imsic_ops,
- imsic, TYPE_RISCV_IMSIC,
- IMSIC_MMIO_SIZE(imsic->num_pages));
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+if (!kvm_irqchip_in_kernel()) {
+memory_region_init_io(>mmio, OBJECT(dev), _imsic_ops,
+ imsic, TYPE_RISCV_IMSIC,
+ IMSIC_MMIO_SIZE(imsic->num_pages));
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/* Claim the CPU interrupt to be triggered by this IMSIC */
if (riscv_cpu_claim_interrupts(rcpu,
@@ -432,7 +435,10 @@ DeviceState *riscv_imsic_create(hwaddr addr, uint32_t
hartid, bool mmode,
qdev_prop_set_uint32(dev, "num-irqs", num_ids + 1);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!kvm_irqchip_in_kernel()) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
for (i = 0; i < num_pages; i++) {
if (!i) {
--
2.17.1
[PATCH v3 0/6] Add RISC-V KVM AIA Support
This series adds support for KVM AIA in RISC-V architecture. In order to test these patches, we require Linux with KVM AIA support which can be found in the qemu_kvm_aia branch at https://github.com/yong-xuan/linux.git This kernel branch is based on the riscv_aia_v1 branch available at https://github.com/avpatel/linux.git, and it also includes two additional patches that fix a KVM AIA bug and reply to the query of KVM_CAP_IRQCHIP. --- v3: - fix typo - tag the linux-header patch as placeholder v2: - rebase to riscv-to-apply.next - update the linux header by the scripts/update-linux-headers.sh Yong-Xuan Wang (6): update-linux-headers: sync-up header with Linux for KVM AIA support placeholder target/riscv: support the AIA device emulation with KVM enabled target/riscv: check the in-kernel irqchip support target/riscv: Create an KVM AIA irqchip target/riscv: update APLIC and IMSIC to support KVM AIA target/riscv: select KVM AIA in riscv virt machine hw/intc/riscv_aplic.c | 19 +++- hw/intc/riscv_imsic.c | 16 ++- hw/riscv/virt.c | 209 +- linux-headers/linux/kvm.h | 2 + target/riscv/kvm.c| 96 - target/riscv/kvm_riscv.h | 36 +++ 6 files changed, 274 insertions(+), 104 deletions(-) -- 2.17.1
[PATCH v3 6/6] target/riscv: select KVM AIA in riscv virt machine
Select KVM AIA when the host kernel has in-kernel AIA chip support.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 18b94888ab..57a07fa6c5 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
#include "hw/riscv/virt.h"
#include "hw/riscv/boot.h"
#include "hw/riscv/numa.h"
+#include "kvm_riscv.h"
#include "hw/intc/riscv_aclint.h"
#include "hw/intc/riscv_aplic.h"
#include "hw/intc/riscv_imsic.h"
@@ -1217,6 +1218,15 @@ static DeviceState *virt_create_aia(RISCVVirtAIAType
aia_type, int aia_guests,
msimode, false, aplic_m);
}
+if (kvm_irqchip_in_kernel()) {
+kvm_riscv_aia_create(
+aplic_s, msimode, socket,
+VIRT_IRQCHIP_NUM_SOURCES,
+hart_count,
+memmap[VIRT_APLIC_S].base + socket * memmap[VIRT_APLIC_S].size,
+memmap[VIRT_IMSIC_S].base + socket * VIRT_IMSIC_GROUP_MAX_SIZE);
+}
+
return kvm_enabled() ? aplic_s : aplic_m;
}
--
2.17.1
[PATCH v3 1/6] update-linux-headers: sync-up header with Linux for KVM AIA support placeholder
Sync-up Linux header to get latest KVM RISC-V headers having AIA support.
Note: This is a placeholder commit and could be replaced when all referenced
Linux patchsets are mainlined.
The linux-headers changes are from 2 different patchsets.
[1]
https://lore.kernel.org/lkml/20230404153452.2405681-1-apa...@ventanamicro.com/
[2] https://www.spinics.net/lists/kernel/msg4791872.html
Currently, patchset 1 is already merged into mainline kernel in v6.4-rc1 and
patchset 2 is not.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
linux-headers/linux/kvm.h | 2 ++
target/riscv/kvm_riscv.h | 33 +
2 files changed, 35 insertions(+)
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 599de3c6e3..a9a4f5791d 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -1434,6 +1434,8 @@ enum kvm_device_type {
#define KVM_DEV_TYPE_XIVE KVM_DEV_TYPE_XIVE
KVM_DEV_TYPE_ARM_PV_TIME,
#define KVM_DEV_TYPE_ARM_PV_TIME KVM_DEV_TYPE_ARM_PV_TIME
+ KVM_DEV_TYPE_RISCV_AIA,
+#define KVM_DEV_TYPE_RISCV_AIA KVM_DEV_TYPE_RISCV_AIA
KVM_DEV_TYPE_MAX,
};
diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h
index ed281bdce0..606968a4b7 100644
--- a/target/riscv/kvm_riscv.h
+++ b/target/riscv/kvm_riscv.h
@@ -22,4 +22,37 @@
void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
+#define KVM_DEV_RISCV_AIA_GRP_CONFIG0
+#define KVM_DEV_RISCV_AIA_CONFIG_MODE 0
+#define KVM_DEV_RISCV_AIA_CONFIG_IDS1
+#define KVM_DEV_RISCV_AIA_CONFIG_SRCS 2
+#define KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS 3
+#define KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT4
+#define KVM_DEV_RISCV_AIA_CONFIG_HART_BITS 5
+#define KVM_DEV_RISCV_AIA_CONFIG_GUEST_BITS 6
+#define KVM_DEV_RISCV_AIA_MODE_EMUL 0
+#define KVM_DEV_RISCV_AIA_MODE_HWACCEL 1
+#define KVM_DEV_RISCV_AIA_MODE_AUTO 2
+#define KVM_DEV_RISCV_AIA_IDS_MIN 63
+#define KVM_DEV_RISCV_AIA_IDS_MAX 2048
+#define KVM_DEV_RISCV_AIA_SRCS_MAX 1024
+#define KVM_DEV_RISCV_AIA_GROUP_BITS_MAX8
+#define KVM_DEV_RISCV_AIA_GROUP_SHIFT_MIN 24
+#define KVM_DEV_RISCV_AIA_GROUP_SHIFT_MAX 56
+#define KVM_DEV_RISCV_AIA_HART_BITS_MAX 16
+#define KVM_DEV_RISCV_AIA_GUEST_BITS_MAX8
+
+#define KVM_DEV_RISCV_AIA_GRP_ADDR 1
+#define KVM_DEV_RISCV_AIA_ADDR_APLIC0
+#define KVM_DEV_RISCV_AIA_ADDR_IMSIC(__vcpu)(1 + (__vcpu))
+#define KVM_DEV_RISCV_AIA_ADDR_MAX \
+(1 + KVM_DEV_RISCV_APLIC_MAX_HARTS)
+
+#define KVM_DEV_RISCV_AIA_GRP_CTRL 2
+#define KVM_DEV_RISCV_AIA_CTRL_INIT 0
+
+#define KVM_DEV_RISCV_AIA_GRP_APLIC 3
+
+#define KVM_DEV_RISCV_AIA_GRP_IMSIC 4
+
#endif
--
2.17.1
Re: [PTACH v2 1/6] update-linux-headers: sync-up header with Linux for KVM AIA support
On Mon, May 8, 2023 at 3:39 PM Cornelia Huck wrote: > > On Fri, May 05 2023, Alex Williamson wrote: > > > On Fri, 5 May 2023 11:39:36 +0000 > > Yong-Xuan Wang wrote: > > > >> Update the linux headers to get the latest KVM RISC-V headers with AIA > >> support > >> by the scripts/update-linux-headers.sh. > >> The linux headers is comes from the riscv_aia_v1 branch available at > >> https://github.com/avpatel/linux.git. It hasn't merged into the mainline > >> kernel. > > > > Updating linux-headers outside of code accepted to mainline gets a down > > vote from me. This sets a poor precedent and can potentially lead to > > complicated compatibility issues. Thanks, > > > > Alex > > Indeed, this needs to be clearly marked as a placeholder patch, and > replaced with a proper header sync after the changes hit the mainline > kernel. > Hi Alex and Cornelia, We found that the changes are from 2 different patchsets. [1] https://lore.kernel.org/lkml/20230404153452.2405681-1-apa...@ventanamicro.com/ [2] https://www.spinics.net/lists/kernel/msg4791872.html Patchset 1 is already merged into mainline kernel in v6.4-rc1 and patchset 2 is not. Maybe we can split them into two placeholder patches? Regards, Yong-Xuan
[PTACH v2 6/6] target/riscv: select KVM AIA in riscv virt machine
Select KVM AIA when the host kernel has in-kernel AIA chip support.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 396025b5a5..9fad01a5ab 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
#include "hw/riscv/virt.h"
#include "hw/riscv/boot.h"
#include "hw/riscv/numa.h"
+#include "kvm_riscv.h"
#include "hw/intc/riscv_aclint.h"
#include "hw/intc/riscv_aplic.h"
#include "hw/intc/riscv_imsic.h"
@@ -1216,6 +1217,15 @@ static DeviceState *virt_create_aia(RISCVVirtAIAType
aia_type, int aia_guests,
msimode, false, aplic_m);
}
+if (kvm_irqchip_in_kernel()) {
+kvm_riscv_aia_create(
+aplic_s, msimode, socket,
+VIRT_IRQCHIP_NUM_SOURCES,
+hart_count,
+memmap[VIRT_APLIC_S].base + socket * memmap[VIRT_APLIC_S].size,
+memmap[VIRT_IMSIC_S].base + socket * VIRT_IMSIC_GROUP_MAX_SIZE);
+}
+
return kvm_enabled() ? aplic_s : aplic_m;
}
--
2.17.1
[PTACH v2 5/6] target/riscv: update APLIC and IMSIC to support KVM AIA
- Do not set the mmio operations of APLIC and IMSIC when using KVM AIA
- Send interrupt signal to KVM AIA via KVM_IRQ_LINE API
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/intc/riscv_aplic.c | 19 +++
hw/intc/riscv_imsic.c | 16 +++-
2 files changed, 26 insertions(+), 9 deletions(-)
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index afc5b54dbb..adf5427f22 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -31,6 +31,7 @@
#include "hw/irq.h"
#include "target/riscv/cpu.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define APLIC_MAX_IDC (1UL << 14)
@@ -479,6 +480,11 @@ static void riscv_aplic_request(void *opaque, int irq, int
level)
assert((0 < irq) && (irq < aplic->num_irqs));
+if (kvm_irqchip_in_kernel()) {
+kvm_set_irq(kvm_state, irq, !!level);
+return;
+}
+
sourcecfg = aplic->sourcecfg[irq];
if (sourcecfg & APLIC_SOURCECFG_D) {
childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
@@ -814,9 +820,11 @@ static void riscv_aplic_realize(DeviceState *dev, Error
**errp)
aplic->iforce = g_new0(uint32_t, aplic->num_harts);
aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
-memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops, aplic,
- TYPE_RISCV_APLIC, aplic->aperture_size);
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+if (!kvm_irqchip_in_kernel()) {
+memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops,
+ aplic, TYPE_RISCV_APLIC, aplic->aperture_size);
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/*
* Only root APLICs have hardware IRQ lines. All non-root APLICs
@@ -958,7 +966,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
qdev_prop_set_bit(dev, "mmode", mmode);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!kvm_irqchip_in_kernel()) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
if (parent) {
riscv_aplic_add_child(parent, dev);
diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
index fea3385b51..8bfa480f7c 100644
--- a/hw/intc/riscv_imsic.c
+++ b/hw/intc/riscv_imsic.c
@@ -32,6 +32,7 @@
#include "target/riscv/cpu.h"
#include "target/riscv/cpu_bits.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define IMSIC_MMIO_PAGE_LE 0x00
@@ -325,10 +326,12 @@ static void riscv_imsic_realize(DeviceState *dev, Error
**errp)
imsic->eithreshold = g_new0(uint32_t, imsic->num_pages);
imsic->eistate = g_new0(uint32_t, imsic->num_eistate);
-memory_region_init_io(>mmio, OBJECT(dev), _imsic_ops,
- imsic, TYPE_RISCV_IMSIC,
- IMSIC_MMIO_SIZE(imsic->num_pages));
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+if (!kvm_irqchip_in_kernel()) {
+memory_region_init_io(>mmio, OBJECT(dev), _imsic_ops,
+ imsic, TYPE_RISCV_IMSIC,
+ IMSIC_MMIO_SIZE(imsic->num_pages));
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/* Claim the CPU interrupt to be triggered by this IMSIC */
if (riscv_cpu_claim_interrupts(rcpu,
@@ -432,7 +435,10 @@ DeviceState *riscv_imsic_create(hwaddr addr, uint32_t
hartid, bool mmode,
qdev_prop_set_uint32(dev, "num-irqs", num_ids + 1);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!kvm_irqchip_in_kernel()) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
for (i = 0; i < num_pages; i++) {
if (!i) {
--
2.17.1
[PTACH v2 4/6] target/riscv: Create an KVM AIA irqchip
implement a function to create an KVM AIA chip
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm.c | 83
target/riscv/kvm_riscv.h | 3 ++
2 files changed, 86 insertions(+)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index eb469e8ca5..ead121154f 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -34,6 +34,7 @@
#include "exec/address-spaces.h"
#include "hw/boards.h"
#include "hw/irq.h"
+#include "hw/intc/riscv_imsic.h"
#include "qemu/log.h"
#include "hw/loader.h"
#include "kvm_riscv.h"
@@ -548,3 +549,85 @@ bool kvm_arch_cpu_check_are_resettable(void)
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
+
+void kvm_riscv_aia_create(DeviceState *aplic_s, bool msimode, int socket,
+ uint64_t aia_irq_num, uint64_t hart_count,
+ uint64_t aplic_base, uint64_t imsic_base)
+{
+int ret;
+int aia_fd = -1;
+uint64_t aia_mode;
+uint64_t aia_nr_ids;
+uint64_t aia_hart_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
+
+if (!msimode) {
+error_report("Currently KVM AIA only supports aplic_imsic mode");
+exit(1);
+}
+
+aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
+
+if (aia_fd < 0) {
+error_report("Unable to create in-kernel irqchip");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_mode, false, NULL);
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_IDS,
+_nr_ids, false, NULL);
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_SRCS,
+_irq_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number input irq lines");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_HART_BITS,
+_hart_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number of harts");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+KVM_DEV_RISCV_AIA_ADDR_APLIC,
+_base, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set the base address of APLIC");
+exit(1);
+}
+
+for (int i = 0; i < hart_count; i++) {
+uint64_t imsic_addr = imsic_base + i * IMSIC_HART_SIZE(0);
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+KVM_DEV_RISCV_AIA_ADDR_IMSIC(i),
+_addr, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set the base address of IMSICs");
+exit(1);
+}
+}
+
+if (kvm_has_gsi_routing()) {
+for (uint64_t idx = 0; idx < aia_irq_num + 1; ++idx) {
+kvm_irqchip_add_irq_route(kvm_state, idx, socket, idx);
+}
+kvm_gsi_routing_allowed = true;
+kvm_irqchip_commit_routes(kvm_state);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CTRL,
+KVM_DEV_RISCV_AIA_CTRL_INIT,
+NULL, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: initialized fail");
+exit(1);
+}
+}
diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h
index ed281bdce0..d8d7256852 100644
--- a/target/riscv/kvm_riscv.h
+++ b/target/riscv/kvm_riscv.h
@@ -21,5 +21,8 @@
void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
+void kvm_riscv_aia_create(DeviceState *aplic_s, bool msimode, int socket,
+ uint64_t aia_irq_num, uint64_t hart_count,
+ uint64_t aplic_base, uint64_t imsic_base);
#endif
--
2.17.1
[PTACH v2 2/6] target/riscv: support the AIA device emulateion with KVM enabled
Remove M mode AIA devices when using KVM acceleration
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 198 +---
1 file changed, 104 insertions(+), 94 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 4e3efbee16..396025b5a5 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -531,52 +531,53 @@ static void create_fdt_imsic(RISCVVirtState *s, const
MemMapEntry *memmap,
imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
imsic_regs = g_new0(uint32_t, socket_count * 4);
-/* M-level IMSIC node */
-for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
-imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
-}
-imsic_max_hart_per_socket = 0;
-for (socket = 0; socket < socket_count; socket++) {
-imsic_addr = memmap[VIRT_IMSIC_M].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
-imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
-imsic_regs[socket * 4 + 0] = 0;
-imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
-imsic_regs[socket * 4 + 2] = 0;
-imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
-if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
-imsic_max_hart_per_socket = s->soc[socket].num_harts;
+if (!kvm_enabled()) {
+/* M-level IMSIC node */
+for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
}
+imsic_max_hart_per_socket = 0;
+for (socket = 0; socket < socket_count; socket++) {
+imsic_addr = memmap[VIRT_IMSIC_M].base +
+ socket * VIRT_IMSIC_GROUP_MAX_SIZE;
+imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
+imsic_regs[socket * 4 + 0] = 0;
+imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
+imsic_regs[socket * 4 + 2] = 0;
+imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
+if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
+imsic_max_hart_per_socket = s->soc[socket].num_harts;
+}
+}
+imsic_name = g_strdup_printf("/soc/imsics@%lx",
+(unsigned long)memmap[VIRT_IMSIC_M].base);
+qemu_fdt_add_subnode(ms->fdt, imsic_name);
+qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
+"riscv,imsics");
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
+FDT_IMSIC_INT_CELLS);
+qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
+NULL, 0);
+qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
+NULL, 0);
+qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
+imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
+qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
+socket_count * sizeof(uint32_t) * 4);
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
+VIRT_IRQCHIP_NUM_MSIS);
+if (socket_count > 1) {
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits",
+imsic_num_bits(imsic_max_hart_per_socket));
+qemu_fdt_setprop_cell(ms->fdt, imsic_name,
"riscv,group-index-bits",
+imsic_num_bits(socket_count));
+qemu_fdt_setprop_cell(ms->fdt, imsic_name,
+"riscv,group-index-shift", IMSIC_MMIO_GROUP_MIN_SHIFT);
+}
+qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle);
+g_free(imsic_name);
}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_M].base);
-qemu_fdt_add_subnode(ms->fdt, imsic_name);
-qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible",
-"riscv,imsics");
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells",
-FDT_IMSIC_INT_CELLS);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
-NULL, 0);
-qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
-qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
-socket_count * sizeof(uint32_t) * 4);
-qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
-VIRT_IRQCHIP_NUM_MSIS);
-
[PTACH v2 3/6] target/riscv: check the in-kernel irqchip support
We check the in-kernel irqchip support when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 0f932a5b96..eb469e8ca5 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -433,7 +433,18 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
int kvm_arch_irqchip_create(KVMState *s)
{
-return 0;
+if (kvm_kernel_irqchip_split()) {
+error_report("-machine kernel_irqchip=split is not supported "
+ "on RISC-V.");
+exit(1);
+}
+
+/*
+ * If we can create the VAIA using the newer device control API, we
+ * let the device do this when it initializes itself, otherwise we
+ * fall back to the old API
+ */
+return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
}
int kvm_arch_process_async_events(CPUState *cs)
--
2.17.1
[PTACH v2 1/6] update-linux-headers: sync-up header with Linux for KVM AIA support
Update the linux headers to get the latest KVM RISC-V headers with AIA support
by the scripts/update-linux-headers.sh.
The linux headers is comes from the riscv_aia_v1 branch available at
https://github.com/avpatel/linux.git. It hasn't merged into the mainline kernel.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
linux-headers/asm-riscv/kvm.h | 105 +-
linux-headers/linux/kvm.h | 2 +
2 files changed, 106 insertions(+), 1 deletion(-)
diff --git a/linux-headers/asm-riscv/kvm.h b/linux-headers/asm-riscv/kvm.h
index 92af6f3f05..deba7ff304 100644
--- a/linux-headers/asm-riscv/kvm.h
+++ b/linux-headers/asm-riscv/kvm.h
@@ -12,8 +12,10 @@
#ifndef __ASSEMBLY__
#include
+#include
#include
+#define __KVM_HAVE_IRQ_LINE
#define __KVM_HAVE_READONLY_MEM
#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
@@ -64,7 +66,7 @@ struct kvm_riscv_core {
#define KVM_RISCV_MODE_S 1
#define KVM_RISCV_MODE_U 0
-/* CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+/* General CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
struct kvm_riscv_csr {
unsigned long sstatus;
unsigned long sie;
@@ -78,6 +80,17 @@ struct kvm_riscv_csr {
unsigned long scounteren;
};
+/* AIA CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_aia_csr {
+ unsigned long siselect;
+ unsigned long siprio1;
+ unsigned long siprio2;
+ unsigned long sieh;
+ unsigned long siph;
+ unsigned long siprio1h;
+ unsigned long siprio2h;
+};
+
/* TIMER registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
struct kvm_riscv_timer {
__u64 frequency;
@@ -105,9 +118,28 @@ enum KVM_RISCV_ISA_EXT_ID {
KVM_RISCV_ISA_EXT_SVINVAL,
KVM_RISCV_ISA_EXT_ZIHINTPAUSE,
KVM_RISCV_ISA_EXT_ZICBOM,
+ KVM_RISCV_ISA_EXT_ZBB,
+ KVM_RISCV_ISA_EXT_SSAIA,
KVM_RISCV_ISA_EXT_MAX,
};
+/*
+ * SBI extension IDs specific to KVM. This is not the same as the SBI
+ * extension IDs defined by the RISC-V SBI specification.
+ */
+enum KVM_RISCV_SBI_EXT_ID {
+ KVM_RISCV_SBI_EXT_V01 = 0,
+ KVM_RISCV_SBI_EXT_TIME,
+ KVM_RISCV_SBI_EXT_IPI,
+ KVM_RISCV_SBI_EXT_RFENCE,
+ KVM_RISCV_SBI_EXT_SRST,
+ KVM_RISCV_SBI_EXT_HSM,
+ KVM_RISCV_SBI_EXT_PMU,
+ KVM_RISCV_SBI_EXT_EXPERIMENTAL,
+ KVM_RISCV_SBI_EXT_VENDOR,
+ KVM_RISCV_SBI_EXT_MAX,
+};
+
/* Possible states for kvm_riscv_timer */
#define KVM_RISCV_TIMER_STATE_OFF 0
#define KVM_RISCV_TIMER_STATE_ON 1
@@ -118,6 +150,8 @@ enum KVM_RISCV_ISA_EXT_ID {
/* If you need to interpret the index values, here is the key: */
#define KVM_REG_RISCV_TYPE_MASK0xFF00
#define KVM_REG_RISCV_TYPE_SHIFT 24
+#define KVM_REG_RISCV_SUBTYPE_MASK 0x00FF
+#define KVM_REG_RISCV_SUBTYPE_SHIFT16
/* Config registers are mapped as type 1 */
#define KVM_REG_RISCV_CONFIG (0x01 << KVM_REG_RISCV_TYPE_SHIFT)
@@ -131,8 +165,12 @@ enum KVM_RISCV_ISA_EXT_ID {
/* Control and status registers are mapped as type 3 */
#define KVM_REG_RISCV_CSR (0x03 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_CSR_GENERAL (0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_CSR_AIA (0x1 << KVM_REG_RISCV_SUBTYPE_SHIFT)
#define KVM_REG_RISCV_CSR_REG(name)\
(offsetof(struct kvm_riscv_csr, name) / sizeof(unsigned long))
+#define KVM_REG_RISCV_CSR_AIA_REG(name)\
+ (offsetof(struct kvm_riscv_aia_csr, name) / sizeof(unsigned long))
/* Timer registers are mapped as type 4 */
#define KVM_REG_RISCV_TIMER(0x04 << KVM_REG_RISCV_TYPE_SHIFT)
@@ -152,6 +190,71 @@ enum KVM_RISCV_ISA_EXT_ID {
/* ISA Extension registers are mapped as type 7 */
#define KVM_REG_RISCV_ISA_EXT (0x07 << KVM_REG_RISCV_TYPE_SHIFT)
+/* SBI extension registers are mapped as type 8 */
+#define KVM_REG_RISCV_SBI_EXT (0x08 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_SINGLE (0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_MULTI_EN (0x1 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_MULTI_DIS(0x2 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_MULTI_REG(__ext_id) \
+ ((__ext_id) / __BITS_PER_LONG)
+#define KVM_REG_RISCV_SBI_MULTI_MASK(__ext_id) \
+ (1UL << ((__ext_id) % __BITS_PER_LONG))
+#define KVM_REG_RISCV_SBI_MULTI_REG_LAST \
+ KVM_REG_RISCV_SBI_MULTI_REG(KVM_RISCV_SBI_EXT_MAX - 1)
+
+/* Device Control API: RISC-V AIA */
+#define KVM_DEV_RISCV_APLIC_ALIGN 0x1000
+#define KVM_DEV_RISCV_APLIC_SIZE 0x4000
+#define KVM_DEV_RISCV_APLIC_MAX_HARTS 0x4000
+#define KVM_DEV_RISCV_IMSIC_ALIGN 0x1000
+#define KVM_DEV_RISCV_IMSIC_SIZE 0x1000
+
+#define KVM_DEV_RISCV
[PTACH v2 0/6] Add RISC-V KVM AIA Support
This series adds support for KVM AIA in RISC-V architecture. In order to test these patches, we require Linux with KVM AIA support which can be found in the qemu_kvm_aia branch at https://github.com/yong-xuan/linux.git This kernel branch is based on the riscv_aia_v1 branch available at https://github.com/avpatel/linux.git, and it also includes two additional patches that fix a KVM AIA bug and reply to the query of KVM_CAP_IRQCHIP. --- v2: - rebase to riscv-to-apply.next - update the linux header by the scripts/update-linux-headers.sh Yong-Xuan Wang (6): update-linux-headers: sync-up header with Linux for KVM AIA support target/riscv: support the AIA device emulateion with KVM enabled target/riscv: check the in-kernel irqchip support target/riscv: Create an KVM AIA irqchip target/riscv: update APLIC and IMSIC to support KVM AIA target/riscv: select KVM AIA in riscv virt machine hw/intc/riscv_aplic.c | 19 +++- hw/intc/riscv_imsic.c | 16 ++- hw/riscv/virt.c | 207 +++--- linux-headers/asm-riscv/kvm.h | 105 - linux-headers/linux/kvm.h | 2 + target/riscv/kvm.c| 96 +++- target/riscv/kvm_riscv.h | 3 + 7 files changed, 343 insertions(+), 105 deletions(-) -- 2.17.1
[PATCH 2/6] target/riscv: support the AIA device emulateion with KVM enabled
Remove M mode AIA devices when using KVM acceleration
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 204 +---
1 file changed, 108 insertions(+), 96 deletions(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index a5bc7353b4..18aede7b23 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -511,54 +511,56 @@ static void create_fdt_imsic(RISCVVirtState *s, const
MemMapEntry *memmap,
imsic_cells = g_new0(uint32_t, mc->smp.cpus * 2);
imsic_regs = g_new0(uint32_t, riscv_socket_count(mc) * 4);
-/* M-level IMSIC node */
-for (cpu = 0; cpu < mc->smp.cpus; cpu++) {
-imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
-imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
-}
-imsic_max_hart_per_socket = 0;
-for (socket = 0; socket < riscv_socket_count(mc); socket++) {
-imsic_addr = memmap[VIRT_IMSIC_M].base +
- socket * VIRT_IMSIC_GROUP_MAX_SIZE;
-imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
-imsic_regs[socket * 4 + 0] = 0;
-imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
-imsic_regs[socket * 4 + 2] = 0;
-imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
-if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
-imsic_max_hart_per_socket = s->soc[socket].num_harts;
+if (!kvm_enabled()) {
+/* M-level IMSIC node */
+for (cpu = 0; cpu < mc->smp.cpus; cpu++) {
+imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
}
-}
-imsic_name = g_strdup_printf("/soc/imsics@%lx",
-(unsigned long)memmap[VIRT_IMSIC_M].base);
-qemu_fdt_add_subnode(mc->fdt, imsic_name);
-qemu_fdt_setprop_string(mc->fdt, imsic_name, "compatible",
-"riscv,imsics");
-qemu_fdt_setprop_cell(mc->fdt, imsic_name, "#interrupt-cells",
-FDT_IMSIC_INT_CELLS);
-qemu_fdt_setprop(mc->fdt, imsic_name, "interrupt-controller",
-NULL, 0);
-qemu_fdt_setprop(mc->fdt, imsic_name, "msi-controller",
-NULL, 0);
-qemu_fdt_setprop(mc->fdt, imsic_name, "interrupts-extended",
-imsic_cells, mc->smp.cpus * sizeof(uint32_t) * 2);
-qemu_fdt_setprop(mc->fdt, imsic_name, "reg", imsic_regs,
-riscv_socket_count(mc) * sizeof(uint32_t) * 4);
-qemu_fdt_setprop_cell(mc->fdt, imsic_name, "riscv,num-ids",
-VIRT_IRQCHIP_NUM_MSIS);
-qemu_fdt_setprop_cells(mc->fdt, imsic_name, "riscv,ipi-id",
-VIRT_IRQCHIP_IPI_MSI);
-if (riscv_socket_count(mc) > 1) {
-qemu_fdt_setprop_cell(mc->fdt, imsic_name, "riscv,hart-index-bits",
-imsic_num_bits(imsic_max_hart_per_socket));
-qemu_fdt_setprop_cell(mc->fdt, imsic_name, "riscv,group-index-bits",
-imsic_num_bits(riscv_socket_count(mc)));
-qemu_fdt_setprop_cell(mc->fdt, imsic_name, "riscv,group-index-shift",
-IMSIC_MMIO_GROUP_MIN_SHIFT);
-}
-qemu_fdt_setprop_cell(mc->fdt, imsic_name, "phandle", *msi_m_phandle);
+imsic_max_hart_per_socket = 0;
+for (socket = 0; socket < riscv_socket_count(mc); socket++) {
+imsic_addr = memmap[VIRT_IMSIC_M].base +
+socket * VIRT_IMSIC_GROUP_MAX_SIZE;
+imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts;
+imsic_regs[socket * 4 + 0] = 0;
+imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr);
+imsic_regs[socket * 4 + 2] = 0;
+imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size);
+if (imsic_max_hart_per_socket < s->soc[socket].num_harts) {
+imsic_max_hart_per_socket = s->soc[socket].num_harts;
+}
+}
+imsic_name = g_strdup_printf("/soc/imsics@%lx",
+(unsigned long)memmap[VIRT_IMSIC_M].base);
+qemu_fdt_add_subnode(mc->fdt, imsic_name);
+qemu_fdt_setprop_string(mc->fdt, imsic_name, "compatible",
+"riscv,imsics");
+qemu_fdt_setprop_cell(mc->fdt, imsic_name, "#interrupt-cells",
+FDT_IMSIC_INT_CELLS);
+qemu_fdt_setprop(mc->fdt, imsic_name, "interrupt-controller",
+NULL, 0);
+qemu_fdt_setprop(mc->fdt, imsic_name, "msi-controller",
+NULL, 0);
+qemu_fdt_setprop(mc->fdt, imsic_name, "interrupts-extended",
+imsic_cells, mc->smp.cpus * sizeof(uint32_t) * 2);
+qemu_fdt_setprop(mc->fdt, imsic_name, "reg", imsic_regs,
+riscv_socket_count(mc) * sizeof(uint32
[PATCH 3/6] target/riscv: check the in-kernel irqchip support
We check the in-kernel irqchip support when using KVM acceleration.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 30f21453d6..6aad25bbc3 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -433,7 +433,18 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
int kvm_arch_irqchip_create(KVMState *s)
{
-return 0;
+if (kvm_kernel_irqchip_split()) {
+error_report("-machine kernel_irqchip=split is not supported "
+ "on RISC-V.");
+exit(1);
+}
+
+/*
+ * If we can create the VAIA using the newer device control API, we
+ * let the device do this when it initializes itself, otherwise we
+ * fall back to the old API
+ */
+return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
}
int kvm_arch_process_async_events(CPUState *cs)
--
2.17.1
[PATCH 1/6] update-linux-headers: sync-up header with Linux for KVM AIA support
Sync-up Linux header to get latest KVM RISC-V headers having AIA support.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
linux-headers/linux/kvm.h | 2 ++
target/riscv/kvm_riscv.h | 33 +
2 files changed, 35 insertions(+)
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index ebdafa576d..316732a617 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -1426,6 +1426,8 @@ enum kvm_device_type {
#define KVM_DEV_TYPE_XIVE KVM_DEV_TYPE_XIVE
KVM_DEV_TYPE_ARM_PV_TIME,
#define KVM_DEV_TYPE_ARM_PV_TIME KVM_DEV_TYPE_ARM_PV_TIME
+ KVM_DEV_TYPE_RISCV_AIA,
+#define KVM_DEV_TYPE_RISCV_AIA KVM_DEV_TYPE_RISCV_AIA
KVM_DEV_TYPE_MAX,
};
diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h
index ed281bdce0..606968a4b7 100644
--- a/target/riscv/kvm_riscv.h
+++ b/target/riscv/kvm_riscv.h
@@ -22,4 +22,37 @@
void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
+#define KVM_DEV_RISCV_AIA_GRP_CONFIG0
+#define KVM_DEV_RISCV_AIA_CONFIG_MODE 0
+#define KVM_DEV_RISCV_AIA_CONFIG_IDS1
+#define KVM_DEV_RISCV_AIA_CONFIG_SRCS 2
+#define KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS 3
+#define KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT4
+#define KVM_DEV_RISCV_AIA_CONFIG_HART_BITS 5
+#define KVM_DEV_RISCV_AIA_CONFIG_GUEST_BITS 6
+#define KVM_DEV_RISCV_AIA_MODE_EMUL 0
+#define KVM_DEV_RISCV_AIA_MODE_HWACCEL 1
+#define KVM_DEV_RISCV_AIA_MODE_AUTO 2
+#define KVM_DEV_RISCV_AIA_IDS_MIN 63
+#define KVM_DEV_RISCV_AIA_IDS_MAX 2048
+#define KVM_DEV_RISCV_AIA_SRCS_MAX 1024
+#define KVM_DEV_RISCV_AIA_GROUP_BITS_MAX8
+#define KVM_DEV_RISCV_AIA_GROUP_SHIFT_MIN 24
+#define KVM_DEV_RISCV_AIA_GROUP_SHIFT_MAX 56
+#define KVM_DEV_RISCV_AIA_HART_BITS_MAX 16
+#define KVM_DEV_RISCV_AIA_GUEST_BITS_MAX8
+
+#define KVM_DEV_RISCV_AIA_GRP_ADDR 1
+#define KVM_DEV_RISCV_AIA_ADDR_APLIC0
+#define KVM_DEV_RISCV_AIA_ADDR_IMSIC(__vcpu)(1 + (__vcpu))
+#define KVM_DEV_RISCV_AIA_ADDR_MAX \
+(1 + KVM_DEV_RISCV_APLIC_MAX_HARTS)
+
+#define KVM_DEV_RISCV_AIA_GRP_CTRL 2
+#define KVM_DEV_RISCV_AIA_CTRL_INIT 0
+
+#define KVM_DEV_RISCV_AIA_GRP_APLIC 3
+
+#define KVM_DEV_RISCV_AIA_GRP_IMSIC 4
+
#endif
--
2.17.1
[PATCH 6/6] target/riscv: select KVM AIA in riscv virt machine
Select KVM AIA when the host kernel has in-kernel AIA chip support.
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/riscv/virt.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 18aede7b23..908b304991 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
#include "hw/riscv/virt.h"
#include "hw/riscv/boot.h"
#include "hw/riscv/numa.h"
+#include "kvm_riscv.h"
#include "hw/intc/riscv_aclint.h"
#include "hw/intc/riscv_aplic.h"
#include "hw/intc/riscv_imsic.h"
@@ -1215,6 +1216,15 @@ static DeviceState *virt_create_aia(RISCVVirtAIAType
aia_type, int aia_guests,
msimode, false, aplic_m);
}
+if (kvm_irqchip_in_kernel()) {
+kvm_riscv_aia_create(
+aplic_s, msimode, socket,
+VIRT_IRQCHIP_NUM_SOURCES,
+hart_count,
+memmap[VIRT_APLIC_S].base + socket * memmap[VIRT_APLIC_S].size,
+memmap[VIRT_IMSIC_S].base + socket * VIRT_IMSIC_GROUP_MAX_SIZE);
+}
+
return kvm_enabled() ? aplic_s : aplic_m;
}
--
2.17.1
[PATCH 4/6] target/riscv: Create an KVM AIA irqchip
implement a function to create an KVM AIA chip
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
target/riscv/kvm.c | 83
target/riscv/kvm_riscv.h | 3 ++
2 files changed, 86 insertions(+)
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 6aad25bbc3..1c21f5a180 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -34,6 +34,7 @@
#include "exec/address-spaces.h"
#include "hw/boards.h"
#include "hw/irq.h"
+#include "hw/intc/riscv_imsic.h"
#include "qemu/log.h"
#include "hw/loader.h"
#include "kvm_riscv.h"
@@ -547,3 +548,85 @@ bool kvm_arch_cpu_check_are_resettable(void)
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
+
+void kvm_riscv_aia_create(DeviceState *aplic_s, bool msimode, int socket,
+ uint64_t aia_irq_num, uint64_t hart_count,
+ uint64_t aplic_base, uint64_t imsic_base)
+{
+int ret;
+int aia_fd = -1;
+uint64_t aia_mode;
+uint64_t aia_nr_ids;
+uint64_t aia_hart_bits = find_last_bit(_count, BITS_PER_LONG) + 1;
+
+if (!msimode) {
+error_report("Currently KVM AIA only supports aplic_imsic mode");
+exit(1);
+}
+
+aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false);
+
+if (aia_fd < 0) {
+error_report("Unable to create in-kernel irqchip");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_MODE,
+_mode, false, NULL);
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_IDS,
+_nr_ids, false, NULL);
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_SRCS,
+_irq_num, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number input irq lines");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
+KVM_DEV_RISCV_AIA_CONFIG_HART_BITS,
+_hart_bits, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set number of harts");
+exit(1);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+KVM_DEV_RISCV_AIA_ADDR_APLIC,
+_base, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set the base address of APLIC");
+exit(1);
+}
+
+for (int i = 0; i < hart_count; i++) {
+uint64_t imsic_addr = imsic_base + i * IMSIC_HART_SIZE(0);
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR,
+KVM_DEV_RISCV_AIA_ADDR_IMSIC(i),
+_addr, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: fail to set the base address of IMSICs");
+exit(1);
+}
+}
+
+if (kvm_has_gsi_routing()) {
+for (uint64_t idx = 0; idx < aia_irq_num + 1; ++idx) {
+kvm_irqchip_add_irq_route(kvm_state, idx, socket, idx);
+}
+kvm_gsi_routing_allowed = true;
+kvm_irqchip_commit_routes(kvm_state);
+}
+
+ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CTRL,
+KVM_DEV_RISCV_AIA_CTRL_INIT,
+NULL, true, NULL);
+if (ret < 0) {
+error_report("KVM AIA: initialized fail");
+exit(1);
+}
+}
diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h
index 606968a4b7..6067adff51 100644
--- a/target/riscv/kvm_riscv.h
+++ b/target/riscv/kvm_riscv.h
@@ -21,6 +21,9 @@
void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
+void kvm_riscv_aia_create(DeviceState *aplic_s, bool msimode, int socket,
+ uint64_t aia_irq_num, uint64_t hart_count,
+ uint64_t aplic_base, uint64_t imsic_base);
#define KVM_DEV_RISCV_AIA_GRP_CONFIG0
#define KVM_DEV_RISCV_AIA_CONFIG_MODE 0
--
2.17.1
[PATCH 5/6] target/riscv: update APLIC and IMSIC to support KVM AIA
- Do not set the mmio operations of APLIC and IMSIC when using KVM AIA
- Send interrupt signal to KVM AIA via KVM_IRQ_LINE API
Signed-off-by: Yong-Xuan Wang
Reviewed-by: Jim Shu
---
hw/intc/riscv_aplic.c | 19 +++
hw/intc/riscv_imsic.c | 16 +++-
2 files changed, 26 insertions(+), 9 deletions(-)
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index cfd007e629..5f68900e9a 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -31,6 +31,7 @@
#include "hw/irq.h"
#include "target/riscv/cpu.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define APLIC_MAX_IDC (1UL << 14)
@@ -479,6 +480,11 @@ static void riscv_aplic_request(void *opaque, int irq, int
level)
assert((0 < irq) && (irq < aplic->num_irqs));
+if (kvm_irqchip_in_kernel()) {
+kvm_set_irq(kvm_state, irq, !!level);
+return;
+}
+
sourcecfg = aplic->sourcecfg[irq];
if (sourcecfg & APLIC_SOURCECFG_D) {
childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
@@ -814,9 +820,11 @@ static void riscv_aplic_realize(DeviceState *dev, Error
**errp)
aplic->iforce = g_new0(uint32_t, aplic->num_harts);
aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
-memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops, aplic,
- TYPE_RISCV_APLIC, aplic->aperture_size);
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+if (!kvm_irqchip_in_kernel()) {
+memory_region_init_io(>mmio, OBJECT(dev), _aplic_ops,
+ aplic, TYPE_RISCV_APLIC, aplic->aperture_size);
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/*
* Only root APLICs have hardware IRQ lines. All non-root APLICs
@@ -958,7 +966,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
qdev_prop_set_bit(dev, "mmode", mmode);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!kvm_irqchip_in_kernel()) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
if (parent) {
riscv_aplic_add_child(parent, dev);
diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c
index 4d4d5b50ca..9b2a442b40 100644
--- a/hw/intc/riscv_imsic.c
+++ b/hw/intc/riscv_imsic.c
@@ -32,6 +32,7 @@
#include "target/riscv/cpu.h"
#include "target/riscv/cpu_bits.h"
#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
#include "migration/vmstate.h"
#define IMSIC_MMIO_PAGE_LE 0x00
@@ -325,10 +326,12 @@ static void riscv_imsic_realize(DeviceState *dev, Error
**errp)
imsic->eithreshold = g_new0(uint32_t, imsic->num_pages);
imsic->eistate = g_new0(uint32_t, imsic->num_eistate);
-memory_region_init_io(>mmio, OBJECT(dev), _imsic_ops,
- imsic, TYPE_RISCV_IMSIC,
- IMSIC_MMIO_SIZE(imsic->num_pages));
-sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+if (!kvm_irqchip_in_kernel()) {
+memory_region_init_io(>mmio, OBJECT(dev), _imsic_ops,
+ imsic, TYPE_RISCV_IMSIC,
+ IMSIC_MMIO_SIZE(imsic->num_pages));
+sysbus_init_mmio(SYS_BUS_DEVICE(dev), >mmio);
+}
/* Claim the CPU interrupt to be triggered by this IMSIC */
if (riscv_cpu_claim_interrupts(rcpu,
@@ -432,7 +435,10 @@ DeviceState *riscv_imsic_create(hwaddr addr, uint32_t
hartid, bool mmode,
qdev_prop_set_uint32(dev, "num-irqs", num_ids + 1);
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), _fatal);
-sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+
+if (!kvm_irqchip_in_kernel()) {
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
+}
for (i = 0; i < num_pages; i++) {
if (!i) {
--
2.17.1
[PATCH 0/6] Add RISC-V KVM AIA Support
This series introduces support for KVM AIA in the RISC-V architecture. The implementation is refered to Anup's KVM AIA implementation in kvmtool (https://github.com/avpatel/kvmtool.git). To test these patches, a Linux kernel with KVM AIA support is required, which can be found in the qemu_kvm_aia branch at https://github.com/yong-xuan/linux.git. This kernel branch is based on the riscv_aia_v1 branch from https://github.com/avpatel/linux.git and includes two additional patches. Yong-Xuan Wang (6): update-linux-headers: sync-up header with Linux for KVM AIA support target/riscv: support the AIA device emulateion with KVM enabled target/riscv: check the in-kernel irqchip support target/riscv: Create an KVM AIA irqchip target/riscv: update APLIC and IMSIC to support KVM AIA target/riscv: select KVM AIA in riscv virt machine hw/intc/riscv_aplic.c | 19 +++- hw/intc/riscv_imsic.c | 16 ++- hw/riscv/virt.c | 214 +- linux-headers/linux/kvm.h | 2 + target/riscv/kvm.c| 96 - target/riscv/kvm_riscv.h | 36 +++ 6 files changed, 277 insertions(+), 106 deletions(-) -- 2.17.1
