>>>>> I'm thinking about the END->NVT stage of the process here, rather than
>>>>> the NVT->TCTX stage.
>>>>>
>>>>> Oh, also, you're using "VP" here which IIUC == "NVT". Can we
>>>>> standardize on one, please.
>>>>
>>>> VP is used in Linux/KVM Linux/Native and skiboot. Yes. it's a mess.
>>>> Let's have consistent naming in QEMU and use NVT.
>>>
>>> Right. And to cover any inevitable missed ones is why I'd like to see
>>> a cheatsheet giving both terms in the header comments somewhere.
>>
>> yes. I have added a list of names in xive.h.
>
> Great. Oh BTW - this is getting big enough, that I wonder if it makes
> sense to create a hw/intc/xive subdir to put things in, then splitting
> IVSE, IVRE, IVPE related code into separate .c files (I'd still expect
> a common .h though).
Here is what I have for now, for the new files only :
190 include/hw/ppc/xive_regs.h
343 include/hw/ppc/xive.h
78 include/hw/ppc/spapr_xive.h
1453 hw/intc/spapr_xive.c (sPAPRXive + hcalls)
1678 hw/intc/xive.c (XiveSource, XiveRouter, XiveENDSource, XiveTCTX,
END helpers)
864 hw/intc/spapr_xive_kvm.c
4606 total
I am putting the KVM export definitions in spapr_xive.h and xive.h but all
the code is in spapr_xive_kvm.c.
So you would rather have something like :
include/hw/ppc/xive_regs.h
include/hw/ppc/xive.h
include/hw/ppc/spapr_xive.h
hw/intc/xive/spapr.c
hw/intc/xive/spapr_kvm.c
hw/intc/xive/source.c
hw/intc/xive/router.c
hw/intc/xive/presenter.c
hw/intc/xive/tcontext.c
>> I was wondering if I should put the diagram below somewhere in a .h file
>> or under doc/specs/.
>
> I'd prefer it in the .h file.
OK. will do in xive.h
>>
>> Thanks,
>>
>> C.
>>
>>
>> = XIVE =================================================================
>>
>> The POWER9 processor comes with a new interrupt controller, called
>> XIVE as "eXternal Interrupt Virtualization Engine".
>>
>>
>> * Overall architecture
>>
>>
>> XIVE Interrupt Controller
>> +------------------------------------+ IPIs
>> | +---------+ +---------+ +--------+ | +-------+
>> | |VC | |CQ | |PC |----> | CORES |
>> | | esb | | | | |----> | |
>> | | eas | | Bridge | | tctx |----> | |
>> | |SC end | | | | nvt | | | |
>> +------+ | +---------+ +----+----+ +--------+ | +-+-+-+-+
>> | RAM | +------------------|-----------------+ | | |
>> | | | | | |
>> | | | | | |
>> | | +--------------------v------------------------v-v-v--+ other
>> | <--+ Power Bus +--> chips
>> | esb | +---------+-----------------------+------------------+
>> | eas | | |
>> | end | +---|-----+ |
>> | nvt | +----+----+| +----+----+
>> +------+ |SC || |SC |
>> | || | |
>> | PQ-bits || | PQ-bits |
>> | local |+ | in VC |
>> +---------+ +---------+
>> PCIe NX,NPU,CAPI
>>
>> SC: Source Controller (aka. IVSE)
>> VC: Virtualization Controller (aka. IVRE)
>> PC: Presentation Controller (aka. IVPE)
>> CQ: Common Queue (Bridge)
>>
>> PQ-bits: 2 bits source state machine (P:pending Q:queued)
>> esb: Event State Buffer (Array of PQ bits in an IVSE)
>> eas: Event Assignment Structure
>> end: Event Notification Descriptor
>> nvt: Notification Virtual Target
>> tctx: Thread interrupt Context
>>
>>
>> The XIVE IC is composed of three sub-engines :
>>
>> - Interrupt Virtualization Source Engine (IVSE), or Source
>> Controller (SC). These are found in PCI PHBs, in the PSI host
>> bridge controller, but also inside the main controller for the
>> core IPIs and other sub-chips (NX, CAP, NPU) of the
>> chip/processor. They are configured to feed the IVRE with events.
>>
>> - Interrupt Virtualization Routing Engine (IVRE) or Virtualization
>> Controller (VC). Its job is to match an event source with an Event
>> Notification Descriptor (END).
>>
>> - Interrupt Virtualization Presentation Engine (IVPE) or Presentation
>> Controller (PC). It maintains the interrupt context state of each
>> thread and handles the delivery of the external exception to the
>> thread.
>>
>>
>> * XIVE internal tables
>>
>> Each of the sub-engines uses a set of tables to redirect exceptions
>> from event sources to CPU threads.
>>
>> +-------+
>> User or OS | EQ |
>> or +------>|entries|
>> Hypervisor | | .. |
>> Memory | +-------+
>> | ^
>> | |
>> +-------------------------------------------------+
>> | |
>> Hypervisor +------+ +---+--+ +---+--+ +------+
>> Memory | ESB | | EAT | | ENDT | | NVTT |
>> (skiboot) +----+-+ +----+-+ +----+-+ +------+
>> ^ | ^ | ^ | ^
>> | | | | | | |
>> +-------------------------------------------------+
>> | | | | | | |
>> | | | | | | |
>> +----|--|--------|--|--------|--|-+ +-|-----+ +------+
>> | | | | | | | | | | tctx| |Thread|
>> IPI or ---+ + v + v + v |---| + .. |-----> |
>> HW events | | | | | |
>> | IVRE | | IVPE | +------+
>> +---------------------------------+ +-------+
>>
>>
>>
>> The IVSE have a 2-bits, P for pending and Q for queued, state machine
>> for each source that allows events to be triggered. They are stored in
>> an array, the Event State Buffer (ESB) and controlled by MMIOs.
>>
>> If the event is let through, the IVRE looks up in the Event Assignment
>> Structure (EAS) table for an Event Notification Descriptor (END)
>> configured for the source. Each Event Notification Descriptor defines
>> a notification path to a CPU and an in-memory Event Queue, in which
>> will be pushed an EQ data for the OS to pull.
>>
>> The IVPE determines if a Notification Virtual Target (NVT) can handle
>> the event by scanning the thread contexts of the VPs dispatched on the
>> processor HW threads. It maintains the interrupt context state of each
>> thread in a NVT table.
>>
>
