On 06.03.2013, at 15:48, Alexander Graf wrote:
>
> On 06.03.2013, at 15:41, Gleb Natapov wrote:
>
>> On Wed, Mar 06, 2013 at 03:03:53PM +0100, Alexander Graf wrote:
>>>
>>> On 06.03.2013, at 14:56, Gleb Natapov wrote:
>>>
>>>> On Wed, Mar 06, 2013 at 02:22:15PM +0100, Alexander Graf wrote:
>>>>>
>>>>> On 06.03.2013, at 14:14, Gleb Natapov wrote:
>>>>>
>>>>>> On Wed, Mar 06, 2013 at 01:20:39PM +0100, Alexander Graf wrote:
>>>>>>>> The problem would only start if KVM_SET_IRQCHIP_TYPE (new name of
>>>>>>>> KVM_CREATE_IRQCHIP_ARGS) forced you to later call KVM_CREATE_DEVICE.
>>>>>>>
>>>>>>> Ah, I see. I don't see why it would. The fact that there is a "LAPIC"
>>>>>>> doesn't mean that the per-vcpu SET_INTERRUPT ioctl stops working. So if
>>>>>>> SET_IRQCHIP_TYPE(!none) breaks user-space interrupt controller
>>>>>>> emulation I would consider that a bug.
>>>>>>>
>>>>>> For x86 this is the case though. I do not see how it can't be. If
>>>>>> LAPIC is emulated in userspace SET_INTERRUPT is used to pass IRQ
>>>>>> vector that should be handled as a result of LAPIC emulation.
>>>>>
>>>>> So SET_INTERRUPT on a vcpu triggers a line on the LAPIC emulation in that
>>>>> vcpu? For us it directly controls the CPU interrupt pin.
>>>>>
>>>> No SET_INTERRUPT on a vcpu tells vcpu to which vector in IDT it needs to
>>>> jump immediately. LAPIC is really part of a cpu and we cut it and put into
>>>> userspace, so interface between userspace LAPIC emulation is really low
>>>> level and has to be synchronous. X86 has two interrupt lines NMI and INTR
>>>> and we do not have interface to trigger the later. KVM_IRQ_LINE works on
>>>> GSI lines which do not go into CPU directly. They go either via PIC (which
>>>> triggers INTR or APIC LINT0) or via IOAPIC which on real HW communicates
>>>> with APICs via bus, but in our emulation just calls APICs directly.
>>>
>>> Great :). It's similar for us. SET_INTERRUPT directly asserts the INTR line
>>> of the vcpu. There is nothing like an IDT on PPC, so external interrupts
>>> simply arrive at a specific vector. That vector can differ for critical or
>>> NMI interrupts IIRC, but I'm not sure we implement that right now. If so,
>>> it'd be a different line for SET_INTERRUPT.
>>>
>>> So in a way, it's the same. And SET_INTERRUPT should work regardless of
>>> whether a LAPIC is used or not really. At least it would for us :).
>>>
>> Is it possible for some devices to inject interrupt directly and other
>> to go through interrupt controller?
>
> It would be racy if both assert + deassert the same line, but I don't see why
> we should keep anyone from doing it. If user space wants to run such a
> configuration, it needs to ensure that only one of the 2 is actively used at
> any given time.
>
>>> KVM_IRQ_LINE is basically an IOAPIC interrupt line assert. That's fine.
>>> That ioctl should get an ioapic device handle to work on. Whether we call
>>> the IOAPIC PINs GSIs or something different is really just a naming
>>> question. I'd probably call it IRQ number :).
>> Yes and no. On sane archs we can call it IRQ number (lucky you!), but on
>> X86 there is a GSI that can be IRQ2 if it goes through IOAPIC and IRQ0
>> if it goes through PIC, so additional entity was invented: irq routing.
>> It maps between GSI and irqchips pin. Same GSI may go to more than one
>> irqchip. This is why for x86 having irqchip device handle as a parameter
>> to KVM_IRQ_LINE does not make sense. It make sense to provide it to irq
>> router and this is how it work now except that "device handlers" are
>> hard coded.
>
> Then you would create a new "irq router" device that does the multiplexing
> and can also receive IRQs. You could then directly assert an IOAPIC/PIC line
> or a multiplexer line. Or am I misunderstanding something?
>
>>
>>> But it's the same idea. The "IOAPIC" would then talk to to in-kernel
>>> "LAPIC" style bits (or in case of the MPIC just integrate them inside of
>>> itself). That's why by the time we create an "IOAPIC", the "LAPIC"s in the
>>> system have to be populated.
>> The restriction that LAPIC has to be created before IOAPIC would be a
>> bug that need to be fixed on X86. The reason is cpu hotplug. If you have
>> to support cpu hotplug you have to be able to create LAPICs after IOAPIC
>> and at this point you can create IOAPIC before any LAPICs as well. I
>> understand this may not be the case for all architectures right now, but
>> something to keep in mind.
>
> Paul, Scott, do you think we can move the "this CPU can receive interrupts
> from MPIC / XICS" part into an ENABLE_CAP that gets set dynamically? That
> ENABLE_CAP would allocate the structures in the vcpu and register the vcpu
> with the interrupt controller pool.
>
> The interrupt controller device
creation
> would still iterate through all vcpus to find the ones that match so that we
> support the ENABLE_CAP at any point in time.
Actually, thinking about this a bit more. If we had explicit interrupt
connections, user space would take care of all this:
-- machine init --
for (i = 0; i < smp_cpus; i++) {
create_cpu();
}
mpic = create_device(DEVICE_MPIC)
for (i = 0; i < smp_cpus; i++) {
enable_cap(cpus[i], CAP_MPIC_LISTENER);
mpic_hook_up_irqline(i, cpus[i]);
}
-- hotplug add --
create_cpu();
enable_cap(cpus[i], CAP_MPIC_LISTENER);
mpic_hook_up_irqline(mpic, i, cpus[i]);
Then we don't care about any ordering at all anymore from KVM's perspective.
Alternatively, the above code could live inside kvm as well of course.
create_vcpu() would have to register itself with "the interrupt controller"
then to allow for hotplug.
Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
