On 09/18/2012 11:06 AM, Gilles Chanteperdrix wrote: > On 09/18/2012 10:48 AM, Jan Kiszka wrote: >> On 2012-09-17 23:50, Gilles Chanteperdrix wrote: >>> On 09/17/2012 08:54 PM, Jan Kiszka wrote: >>> >>>> On 2012-09-17 20:37, Gilles Chanteperdrix wrote: >>>>> On 09/17/2012 08:29 PM, Jan Kiszka wrote: >>>>> >>>>>> On 2012-09-17 20:18, Gilles Chanteperdrix wrote: >>>>>>> On 09/17/2012 08:15 PM, Jan Kiszka wrote: >>>>>>> >>>>>>>> On 2012-09-17 20:12, Jan Kiszka wrote: >>>>>>>>> On 2012-09-17 20:08, Gilles Chanteperdrix wrote: >>>>>>>>>> On 09/17/2012 08:05 PM, Jan Kiszka wrote: >>>>>>>>>> >>>>>>>>>>> On 2012-09-17 19:46, Gilles Chanteperdrix wrote: >>>>>>>>>>>> ipipe_end is a nop when called from primary domain, yes, but this >>>>>>>>>>>> is not >>>>>>>>>>>> very different from edge irqs. Also, fasteoi become a bit like >>>>>>>>>>>> MSI: in >>>>>>>>>>>> the same way as we can not mask MSI from primary domain, we should >>>>>>>>>>>> not >>>>>>>>>>>> mask IO-APIC fasteoi irqs, because the cost is too prohibitive. If >>>>>>>>>>>> we >>>>>>>>>>>> can live with MSI without masking them in primary mode, I guess we >>>>>>>>>>>> can >>>>>>>>>>>> do the same with fasteoi irqs. >>>>>>>>>>> >>>>>>>>>>> MSIs are edge triggered, fasteois are still level-based. They >>>>>>>>>>> require >>>>>>>>>>> masking at the point you defer them - what we do and what Linux may >>>>>>>>>>> even >>>>>>>>>>> extend beyond that. If you mask them by raising the task priority, >>>>>>>>>>> you >>>>>>>>>>> have to keep it raised until Linux finally handled the IRQ. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes. >>>>>>>>>> >>>>>>>>>>> Or you >>>>>>>>>>> decide to mask it at IO-APIC level again. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> We do not want that. >>>>>>>>>> >>>>>>>>>>> If you keep the TPR raised, >>>>>>>>>>> you will block more than what Linux wants to block. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The point is that if the TPR keeps raised, it means that primary >>>>>>>>>> domain >>>>>>>>>> has preempted Linux, so, we want it to keep that way. Otherwise the >>>>>>>>>> TPR >>>>>>>>>> gets lowered when Linux has handled the interrupt. >>>>>>>>>> >>>>>>>>>> A week-end of testing made me sure of one thing: it works. I assure >>>>>>>>>> you. >>>>>>>>> >>>>>>>>> Probably, in the absence of IRQF_ONESHOT Linux interrupts. No longer >>>>>>>>> if >>>>>>>>> you face threaded IRQs - I assure you. >>>>>>>> >>>>>>>> Well, it may work (if mask/unmask callbacks work as native) but the >>>>>>>> benefit is gone: masking at IO-APIC level will be done again. Given >>>>>>>> that >>>>>>>> threaded IRQs become increasingly popular, it will also be hard to >>>>>>>> avoid >>>>>>>> them in common setups. >>>>>>> >>>>>>> >>>>>>> The thing is, if we no longer use the IO-APIC spinlock from primary >>>>>>> domain, we may not have to turn it into an ipipe_spinlock, and may be >>>>>>> able to preempt the IO-APIC masking. >>>>>> >>>>>> That might be true - but is the latency related to the lock or the >>>>>> hardware access? In the latter case, you will still stall the CPU on it >>>>>> and have to isolate the load on a non-RT CPU again. >>>>>> >>>>>> BTW, the task priority for the RT domain is a quite important parameter. >>>>>> If you put it too low, Linux can run out of vectors. If you put it too >>>>>> high, the same may happen to Xenomai - on bigger boxes. >>>>> >>>>> >>>>> Yes, and there are only 16 levels. But Xenomai does not need to many >>>>> levels. >>>> >>>> How is telling you this? It's part of the system setup. And that may >>>> lean toward RT or toward non-RT. This level should be adjusted according >>>> to the current allocation of Linux and the RT domain for a particular >>>> CPU, not hard-coded or compile-time defined. >>> >>> >>> In theory, I agree, in practice, lets be crasy, assume someone would >>> want an RT serial driver with 4 irqs, an RT USB driver with 2 irqs, an >>> RT CAN driver, and say, 4 RTnet boards. That is still less than the 16 >>> vectors that a single level provides, so, we can probably get along with >>> 2 levels. Or we can use a kernel parameter. >> >> Linux - and so should we do - allocates separate levels first as that >> provides better performance for external interrupts (need to look up the >> precise reason, should be documented in the x86 code). Only if levels >> are used up, interrupts will share them. > > I have seen this code, and I wondered if it was not, in fact, only > useful, where the irq flow handler were reenabling irqs (that is, before > the removal of IRQF_DISABLED), but am really not sure. > > Also, some additional results on my atom: > the IO-APIC is on IO controller HUB, which is... an ICH4 if I read lspci > and the datasheets correctly. And what is more, its registers are > accessed through the (slow) LPC bus, the ISA bus replacement. It is > probably the reason why it is so slow. > > And last but not least, it is not really a multi-core processor, it has > hyper-threading. Booting the processor in UP mode yields a much more > reasonable latency of 23us (still with using the TPR), whereas the usual > latency was around 30u (running the test now, will have results at > noon), so, the real gain of using the TPR is in fact much lower than > what originally announced. Basically, it seems with hyper threading, > everything is doubled. > > http://sisyphus.hd.free.fr/core-3.4-latencies/atom.png
The results are ready actually, so, the net gain is 6.5us over 30us, that is around 20%. -- Gilles. _______________________________________________ Xenomai mailing list Xenomai@xenomai.org http://www.xenomai.org/mailman/listinfo/xenomai