Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Karsten Wiese <[EMAIL PROTECTED]> wrote: > This is it. Apply it on top of -51-30 and 'make oldconfig'. You'll be > asked about the new CONFIG_ONE_IOAPIC. Answering yes, io_apic_one.c > will be used, which is ((io_apic.c minus quirks) minus > multi_io_apic_capability) . It'll propably crash some machines, as the > quirks are there for some reason... With CONFIG_ONE_IOAPIC unset, > you'll get the same kernel as with my previously sent patch. have fun > :-) this could only be offered if it's merged into the existing io_apic.c. The current IOAPIC_FAST option will go away eventually, but ONE_IOAPIC would/could be intentionally incompatible. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Daniel Walker <[EMAIL PROTECTED]> wrote: > On Tue, 2005-07-12 at 07:56 -0700, Daniel Walker wrote: > > On Tue, 2005-07-12 at 16:46 +0200, Ingo Molnar wrote: > > > * Daniel Walker <[EMAIL PROTECTED]> wrote: > > > > > > > I haven't tested it recently . This was on an older version of RT > > > > though . I could try it if it's interesting ? Or do you think it's > > > > already fixed? > > > > > > it would be definitely interesting to see how robust the latest IO-APIC > > > code is. > > > > I'm sure you test that all the time, but I'll try it .. > > I tried it, and it appears to be fixed in current RT .. cool, thanks. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Daniel Walker [EMAIL PROTECTED] wrote: On Tue, 2005-07-12 at 07:56 -0700, Daniel Walker wrote: On Tue, 2005-07-12 at 16:46 +0200, Ingo Molnar wrote: * Daniel Walker [EMAIL PROTECTED] wrote: I haven't tested it recently . This was on an older version of RT though . I could try it if it's interesting ? Or do you think it's already fixed? it would be definitely interesting to see how robust the latest IO-APIC code is. I'm sure you test that all the time, but I'll try it .. I tried it, and it appears to be fixed in current RT .. cool, thanks. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Karsten Wiese [EMAIL PROTECTED] wrote: This is it. Apply it on top of -51-30 and 'make oldconfig'. You'll be asked about the new CONFIG_ONE_IOAPIC. Answering yes, io_apic_one.c will be used, which is ((io_apic.c minus quirks) minus multi_io_apic_capability) . It'll propably crash some machines, as the quirks are there for some reason... With CONFIG_ONE_IOAPIC unset, you'll get the same kernel as with my previously sent patch. have fun :-) this could only be offered if it's merged into the existing io_apic.c. The current IOAPIC_FAST option will go away eventually, but ONE_IOAPIC would/could be intentionally incompatible. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 07:56 -0700, Daniel Walker wrote: > On Tue, 2005-07-12 at 16:46 +0200, Ingo Molnar wrote: > > * Daniel Walker <[EMAIL PROTECTED]> wrote: > > > > > I haven't tested it recently . This was on an older version of RT > > > though . I could try it if it's interesting ? Or do you think it's > > > already fixed? > > > > it would be definitely interesting to see how robust the latest IO-APIC > > code is. > > I'm sure you test that all the time, but I'll try it .. I tried it, and it appears to be fixed in current RT .. Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Am Mittwoch, 13. Juli 2005 02:08 schrieb William Weston: > On Wed, 13 Jul 2005, Karsten Wiese wrote: > > > i've only tested on 2005ish [EMAIL PROTECTED]: it doesn't need any of the > > quirks > > IOAPIC_POSTFLUSH, sis_bug, level-edge cleanup. > > IOAPIC_POSTFLUSH caused no negative influence neither. > > i've an io_apic_one.c here, that doesn't have any of the quirks and is > > stripped down > > to handle just one ioapic. It runs just fine on PREEMPT_RT. > > Could be used as a "Do i crash/suffer without the quirks?" test for 1 > > ioapic equipped machines. > > Should i post it? > > Please do. I'll give it a spin on several machines and let you know how > it goes ;-} > This is it. Apply it on top of -51-30 and 'make oldconfig'. You'll be asked about the new CONFIG_ONE_IOAPIC. Answering yes, io_apic_one.c will be used, which is ((io_apic.c minus quirks) minus multi_io_apic_capability) . It'll propably crash some machines, as the quirks are there for some reason... With CONFIG_ONE_IOAPIC unset, you'll get the same kernel as with my previously sent patch. have fun :-) Karsten io_apic-RT-51-30+io_apic_one.bz2 Description: BZip2 compressed data
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Am Mittwoch, 13. Juli 2005 02:08 schrieb William Weston: On Wed, 13 Jul 2005, Karsten Wiese wrote: i've only tested on 2005ish [EMAIL PROTECTED]: it doesn't need any of the quirks IOAPIC_POSTFLUSH, sis_bug, level-edge cleanup. IOAPIC_POSTFLUSH caused no negative influence neither. i've an io_apic_one.c here, that doesn't have any of the quirks and is stripped down to handle just one ioapic. It runs just fine on PREEMPT_RT. Could be used as a Do i crash/suffer without the quirks? test for 1 ioapic equipped machines. Should i post it? Please do. I'll give it a spin on several machines and let you know how it goes ;-} This is it. Apply it on top of -51-30 and 'make oldconfig'. You'll be asked about the new CONFIG_ONE_IOAPIC. Answering yes, io_apic_one.c will be used, which is ((io_apic.c minus quirks) minus multi_io_apic_capability) . It'll propably crash some machines, as the quirks are there for some reason... With CONFIG_ONE_IOAPIC unset, you'll get the same kernel as with my previously sent patch. have fun :-) Karsten io_apic-RT-51-30+io_apic_one.bz2 Description: BZip2 compressed data
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 07:56 -0700, Daniel Walker wrote: On Tue, 2005-07-12 at 16:46 +0200, Ingo Molnar wrote: * Daniel Walker [EMAIL PROTECTED] wrote: I haven't tested it recently . This was on an older version of RT though . I could try it if it's interesting ? Or do you think it's already fixed? it would be definitely interesting to see how robust the latest IO-APIC code is. I'm sure you test that all the time, but I'll try it .. I tried it, and it appears to be fixed in current RT .. Daniel - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Wed, 2005-07-13 at 08:35 +0200, Ingo Molnar wrote: > * Lee Revell <[EMAIL PROTECTED]> wrote: > > > > I have found that heavy network traffic really kills the interactive > > > performance. In the top excerpt below, gtk-gnutella is generating about > > > 320KB/sec of traffic. > > > > > > These priorities do not look right: > > > > Never mind, I just rediscovered the RT priority leakage bug. > > ok. And is it fixed by recent kernels? > Yes, I have not seen the problem yet with V0.7.51-28. Lee - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Lee Revell <[EMAIL PROTECTED]> wrote: > > I have found that heavy network traffic really kills the interactive > > performance. In the top excerpt below, gtk-gnutella is generating about > > 320KB/sec of traffic. > > > > These priorities do not look right: > > Never mind, I just rediscovered the RT priority leakage bug. ok. And is it fixed by recent kernels? Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Lee Revell [EMAIL PROTECTED] wrote: I have found that heavy network traffic really kills the interactive performance. In the top excerpt below, gtk-gnutella is generating about 320KB/sec of traffic. These priorities do not look right: Never mind, I just rediscovered the RT priority leakage bug. ok. And is it fixed by recent kernels? Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Wed, 2005-07-13 at 08:35 +0200, Ingo Molnar wrote: * Lee Revell [EMAIL PROTECTED] wrote: I have found that heavy network traffic really kills the interactive performance. In the top excerpt below, gtk-gnutella is generating about 320KB/sec of traffic. These priorities do not look right: Never mind, I just rediscovered the RT priority leakage bug. ok. And is it fixed by recent kernels? Yes, I have not seen the problem yet with V0.7.51-28. Lee - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 21:18 -0400, Lee Revell wrote: > On Mon, 2005-07-04 at 13:16 +0200, Ingo Molnar wrote: > > i'd first suggest to try the latest kernel, before changing your X > > config - i think the bug might have been fixed meanwhile. > > I have found that heavy network traffic really kills the interactive > performance. In the top excerpt below, gtk-gnutella is generating about > 320KB/sec of traffic. > > These priorities do not look right: Never mind, I just rediscovered the RT priority leakage bug. Lee - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Mon, 2005-07-04 at 13:16 +0200, Ingo Molnar wrote: > i'd first suggest to try the latest kernel, before changing your X > config - i think the bug might have been fixed meanwhile. I have found that heavy network traffic really kills the interactive performance. In the top excerpt below, gtk-gnutella is generating about 320KB/sec of traffic. These priorities do not look right: 14 root -50 -5 00 S 0.0 0.0 0:00.000000 [IRQ 9]irqd 686 root -49 -5 00 S 0.0 0.0 87:58.180000 [IRQ 8]irqd 689 root -48 -5 00 S 0.0 0.0 0:00.000000 [IRQ 7]irqd 694 root -47 -5 00 S 0.0 0.0 0:00.000000 [IRQ 12] irqd 714 root -46 -5 00 S 2.5 0.0 5:25.330000 [IRQ 15] irqd 731 root -45 -5 00 S 0.0 0.0 0:04.740000 [IRQ 1]irqd 3 root -44 -10 00 S 0.0 0.0 1:42.550000 [softirq-timer/0] ksoftirqd 5 root -44 -10 00 S 2.5 0.0 5:06.490000 [softirq-net-rx/] ksoftirqd 1541 root -44 -5 00 S 0.0 0.0 9:39.030000 [IRQ 10] irqd 5650 rlrevell -44 0 2804 1616 S 14.8 0.4 5:52.83 1188 52 7841 /usr/lib/gamin/gam_server stext 30313 rlrevell -44 0 24572 19m S 99.9 4.4 68:15.42 4520 1800 12m 31 gtk-gnutella stext 2285 root -43 -5 00 S 0.0 0.0 3:47.340000 [IRQ 11] irqd 2807 root -42 -5 00 S 0.0 0.0 0:00.000000 [IRQ 14] irqd 9 root -2 -5 00 S 0.0 0.0 0:16.050000 [events/0] worker_th 4 root 5 -10 00 S 0.0 0.0 0:25.950000 [softirq-net-tx/] ksoftirqd 7 root 5 -10 00 S 0.0 0.0 0:01.220000 [softirq-tasklet] ksoftirqd 8 root 5 -10 00 S 0.0 0.0 0:01.300000 [desched/0]desched_t 28988 root 5 -10 106m 30m S 4.9 6.9 242:59.04 76m 1556 32m3 /usr/X11R6/bin/X :0 -audit 0 -auth /var/lib/gd stext 2 root 10 -10 00 S 0.0 0.0 0:00.000000 [softirq-high/0] ksoftirqd 6 root 10 -10 00 S 0.0 0.0 0:00.000000 [softirq-scsi/0] Why do gtk-gnutella and gam_server get a higher priority than the disk (14) and network (11) IRQs? And shouldn't the softirq threads run at lower priority than all the hardirq threads? Looking at the top output I'd expect the machine to be crawling, and that's what it does. Lee - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Wed, 13 Jul 2005, Karsten Wiese wrote: > i've only tested on 2005ish [EMAIL PROTECTED]: it doesn't need any of the > quirks > IOAPIC_POSTFLUSH, sis_bug, level-edge cleanup. > IOAPIC_POSTFLUSH caused no negative influence neither. > i've an io_apic_one.c here, that doesn't have any of the quirks and is > stripped down > to handle just one ioapic. It runs just fine on PREEMPT_RT. > Could be used as a "Do i crash/suffer without the quirks?" test for 1 ioapic > equipped machines. > Should i post it? Please do. I'll give it a spin on several machines and let you know how it goes ;-} --ww - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 12 Jul 2005, Ingo Molnar wrote: > * K.R. Foley <[EMAIL PROTECTED]> wrote: > > > Is this why I have been able to boot the latest versions without the > > noapic option (and without noticeable keyboard repeat problems) or has > > it just been dumb luck? > > yes, i think it's related - the IO-APIC code is now more robust than > ever, and that's why any known-broken system would be important to > re-check. Just brought the SIS based Xeon box up on -51-28 with CONFIG_IOAPIC_FAST and without the noapic option, and all is well. On -50-43, noapic was needed to avoid the spurious interrupt and APIC error on CPU0 messages. Are there any non-SIS chipsets out there that need the sis_apic_bug checks? What about IOAPIC_POSTFLUSH? --ww - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Am Dienstag, 12. Juli 2005 16:25 schrieb Daniel Walker: > On Tue, 2005-07-12 at 16:02 +0200, Ingo Molnar wrote: > > * Karsten Wiese <[EMAIL PROTECTED]> wrote: > > > > > Hi Ingo > > > > > > I've refined io_apic.c a little more: > > > > great. I've applied these changes and have released the -28 patch. (note > > that the last chunk of your patch was malformed, have applied it by > > hand.) > > > > i'm wondering what your thoughts are about IOAPIC_POSTFLUSH - i had to > > turn it on unconditionally again, to get rid of spurious interrupts and > > outright interrupt storms (and resulting lockups) on some systems. > > IOAPIC_POSTFLUSH is now causing much of the IO-APIC related IRQ handling > > overhead. > > I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, > would actually cause spurious interrupts. It was odd cause it's suppose > to stop them .. If there was a lot of interrupt traffic on one IRQ , it > would cause interrupt traffic on another IRQ. This would result in > "nobody cared" messages , and the storming IRQ line would get shutdown. > > This would only happen in PREEMPT_RT . > i've only tested on 2005ish [EMAIL PROTECTED]: it doesn't need any of the quirks IOAPIC_POSTFLUSH, sis_bug, level-edge cleanup. IOAPIC_POSTFLUSH caused no negative influence neither. i've an io_apic_one.c here, that doesn't have any of the quirks and is stripped down to handle just one ioapic. It runs just fine on PREEMPT_RT. Could be used as a "Do i crash/suffer without the quirks?" test for 1 ioapic equipped machines. Should i post it? On vanilla level-interrupts behave "nervously" showing the "doubled rate" effect: For level interrupts vanilla just sends an EOI to the apic. Nothing else should be necessary (io)apic wise. Only here, when the edge-level triggered hardware-handler acknowledges its client (== resets the interrupt line) the doubled level interrupt is triggered on the same pin. Thus the same interrupt routine runs again as soon as EOI has been sent and IFlag is open again. hardware-handler receives it, does nothing and returns with "uninterested"
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Ingo Molnar wrote: * K.R. Foley <[EMAIL PROTECTED]> wrote: Ingo Molnar wrote: * Daniel Walker <[EMAIL PROTECTED]> wrote: I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in "nobody cared" messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. Ingo Is this why I have been able to boot the latest versions without the noapic option (and without noticeable keyboard repeat problems) or has it just been dumb luck? yes, i think it's related - the IO-APIC code is now more robust than ever, and that's why any known-broken system would be important to re-check. Ingo Well I have booted -27 a couple of times and -28 once now without supplying the noapic boot option and I haven't seen any of the keyboard repeat problems that I reported late last week. This was on my dual 2.6 Xeon w/HT. I have never seen this behavior on any of my older systems. Because of the fact that the problem showed up sporadically, I can't say for sure that it is gone. However, so far so good. I will report any changes that I see. -- kr - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* K.R. Foley <[EMAIL PROTECTED]> wrote: > Ingo Molnar wrote: > >* Daniel Walker <[EMAIL PROTECTED]> wrote: > > > > > >>I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, > >>would actually cause spurious interrupts. It was odd cause it's > >>suppose to stop them .. If there was a lot of interrupt traffic on one > >>IRQ , it would cause interrupt traffic on another IRQ. This would > >>result in "nobody cared" messages , and the storming IRQ line would > >>get shutdown. > >> > >>This would only happen in PREEMPT_RT . > > > > > >does it happen with the latest kernel too? There were a couple of things > >broken in the IOAPIC code in various earlier versions. > > > > Ingo > > Is this why I have been able to boot the latest versions without the > noapic option (and without noticeable keyboard repeat problems) or has > it just been dumb luck? yes, i think it's related - the IO-APIC code is now more robust than ever, and that's why any known-broken system would be important to re-check. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 16:46 +0200, Ingo Molnar wrote: > * Daniel Walker <[EMAIL PROTECTED]> wrote: > > > I haven't tested it recently . This was on an older version of RT > > though . I could try it if it's interesting ? Or do you think it's > > already fixed? > > it would be definitely interesting to see how robust the latest IO-APIC > code is. I'm sure you test that all the time, but I'll try it .. Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Ingo Molnar wrote: * Daniel Walker <[EMAIL PROTECTED]> wrote: I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in "nobody cared" messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. Ingo Is this why I have been able to boot the latest versions without the noapic option (and without noticeable keyboard repeat problems) or has it just been dumb luck? -- kr - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Daniel Walker <[EMAIL PROTECTED]> wrote: > I haven't tested it recently . This was on an older version of RT > though . I could try it if it's interesting ? Or do you think it's > already fixed? it would be definitely interesting to see how robust the latest IO-APIC code is. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 16:28 +0200, Ingo Molnar wrote: > * Daniel Walker <[EMAIL PROTECTED]> wrote: > > > I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, > > would actually cause spurious interrupts. It was odd cause it's > > suppose to stop them .. If there was a lot of interrupt traffic on one > > IRQ , it would cause interrupt traffic on another IRQ. This would > > result in "nobody cared" messages , and the storming IRQ line would > > get shutdown. > > > > This would only happen in PREEMPT_RT . > > does it happen with the latest kernel too? There were a couple of things > broken in the IOAPIC code in various earlier versions. I haven't tested it recently . This was on an older version of RT though . I could try it if it's interesting ? Or do you think it's already fixed? Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Daniel Walker <[EMAIL PROTECTED]> wrote: > I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, > would actually cause spurious interrupts. It was odd cause it's > suppose to stop them .. If there was a lot of interrupt traffic on one > IRQ , it would cause interrupt traffic on another IRQ. This would > result in "nobody cared" messages , and the storming IRQ line would > get shutdown. > > This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 16:02 +0200, Ingo Molnar wrote: > * Karsten Wiese <[EMAIL PROTECTED]> wrote: > > > Hi Ingo > > > > I've refined io_apic.c a little more: > > great. I've applied these changes and have released the -28 patch. (note > that the last chunk of your patch was malformed, have applied it by > hand.) > > i'm wondering what your thoughts are about IOAPIC_POSTFLUSH - i had to > turn it on unconditionally again, to get rid of spurious interrupts and > outright interrupt storms (and resulting lockups) on some systems. > IOAPIC_POSTFLUSH is now causing much of the IO-APIC related IRQ handling > overhead. I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in "nobody cared" messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Karsten Wiese <[EMAIL PROTECTED]> wrote: > Hi Ingo > > I've refined io_apic.c a little more: great. I've applied these changes and have released the -28 patch. (note that the last chunk of your patch was malformed, have applied it by hand.) i'm wondering what your thoughts are about IOAPIC_POSTFLUSH - i had to turn it on unconditionally again, to get rid of spurious interrupts and outright interrupt storms (and resulting lockups) on some systems. IOAPIC_POSTFLUSH is now causing much of the IO-APIC related IRQ handling overhead. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Hi Ingo
I've refined io_apic.c a little more:
- merged all ioapic data into a struct ioapic_data_struct
- Allocate the ioapic cache memory from bootmem. Allocate only,
what is needed ((nr of registers + 5)*sizeod(u64)).
- Removed "cache overrun by reg out of bounds" check.
this should never happen, as reg is calculated out of irq_2_pin[irq].pin.
- Do the index lookup
struct ioapic_data_struct *ioapic = ioapic_data[apic];
as early and as rarely as possible.
In the consequence many functions parameter changed from
io_apic_*(unsigned int apic,
to
io_apic_*(struct ioapic_data_struct *ioapic,
- New function setup_IO_APIC_early(int _ioapic).
It does the setup of the ioapic's struct ioapic_data_struct.
if IOAPIC_CACHE is defined, it calls the also new function
ioapic_cache_init().
ioapic_cache_init() initializes the cache completely.
Thereby we may safely omit to query a caches elements validity before
reading it.
Works fine here applied against -51-27
Karsten
diff -ur ../linux-2.6.12-RT-51-23/arch/i386/kernel/io_apic.c ./arch/i386/kernel/io_apic.c
--- ../linux-2.6.12-RT-51-23/arch/i386/kernel/io_apic.c 2005-07-09 23:49:21.0 +0200
+++ ./arch/i386/kernel/io_apic.c 2005-07-12 01:08:34.0 +0200
@@ -31,6 +31,7 @@
#include
#include
#include
+#include
#include
#include
@@ -55,11 +56,6 @@
int sis_apic_bug = -1;
/*
- * # of IRQ routing registers
- */
-int nr_ioapic_registers[MAX_IO_APICS];
-
-/*
* Rough estimation of how many shared IRQs there are, can
* be changed anytime.
*/
@@ -132,88 +128,74 @@
# define IOAPIC_CACHE
#endif
-#ifdef IOAPIC_CACHE
-# define MAX_IOAPIC_CACHE 512
-/*
- * Cache register values:
- */
-static struct {
- unsigned int reg;
- unsigned int val[MAX_IOAPIC_CACHE];
-} io_apic_cache[MAX_IO_APICS]
- cacheline_aligned_in_smp;
+
+struct ioapic_data_struct {
+ struct sys_device dev;
+ int nr_registers; // # of IRQ routing registers
+ volatile unsigned int *base;
+ struct IO_APIC_route_entry *entry;
+#ifdef IOAPIC_CACHE
+ unsigned int reg_set;
+ u32 cached_val[0];
#endif
+};
-volatile unsigned int *io_apic_base[MAX_IO_APICS];
+static struct ioapic_data_struct *ioapic_data[MAX_IO_APICS];
-static inline unsigned int __raw_io_apic_read(unsigned int apic, unsigned int reg)
+
+static inline unsigned int __raw_io_apic_read(struct ioapic_data_struct *ioapic, unsigned int reg)
{
- volatile unsigned int *io_apic;
-#ifdef IOAPIC_CACHE
- io_apic_cache[apic].reg = reg;
-#endif
- io_apic = io_apic_base[apic];
- io_apic[0] = reg;
- return io_apic[4];
+# ifdef IOAPIC_CACHE
+ ioapic->reg_set = reg;
+# endif
+ ioapic->base[0] = reg;
+ return ioapic->base[4];
}
-unsigned int raw_io_apic_read(unsigned int apic, unsigned int reg)
+
+# ifdef IOAPIC_CACHE
+static void __init ioapic_cache_init(struct ioapic_data_struct *ioapic)
{
- unsigned int val = __raw_io_apic_read(apic, reg);
+ int reg;
+ for (reg = 0; reg < (ioapic->nr_registers + 10); reg++)
+ ioapic->cached_val[reg] = __raw_io_apic_read(ioapic, reg);
+}
+# endif
-#ifdef IOAPIC_CACHE
- io_apic_cache[apic].val[reg] = val;
-#endif
+
+static unsigned int raw_io_apic_read(struct ioapic_data_struct *ioapic, unsigned int reg)
+{
+ unsigned int val = __raw_io_apic_read(ioapic, reg);
+
+# ifdef IOAPIC_CACHE
+ ioapic->cached_val[reg] = val;
+# endif
return val;
}
-unsigned int io_apic_read(unsigned int apic, unsigned int reg)
+static unsigned int io_apic_read(struct ioapic_data_struct *ioapic, unsigned int reg)
{
-#ifdef IOAPIC_CACHE
- if (unlikely(reg >= MAX_IOAPIC_CACHE)) {
- static int once = 1;
-
- if (once) {
- once = 0;
- printk("WARNING: ioapic register cache overflow: %d.\n",
-reg);
- dump_stack();
- }
- return __raw_io_apic_read(apic, reg);
- }
- if (io_apic_cache[apic].val[reg] && !sis_apic_bug) {
- io_apic_cache[apic].reg = -1;
- return io_apic_cache[apic].val[reg];
+# ifdef IOAPIC_CACHE
+ if (likely(!sis_apic_bug)) {
+ ioapic->reg_set = -1;
+ return ioapic->cached_val[reg];
}
-#endif
- return raw_io_apic_read(apic, reg);
+# endif
+ return raw_io_apic_read(ioapic, reg);
}
-void io_apic_write(unsigned int apic, unsigned int reg, unsigned int val)
+static void io_apic_write(struct ioapic_data_struct *ioapic, unsigned int reg, unsigned int val)
{
- volatile unsigned int *io_apic;
-#ifdef IOAPIC_CACHE
- if (unlikely(reg >= MAX_IOAPIC_CACHE)) {
- static int once = 1;
-
- if (once) {
- once = 0;
- printk("WARNING: ioapic register cache overflow: %d.\n",
-reg);
- dump_stack();
- }
- } else
- io_apic_cache[apic].val[reg] = val;
-#endif
- io_apic = io_apic_base[apic];
-#ifdef IOAPIC_CACHE
- io_apic_cache[apic].reg = reg;
-#endif
- io_apic[0] = reg;
- io_apic[4] = val;
+# ifdef IOAPIC_CACHE
+ ioapic->cached_val[reg] = val;
+ ioapic->reg_set = reg;
+# endif
+ ioapic->base[0] = reg;
+ ioapic->base[4] = val;
}
+
/*
* Some systems need a POST flush or else level-triggered interrupts
* generate lots of spurious
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Am Dienstag, 12. Juli 2005 16:25 schrieb Daniel Walker: On Tue, 2005-07-12 at 16:02 +0200, Ingo Molnar wrote: * Karsten Wiese [EMAIL PROTECTED] wrote: Hi Ingo I've refined io_apic.c a little more: great. I've applied these changes and have released the -28 patch. (note that the last chunk of your patch was malformed, have applied it by hand.) i'm wondering what your thoughts are about IOAPIC_POSTFLUSH - i had to turn it on unconditionally again, to get rid of spurious interrupts and outright interrupt storms (and resulting lockups) on some systems. IOAPIC_POSTFLUSH is now causing much of the IO-APIC related IRQ handling overhead. I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in nobody cared messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . i've only tested on 2005ish [EMAIL PROTECTED]: it doesn't need any of the quirks IOAPIC_POSTFLUSH, sis_bug, level-edge cleanup. IOAPIC_POSTFLUSH caused no negative influence neither. i've an io_apic_one.c here, that doesn't have any of the quirks and is stripped down to handle just one ioapic. It runs just fine on PREEMPT_RT. Could be used as a Do i crash/suffer without the quirks? test for 1 ioapic equipped machines. Should i post it? On vanilla level-interrupts behave nervously showing the doubled rate effect: For level interrupts vanilla just sends an EOI to the apic. Nothing else should be necessary (io)apic wise. Only here, when the edge-level triggered hardware-handler acknowledges its client (== resets the interrupt line) the doubled level interrupt is triggered on the same pin. Thus the same interrupt routine runs again as soon as EOI has been sent and IFlag is open again. hardware-handler receives it, does nothing and returns with uninterested insert correct macro here. everything works correctly though. its just some cycles wasted. Odd, no? I found that the correct interrupt handling can be achieve by obeying the PREEMPT_RT method: level-interrupt triggers ioapic pin is masked harware-handler deasserts pin/line apic gets eoi ioapic pin is unmasked this could be optimized by ioapic caching , but there must be some configuration tweak.. that i don't know of. there are no via docs for me. erm.. anybody sends me a link _offlist_ ?-) Or even _knows that tweak_ ? And again IOAPIC_POSTFLUSH has no influence here on the doubled interrupt (VIA K8T800 only ?) bug. Karsten ___ Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 12 Jul 2005, Ingo Molnar wrote: * K.R. Foley [EMAIL PROTECTED] wrote: Is this why I have been able to boot the latest versions without the noapic option (and without noticeable keyboard repeat problems) or has it just been dumb luck? yes, i think it's related - the IO-APIC code is now more robust than ever, and that's why any known-broken system would be important to re-check. Just brought the SIS based Xeon box up on -51-28 with CONFIG_IOAPIC_FAST and without the noapic option, and all is well. On -50-43, noapic was needed to avoid the spurious interrupt and APIC error on CPU0 messages. Are there any non-SIS chipsets out there that need the sis_apic_bug checks? What about IOAPIC_POSTFLUSH? --ww - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Wed, 13 Jul 2005, Karsten Wiese wrote: i've only tested on 2005ish [EMAIL PROTECTED]: it doesn't need any of the quirks IOAPIC_POSTFLUSH, sis_bug, level-edge cleanup. IOAPIC_POSTFLUSH caused no negative influence neither. i've an io_apic_one.c here, that doesn't have any of the quirks and is stripped down to handle just one ioapic. It runs just fine on PREEMPT_RT. Could be used as a Do i crash/suffer without the quirks? test for 1 ioapic equipped machines. Should i post it? Please do. I'll give it a spin on several machines and let you know how it goes ;-} --ww - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Mon, 2005-07-04 at 13:16 +0200, Ingo Molnar wrote: i'd first suggest to try the latest kernel, before changing your X config - i think the bug might have been fixed meanwhile. I have found that heavy network traffic really kills the interactive performance. In the top excerpt below, gtk-gnutella is generating about 320KB/sec of traffic. These priorities do not look right: 14 root -50 -5 00 S 0.0 0.0 0:00.000000 [IRQ 9]irqd 686 root -49 -5 00 S 0.0 0.0 87:58.180000 [IRQ 8]irqd 689 root -48 -5 00 S 0.0 0.0 0:00.000000 [IRQ 7]irqd 694 root -47 -5 00 S 0.0 0.0 0:00.000000 [IRQ 12] irqd 714 root -46 -5 00 S 2.5 0.0 5:25.330000 [IRQ 15] irqd 731 root -45 -5 00 S 0.0 0.0 0:04.740000 [IRQ 1]irqd 3 root -44 -10 00 S 0.0 0.0 1:42.550000 [softirq-timer/0] ksoftirqd 5 root -44 -10 00 S 2.5 0.0 5:06.490000 [softirq-net-rx/] ksoftirqd 1541 root -44 -5 00 S 0.0 0.0 9:39.030000 [IRQ 10] irqd 5650 rlrevell -44 0 2804 1616 S 14.8 0.4 5:52.83 1188 52 7841 /usr/lib/gamin/gam_server stext 30313 rlrevell -44 0 24572 19m S 99.9 4.4 68:15.42 4520 1800 12m 31 gtk-gnutella stext 2285 root -43 -5 00 S 0.0 0.0 3:47.340000 [IRQ 11] irqd 2807 root -42 -5 00 S 0.0 0.0 0:00.000000 [IRQ 14] irqd 9 root -2 -5 00 S 0.0 0.0 0:16.050000 [events/0] worker_th 4 root 5 -10 00 S 0.0 0.0 0:25.950000 [softirq-net-tx/] ksoftirqd 7 root 5 -10 00 S 0.0 0.0 0:01.220000 [softirq-tasklet] ksoftirqd 8 root 5 -10 00 S 0.0 0.0 0:01.300000 [desched/0]desched_t 28988 root 5 -10 106m 30m S 4.9 6.9 242:59.04 76m 1556 32m3 /usr/X11R6/bin/X :0 -audit 0 -auth /var/lib/gd stext 2 root 10 -10 00 S 0.0 0.0 0:00.000000 [softirq-high/0] ksoftirqd 6 root 10 -10 00 S 0.0 0.0 0:00.000000 [softirq-scsi/0] Why do gtk-gnutella and gam_server get a higher priority than the disk (14) and network (11) IRQs? And shouldn't the softirq threads run at lower priority than all the hardirq threads? Looking at the top output I'd expect the machine to be crawling, and that's what it does. Lee - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 21:18 -0400, Lee Revell wrote: On Mon, 2005-07-04 at 13:16 +0200, Ingo Molnar wrote: i'd first suggest to try the latest kernel, before changing your X config - i think the bug might have been fixed meanwhile. I have found that heavy network traffic really kills the interactive performance. In the top excerpt below, gtk-gnutella is generating about 320KB/sec of traffic. These priorities do not look right: Never mind, I just rediscovered the RT priority leakage bug. Lee - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Hi Ingo
I've refined io_apic.c a little more:
- merged all ioapic data into a struct ioapic_data_struct
- Allocate the ioapic cache memory from bootmem. Allocate only,
what is needed ((nr of registers + 5)*sizeod(u64)).
- Removed cache overrun by reg out of bounds check.
this should never happen, as reg is calculated out of irq_2_pin[irq].pin.
- Do the index lookup
struct ioapic_data_struct *ioapic = ioapic_data[apic];
as early and as rarely as possible.
In the consequence many functions parameter changed from
io_apic_*(unsigned int apic,
to
io_apic_*(struct ioapic_data_struct *ioapic,
- New function setup_IO_APIC_early(int _ioapic).
It does the setup of the ioapic's struct ioapic_data_struct.
if IOAPIC_CACHE is defined, it calls the also new function
ioapic_cache_init().
ioapic_cache_init() initializes the cache completely.
Thereby we may safely omit to query a caches elements validity before
reading it.
Works fine here applied against -51-27
Karsten
diff -ur ../linux-2.6.12-RT-51-23/arch/i386/kernel/io_apic.c ./arch/i386/kernel/io_apic.c
--- ../linux-2.6.12-RT-51-23/arch/i386/kernel/io_apic.c 2005-07-09 23:49:21.0 +0200
+++ ./arch/i386/kernel/io_apic.c 2005-07-12 01:08:34.0 +0200
@@ -31,6 +31,7 @@
#include linux/mc146818rtc.h
#include linux/compiler.h
#include linux/acpi.h
+#include linux/bootmem.h
#include linux/sysdev.h
#include asm/io.h
@@ -55,11 +56,6 @@
int sis_apic_bug = -1;
/*
- * # of IRQ routing registers
- */
-int nr_ioapic_registers[MAX_IO_APICS];
-
-/*
* Rough estimation of how many shared IRQs there are, can
* be changed anytime.
*/
@@ -132,88 +128,74 @@
# define IOAPIC_CACHE
#endif
-#ifdef IOAPIC_CACHE
-# define MAX_IOAPIC_CACHE 512
-/*
- * Cache register values:
- */
-static struct {
- unsigned int reg;
- unsigned int val[MAX_IOAPIC_CACHE];
-} io_apic_cache[MAX_IO_APICS]
- cacheline_aligned_in_smp;
+
+struct ioapic_data_struct {
+ struct sys_device dev;
+ int nr_registers; // # of IRQ routing registers
+ volatile unsigned int *base;
+ struct IO_APIC_route_entry *entry;
+#ifdef IOAPIC_CACHE
+ unsigned int reg_set;
+ u32 cached_val[0];
#endif
+};
-volatile unsigned int *io_apic_base[MAX_IO_APICS];
+static struct ioapic_data_struct *ioapic_data[MAX_IO_APICS];
-static inline unsigned int __raw_io_apic_read(unsigned int apic, unsigned int reg)
+
+static inline unsigned int __raw_io_apic_read(struct ioapic_data_struct *ioapic, unsigned int reg)
{
- volatile unsigned int *io_apic;
-#ifdef IOAPIC_CACHE
- io_apic_cache[apic].reg = reg;
-#endif
- io_apic = io_apic_base[apic];
- io_apic[0] = reg;
- return io_apic[4];
+# ifdef IOAPIC_CACHE
+ ioapic-reg_set = reg;
+# endif
+ ioapic-base[0] = reg;
+ return ioapic-base[4];
}
-unsigned int raw_io_apic_read(unsigned int apic, unsigned int reg)
+
+# ifdef IOAPIC_CACHE
+static void __init ioapic_cache_init(struct ioapic_data_struct *ioapic)
{
- unsigned int val = __raw_io_apic_read(apic, reg);
+ int reg;
+ for (reg = 0; reg (ioapic-nr_registers + 10); reg++)
+ ioapic-cached_val[reg] = __raw_io_apic_read(ioapic, reg);
+}
+# endif
-#ifdef IOAPIC_CACHE
- io_apic_cache[apic].val[reg] = val;
-#endif
+
+static unsigned int raw_io_apic_read(struct ioapic_data_struct *ioapic, unsigned int reg)
+{
+ unsigned int val = __raw_io_apic_read(ioapic, reg);
+
+# ifdef IOAPIC_CACHE
+ ioapic-cached_val[reg] = val;
+# endif
return val;
}
-unsigned int io_apic_read(unsigned int apic, unsigned int reg)
+static unsigned int io_apic_read(struct ioapic_data_struct *ioapic, unsigned int reg)
{
-#ifdef IOAPIC_CACHE
- if (unlikely(reg = MAX_IOAPIC_CACHE)) {
- static int once = 1;
-
- if (once) {
- once = 0;
- printk(WARNING: ioapic register cache overflow: %d.\n,
-reg);
- dump_stack();
- }
- return __raw_io_apic_read(apic, reg);
- }
- if (io_apic_cache[apic].val[reg] !sis_apic_bug) {
- io_apic_cache[apic].reg = -1;
- return io_apic_cache[apic].val[reg];
+# ifdef IOAPIC_CACHE
+ if (likely(!sis_apic_bug)) {
+ ioapic-reg_set = -1;
+ return ioapic-cached_val[reg];
}
-#endif
- return raw_io_apic_read(apic, reg);
+# endif
+ return raw_io_apic_read(ioapic, reg);
}
-void io_apic_write(unsigned int apic, unsigned int reg, unsigned int val)
+static void io_apic_write(struct ioapic_data_struct *ioapic, unsigned int reg, unsigned int val)
{
- volatile unsigned int *io_apic;
-#ifdef IOAPIC_CACHE
- if (unlikely(reg = MAX_IOAPIC_CACHE)) {
- static int once = 1;
-
- if (once) {
- once = 0;
- printk(WARNING: ioapic register cache overflow: %d.\n,
-reg);
- dump_stack();
- }
- } else
- io_apic_cache[apic].val[reg] = val;
-#endif
- io_apic = io_apic_base[apic];
-#ifdef IOAPIC_CACHE
- io_apic_cache[apic].reg = reg;
-#endif
- io_apic[0] = reg;
- io_apic[4] = val;
+# ifdef IOAPIC_CACHE
+ ioapic-cached_val[reg] = val;
+ ioapic-reg_set = reg;
+# endif
+ ioapic-base[0] = reg;
+ ioapic-base[4] = val;
}
+
/*
* Some systems need a POST flush or
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Karsten Wiese [EMAIL PROTECTED] wrote: Hi Ingo I've refined io_apic.c a little more: great. I've applied these changes and have released the -28 patch. (note that the last chunk of your patch was malformed, have applied it by hand.) i'm wondering what your thoughts are about IOAPIC_POSTFLUSH - i had to turn it on unconditionally again, to get rid of spurious interrupts and outright interrupt storms (and resulting lockups) on some systems. IOAPIC_POSTFLUSH is now causing much of the IO-APIC related IRQ handling overhead. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 16:02 +0200, Ingo Molnar wrote: * Karsten Wiese [EMAIL PROTECTED] wrote: Hi Ingo I've refined io_apic.c a little more: great. I've applied these changes and have released the -28 patch. (note that the last chunk of your patch was malformed, have applied it by hand.) i'm wondering what your thoughts are about IOAPIC_POSTFLUSH - i had to turn it on unconditionally again, to get rid of spurious interrupts and outright interrupt storms (and resulting lockups) on some systems. IOAPIC_POSTFLUSH is now causing much of the IO-APIC related IRQ handling overhead. I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in nobody cared messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . Daniel - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Daniel Walker [EMAIL PROTECTED] wrote: I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in nobody cared messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 16:28 +0200, Ingo Molnar wrote: * Daniel Walker [EMAIL PROTECTED] wrote: I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in nobody cared messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. I haven't tested it recently . This was on an older version of RT though . I could try it if it's interesting ? Or do you think it's already fixed? Daniel - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* Daniel Walker [EMAIL PROTECTED] wrote: I haven't tested it recently . This was on an older version of RT though . I could try it if it's interesting ? Or do you think it's already fixed? it would be definitely interesting to see how robust the latest IO-APIC code is. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Ingo Molnar wrote: * Daniel Walker [EMAIL PROTECTED] wrote: I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in nobody cared messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. Ingo Is this why I have been able to boot the latest versions without the noapic option (and without noticeable keyboard repeat problems) or has it just been dumb luck? -- kr - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Tue, 2005-07-12 at 16:46 +0200, Ingo Molnar wrote: * Daniel Walker [EMAIL PROTECTED] wrote: I haven't tested it recently . This was on an older version of RT though . I could try it if it's interesting ? Or do you think it's already fixed? it would be definitely interesting to see how robust the latest IO-APIC code is. I'm sure you test that all the time, but I'll try it .. Daniel - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* K.R. Foley [EMAIL PROTECTED] wrote: Ingo Molnar wrote: * Daniel Walker [EMAIL PROTECTED] wrote: I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in nobody cared messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. Ingo Is this why I have been able to boot the latest versions without the noapic option (and without noticeable keyboard repeat problems) or has it just been dumb luck? yes, i think it's related - the IO-APIC code is now more robust than ever, and that's why any known-broken system would be important to re-check. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
Ingo Molnar wrote: * K.R. Foley [EMAIL PROTECTED] wrote: Ingo Molnar wrote: * Daniel Walker [EMAIL PROTECTED] wrote: I observed a situation on a dual xeon where IOAPIC_POSTFLUSH , if on, would actually cause spurious interrupts. It was odd cause it's suppose to stop them .. If there was a lot of interrupt traffic on one IRQ , it would cause interrupt traffic on another IRQ. This would result in nobody cared messages , and the storming IRQ line would get shutdown. This would only happen in PREEMPT_RT . does it happen with the latest kernel too? There were a couple of things broken in the IOAPIC code in various earlier versions. Ingo Is this why I have been able to boot the latest versions without the noapic option (and without noticeable keyboard repeat problems) or has it just been dumb luck? yes, i think it's related - the IO-APIC code is now more robust than ever, and that's why any known-broken system would be important to re-check. Ingo Well I have booted -27 a couple of times and -28 once now without supplying the noapic boot option and I haven't seen any of the keyboard repeat problems that I reported late last week. This was on my dual 2.6 Xeon w/HT. I have never seen this behavior on any of my older systems. Because of the fact that the problem showed up sporadically, I can't say for sure that it is gone. However, so far so good. I will report any changes that I see. -- kr - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* William Weston <[EMAIL PROTECTED]> wrote: > Still looking into this issue on -51-06. Found something really odd: > SCHED_NORMAL tasks will start to inherit the priority value of some > other SCHED_FIFO task. If JACK is started at a given SCHED_FIFO > priority, X and all of its children will inherit the same priority > value after login. Other random processes will inherit this, too -- > sometimes init... > > SCHED_NORMAL tasks suddenly inheriting priority values in the range > normally reserved for SCHED_FIFO could explain at least part of the > meltdown I've been seeing. Any thoughts? is this inheritance perpetual? It is normal for some tasks to be boosted momentarily, but if the condition remains even after jackd has exited, it's clearly an anomaly. (lets call it "RT priority leakage".) Priority leakage on SMP was fixed recently, but there could be other bugs remaining. There's priority-leakage debugging code in the -RT kernel, which is activated if you enable CONFIG_DEBUG_PREEMPT. This debugging code, if triggered, should produce 'BUG: comm/123 leaked RT prio 123 (123)?" type of messages. But ... due to a bug it would not print out anything but would lock up hard if it detects the condition (and if RT_DEADLOCK_DETECT is enabled). I have fixed this reporting bug in the -51-08 kernel. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* William Weston [EMAIL PROTECTED] wrote: Still looking into this issue on -51-06. Found something really odd: SCHED_NORMAL tasks will start to inherit the priority value of some other SCHED_FIFO task. If JACK is started at a given SCHED_FIFO priority, X and all of its children will inherit the same priority value after login. Other random processes will inherit this, too -- sometimes init... SCHED_NORMAL tasks suddenly inheriting priority values in the range normally reserved for SCHED_FIFO could explain at least part of the meltdown I've been seeing. Any thoughts? is this inheritance perpetual? It is normal for some tasks to be boosted momentarily, but if the condition remains even after jackd has exited, it's clearly an anomaly. (lets call it RT priority leakage.) Priority leakage on SMP was fixed recently, but there could be other bugs remaining. There's priority-leakage debugging code in the -RT kernel, which is activated if you enable CONFIG_DEBUG_PREEMPT. This debugging code, if triggered, should produce 'BUG: comm/123 leaked RT prio 123 (123)? type of messages. But ... due to a bug it would not print out anything but would lock up hard if it detects the condition (and if RT_DEADLOCK_DETECT is enabled). I have fixed this reporting bug in the -51-08 kernel. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Sun, 3 Jul 2005, Ingo Molnar wrote: > ok, found a bug that could explain the situation: mutex sleeps+wakeups > were incorrectly credited as 'interactive sleep' periods, causing the dd > processes to be boosted incorrectly. The dd processes created a workload > in which they blocked each other in such a pattern that they got boosted > periodically, starving pretty much every other task. > > the fix is significant and affects alot of workloads, and should further > improve interactivity in noticeable ways. I'm not 100% sure it solves > all the starvation problems (e.g. how could normal-prio dd tasks starve > the SCHED_FIFO irq threads that drove SysRq?), but the results so far > look promising. > > i've uploaded the -50-45 patch, can you under this kernel trigger a > 'meltdown' on your SMT box? Still looking into this issue on -51-06. Found something really odd: SCHED_NORMAL tasks will start to inherit the priority value of some other SCHED_FIFO task. If JACK is started at a given SCHED_FIFO priority, X and all of its children will inherit the same priority value after login. Other random processes will inherit this, too -- sometimes init... SCHED_NORMAL tasks suddenly inheriting priority values in the range normally reserved for SCHED_FIFO could explain at least part of the meltdown I've been seeing. Any thoughts? Cheers, --ww - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Sun, 3 Jul 2005, Ingo Molnar wrote: ok, found a bug that could explain the situation: mutex sleeps+wakeups were incorrectly credited as 'interactive sleep' periods, causing the dd processes to be boosted incorrectly. The dd processes created a workload in which they blocked each other in such a pattern that they got boosted periodically, starving pretty much every other task. the fix is significant and affects alot of workloads, and should further improve interactivity in noticeable ways. I'm not 100% sure it solves all the starvation problems (e.g. how could normal-prio dd tasks starve the SCHED_FIFO irq threads that drove SysRq?), but the results so far look promising. i've uploaded the -50-45 patch, can you under this kernel trigger a 'meltdown' on your SMT box? Still looking into this issue on -51-06. Found something really odd: SCHED_NORMAL tasks will start to inherit the priority value of some other SCHED_FIFO task. If JACK is started at a given SCHED_FIFO priority, X and all of its children will inherit the same priority value after login. Other random processes will inherit this, too -- sometimes init... SCHED_NORMAL tasks suddenly inheriting priority values in the range normally reserved for SCHED_FIFO could explain at least part of the meltdown I've been seeing. Any thoughts? Cheers, --ww - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* William Weston <[EMAIL PROTECTED]> wrote: > > Which video driver is X using? What nice value is the X server running > > at? > > Hardware is Intel 82865G (integrated) with DRM i915 1.1.0 20040405 and > xorg-3.8.2 i810 driver, running at nice 0, priority 15. Should I bump > the priority up? To realtime? i'd first suggest to try the latest kernel, before changing your X config - i think the bug might have been fixed meanwhile. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Fri, 1 Jul 2005, Lee Revell wrote: > On Fri, 2005-07-01 at 18:46 -0700, William Weston wrote: > > FWIW, I'm still seeing the SMT scheduling? meltdown issues with > > -50-42. > > Running two instances of 'dd if=/dev/zero of=/dev/null bs=65536' > > instead > > of 'burnP6' results in the same behavior. Here's a quick recap: > > > > - Start (or login to ) X. > > - Start an X app that constantly updates the screen, like wmcube, or > > vlc. > > Which video driver is X using? What nice value is the X server running > at? Hardware is Intel 82865G (integrated) with DRM i915 1.1.0 20040405 and xorg-3.8.2 i810 driver, running at nice 0, priority 15. Should I bump the priority up? To realtime? > Does adding: > > Option "NoAccel" > > to the Device section of your X config file make any difference? Disabling the dri and drm modules didn't help. I'll turn on NoAccel when I'm back in the office on Tuesday. > (on most systems X is the only thing besides the kernel that can access > hardware directly, which can cause problems) So would running X through the framebuffer device be the way to go for stability under realtime? It's been a couple years since I've used fb. --ww - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
On Fri, 1 Jul 2005, Lee Revell wrote: On Fri, 2005-07-01 at 18:46 -0700, William Weston wrote: FWIW, I'm still seeing the SMT scheduling? meltdown issues with -50-42. Running two instances of 'dd if=/dev/zero of=/dev/null bs=65536' instead of 'burnP6' results in the same behavior. Here's a quick recap: - Start (or login to ) X. - Start an X app that constantly updates the screen, like wmcube, or vlc. Which video driver is X using? What nice value is the X server running at? Hardware is Intel 82865G (integrated) with DRM i915 1.1.0 20040405 and xorg-3.8.2 i810 driver, running at nice 0, priority 15. Should I bump the priority up? To realtime? Does adding: Option NoAccel to the Device section of your X config file make any difference? Disabling the dri and drm modules didn't help. I'll turn on NoAccel when I'm back in the office on Tuesday. (on most systems X is the only thing besides the kernel that can access hardware directly, which can cause problems) So would running X through the framebuffer device be the way to go for stability under realtime? It's been a couple years since I've used fb. --ww - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24
* William Weston [EMAIL PROTECTED] wrote: Which video driver is X using? What nice value is the X server running at? Hardware is Intel 82865G (integrated) with DRM i915 1.1.0 20040405 and xorg-3.8.2 i810 driver, running at nice 0, priority 15. Should I bump the priority up? To realtime? i'd first suggest to try the latest kernel, before changing your X config - i think the bug might have been fixed meanwhile. Ingo - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
