Re: The HPET issue on Linux
On Wed, Jan 06, 2010 at 04:42:30PM -0600, Anthony Liguori wrote: > On 01/06/2010 02:37 PM, Gleb Natapov wrote: > >We have exactly that hook in apic already and that's how RTC determines > >that interrupt was coalesced. > > AFAICT, apic_irq_delivered is only reset explicitly by the RTC when > the line is lowered. It's not currently lowered based on EOI. > Correct. We can expose ACK notifiers to userspace (and if we want to move assigned devices into userspace we have to), but I'd rather avoid it. > How can this mechanism be used with the HPET when operating in edge > triggered mode? > If interrupt is coalesced increment counter and double HPET timer frequency. When counter is zeroed return HPET timer to normal frequency. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On Thursday 07 January 2010 02:36:26 Beth Kon wrote: > Dor Laor wrote: > > On 01/06/2010 12:09 PM, Gleb Natapov wrote: > >> On Wed, Jan 06, 2010 at 05:48:52PM +0800, Sheng Yang wrote: > >>> Hi Beth > >>> > >>> I still found the emulated HPET would result in some boot failure. For > >>> example, on my 2.6.30, with HPET enabled, the kernel would fail > >>> check_timer(), > >>> especially in timer_irq_works(). > >>> > >>> The testing of timer_irq_works() is let 10 ticks pass(using > >>> mdelay()), and > >>> want to confirm the clock source with at least 5 ticks advanced in > >>> jiffies. > >>> I've checked that, on my machine, it would mostly get only 4 ticks > >>> when HPET > >>> enabled, then fail the test. On the other hand, if I using PIT, it > >>> would get > >>> more than 10 ticks(maybe understandable if some complementary ticks > >>> there). Of > >>> course, extend the ticks count/mdelay() time can work. > >>> > >>> I think it's a major issue of HPET. And it maybe just due to a too long > >>> userspace path for interrupt injection... If it's true, I think it's > >>> not easy > >>> to deal with it. > >> > >> PIT tick are reinjected automatically, HPET should probably do the same > >> although it may just create another set of problems. > > > > Older Linux do automatic adjustment for lost ticks so automatic > > reinjection causes time to run too fast. This is why we added the > > -no-kvm-pit-reinject flag... > > > > It took lots of time to pit/rtc to stabilize, in order of seriously > > consider the hpet emulation, lots of testing should be done. > > I will try to look into this. Since HPET is edge-triggered, looks like > this problem is of a different nature than PIT. Is this a solid failure > or intermittent? > At least for v2.6.30 in my box, it always fails... Of course, I believe the chance of successful injecting enough interrupt depends on the many factors. So I think out target can be: not far behind what PIT can do... -- regards Yang, Sheng -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On 01/06/2010 02:37 PM, Gleb Natapov wrote: We have exactly that hook in apic already and that's how RTC determines that interrupt was coalesced. AFAICT, apic_irq_delivered is only reset explicitly by the RTC when the line is lowered. It's not currently lowered based on EOI. How can this mechanism be used with the HPET when operating in edge triggered mode? Regards, Anthony Liguori -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On Wed, Jan 06, 2010 at 01:51:54PM -0600, Anthony Liguori wrote: > On 01/06/2010 01:44 PM, Gleb Natapov wrote: > >On Wed, Jan 06, 2010 at 01:23:28PM -0600, Anthony Liguori wrote: > >>On 01/06/2010 01:20 PM, Beth Kon wrote: > >>>Beth Kon wrote: > I will try to look into this. Since HPET is edge-triggered, > looks like this problem is of a different nature than PIT. Is > this a solid failure or intermittent? > >>>Anthony just explained that on x86, even edge-triggered interrupts > >>>are queued in the apic and an eoi will occur, so this is not > >>>different than the PIT. > >>Not quite queued in the sense that multiple events will be delivered > >>in order, but I think the point is that you can still detect whether > >>delivery succeeded by counting APIC EOIs. > >> > >>The trouble is that historically we've struggled with doing this in > >>userspace. Maybe it's time to revisit. > >> > >We reinject PIT interrupts from kernel and RTC interrupts from > >userspace. > > Because we can determine that we've missed an RTC interrupt in > userspace. We cannot determine this with the PIT without adding a > hook into the userspace apic that lets us know whether an injection > failed or not. > We have exactly that hook in apic already and that's how RTC determines that interrupt was coalesced. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On 01/06/2010 01:44 PM, Gleb Natapov wrote: On Wed, Jan 06, 2010 at 01:23:28PM -0600, Anthony Liguori wrote: On 01/06/2010 01:20 PM, Beth Kon wrote: Beth Kon wrote: I will try to look into this. Since HPET is edge-triggered, looks like this problem is of a different nature than PIT. Is this a solid failure or intermittent? Anthony just explained that on x86, even edge-triggered interrupts are queued in the apic and an eoi will occur, so this is not different than the PIT. Not quite queued in the sense that multiple events will be delivered in order, but I think the point is that you can still detect whether delivery succeeded by counting APIC EOIs. The trouble is that historically we've struggled with doing this in userspace. Maybe it's time to revisit. We reinject PIT interrupts from kernel and RTC interrupts from userspace. Because we can determine that we've missed an RTC interrupt in userspace. We cannot determine this with the PIT without adding a hook into the userspace apic that lets us know whether an injection failed or not. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On Wed, Jan 06, 2010 at 01:23:28PM -0600, Anthony Liguori wrote: > On 01/06/2010 01:20 PM, Beth Kon wrote: > >Beth Kon wrote: > >>I will try to look into this. Since HPET is edge-triggered, > >>looks like this problem is of a different nature than PIT. Is > >>this a solid failure or intermittent? > >Anthony just explained that on x86, even edge-triggered interrupts > >are queued in the apic and an eoi will occur, so this is not > >different than the PIT. > > Not quite queued in the sense that multiple events will be delivered > in order, but I think the point is that you can still detect whether > delivery succeeded by counting APIC EOIs. > > The trouble is that historically we've struggled with doing this in > userspace. Maybe it's time to revisit. > We reinject PIT interrupts from kernel and RTC interrupts from userspace. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On 01/06/2010 01:20 PM, Beth Kon wrote: Beth Kon wrote: I will try to look into this. Since HPET is edge-triggered, looks like this problem is of a different nature than PIT. Is this a solid failure or intermittent? Anthony just explained that on x86, even edge-triggered interrupts are queued in the apic and an eoi will occur, so this is not different than the PIT. Not quite queued in the sense that multiple events will be delivered in order, but I think the point is that you can still detect whether delivery succeeded by counting APIC EOIs. The trouble is that historically we've struggled with doing this in userspace. Maybe it's time to revisit. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
Beth Kon wrote: Dor Laor wrote: On 01/06/2010 12:09 PM, Gleb Natapov wrote: On Wed, Jan 06, 2010 at 05:48:52PM +0800, Sheng Yang wrote: Hi Beth I still found the emulated HPET would result in some boot failure. For example, on my 2.6.30, with HPET enabled, the kernel would fail check_timer(), especially in timer_irq_works(). The testing of timer_irq_works() is let 10 ticks pass(using mdelay()), and want to confirm the clock source with at least 5 ticks advanced in jiffies. I've checked that, on my machine, it would mostly get only 4 ticks when HPET enabled, then fail the test. On the other hand, if I using PIT, it would get more than 10 ticks(maybe understandable if some complementary ticks there). Of course, extend the ticks count/mdelay() time can work. I think it's a major issue of HPET. And it maybe just due to a too long userspace path for interrupt injection... If it's true, I think it's not easy to deal with it. PIT tick are reinjected automatically, HPET should probably do the same although it may just create another set of problems. Older Linux do automatic adjustment for lost ticks so automatic reinjection causes time to run too fast. This is why we added the -no-kvm-pit-reinject flag... It took lots of time to pit/rtc to stabilize, in order of seriously consider the hpet emulation, lots of testing should be done. I will try to look into this. Since HPET is edge-triggered, looks like this problem is of a different nature than PIT. Is this a solid failure or intermittent? Anthony just explained that on x86, even edge-triggered interrupts are queued in the apic and an eoi will occur, so this is not different than the PIT. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- Regards, Beth Kon -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
Dor Laor wrote: On 01/06/2010 12:09 PM, Gleb Natapov wrote: On Wed, Jan 06, 2010 at 05:48:52PM +0800, Sheng Yang wrote: Hi Beth I still found the emulated HPET would result in some boot failure. For example, on my 2.6.30, with HPET enabled, the kernel would fail check_timer(), especially in timer_irq_works(). The testing of timer_irq_works() is let 10 ticks pass(using mdelay()), and want to confirm the clock source with at least 5 ticks advanced in jiffies. I've checked that, on my machine, it would mostly get only 4 ticks when HPET enabled, then fail the test. On the other hand, if I using PIT, it would get more than 10 ticks(maybe understandable if some complementary ticks there). Of course, extend the ticks count/mdelay() time can work. I think it's a major issue of HPET. And it maybe just due to a too long userspace path for interrupt injection... If it's true, I think it's not easy to deal with it. PIT tick are reinjected automatically, HPET should probably do the same although it may just create another set of problems. Older Linux do automatic adjustment for lost ticks so automatic reinjection causes time to run too fast. This is why we added the -no-kvm-pit-reinject flag... It took lots of time to pit/rtc to stabilize, in order of seriously consider the hpet emulation, lots of testing should be done. I will try to look into this. Since HPET is edge-triggered, looks like this problem is of a different nature than PIT. Is this a solid failure or intermittent? -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- Regards, Beth Kon -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On 01/06/2010 12:09 PM, Gleb Natapov wrote: On Wed, Jan 06, 2010 at 05:48:52PM +0800, Sheng Yang wrote: Hi Beth I still found the emulated HPET would result in some boot failure. For example, on my 2.6.30, with HPET enabled, the kernel would fail check_timer(), especially in timer_irq_works(). The testing of timer_irq_works() is let 10 ticks pass(using mdelay()), and want to confirm the clock source with at least 5 ticks advanced in jiffies. I've checked that, on my machine, it would mostly get only 4 ticks when HPET enabled, then fail the test. On the other hand, if I using PIT, it would get more than 10 ticks(maybe understandable if some complementary ticks there). Of course, extend the ticks count/mdelay() time can work. I think it's a major issue of HPET. And it maybe just due to a too long userspace path for interrupt injection... If it's true, I think it's not easy to deal with it. PIT tick are reinjected automatically, HPET should probably do the same although it may just create another set of problems. Older Linux do automatic adjustment for lost ticks so automatic reinjection causes time to run too fast. This is why we added the -no-kvm-pit-reinject flag... It took lots of time to pit/rtc to stabilize, in order of seriously consider the hpet emulation, lots of testing should be done. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The HPET issue on Linux
On Wed, Jan 06, 2010 at 05:48:52PM +0800, Sheng Yang wrote: > Hi Beth > > I still found the emulated HPET would result in some boot failure. For > example, on my 2.6.30, with HPET enabled, the kernel would fail > check_timer(), > especially in timer_irq_works(). > > The testing of timer_irq_works() is let 10 ticks pass(using mdelay()), and > want to confirm the clock source with at least 5 ticks advanced in jiffies. > I've checked that, on my machine, it would mostly get only 4 ticks when HPET > enabled, then fail the test. On the other hand, if I using PIT, it would get > more than 10 ticks(maybe understandable if some complementary ticks there). > Of > course, extend the ticks count/mdelay() time can work. > > I think it's a major issue of HPET. And it maybe just due to a too long > userspace path for interrupt injection... If it's true, I think it's not easy > to deal with it. > PIT tick are reinjected automatically, HPET should probably do the same although it may just create another set of problems. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
The HPET issue on Linux
Hi Beth I still found the emulated HPET would result in some boot failure. For example, on my 2.6.30, with HPET enabled, the kernel would fail check_timer(), especially in timer_irq_works(). The testing of timer_irq_works() is let 10 ticks pass(using mdelay()), and want to confirm the clock source with at least 5 ticks advanced in jiffies. I've checked that, on my machine, it would mostly get only 4 ticks when HPET enabled, then fail the test. On the other hand, if I using PIT, it would get more than 10 ticks(maybe understandable if some complementary ticks there). Of course, extend the ticks count/mdelay() time can work. I think it's a major issue of HPET. And it maybe just due to a too long userspace path for interrupt injection... If it's true, I think it's not easy to deal with it. -- regards Yang, Sheng -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
