Good evening, I am experiencing a weird issue with the O3CPU, X86 and the interrupt handling. I am running in FS mode and one simulation just experienced a weird hang. The simulated machine is doing an spinlock over a value that an interrupt handler writes.
After some debug I found that when the APIC sends two interruptions to the cpu in a very short time window, the first interruption is completely ignored. It can not even complete a commit of the first instruction in the service routine before all its values get replaced by the next interrupt. After this interrupt completes, the execution goes back to the application code and do not execute the code for the first interrupt. The problem is that the Lapic has the vector of the first interruption in the ISR register as it gets restored after the second interruption completes. Therefore, it thinks that the cpu is currently processing that interruption, though the cpu went back to execute application code and will never clear this ISR register. The Lapic uses this ISR value to filter incoming interruptions and in several cases, it does not forward those to the cpu, leading to unattended interruptions and hangs. I have seen this behavior in the kernels' native_flush_tlb_others function when a page fault happens. The core in charge of executing it, sends an interrupt to all the other cores in the system and it does a loop checking that every cores receive the interruption. When each core receives the interruption, they just execute the associated handler and perform a write to a variable, notifying the sender that the interruption was processed and the tlb was flushed. The problem is that one of the cores is ignoring this interruption, which has a vector value of 0xf0. I found that this core lapic has a value of 0xf1 in the ISR, filtering every lower vector. s This 0xf1 vector value was set by an interruption that never got to execute because of the problem explained before, hence the interruption carrying the 0xf0 vector value will never be executed and the native_flush_tlb_others function will not complete. I just took a trace of the moment when the 0xf1 interruption gets dropped. The flags used where Exec, Commit, Faults: system.cpu00.interrupts: Interrupt 0xf1 sent to core. 7175754213000: External Interrupt: RIP 0xffffffff8027a0a0: vector 0xf1: #INTR 7175754213000: system.cpu00.interrupts: NEW IRR 0 NEW ISR f1. Now the interrupt 0xf3 gets to execute and drops all the first interruption. 7175754217000: system.cpu00.interrupts: Got Trigger Interrupt message with vector 0xf3. 7175754217000: system.cpu00.interrupts: Interrupt is an Fixed. 7175754220500: system.cpu00.commit: Interrupt detected. 7175754220500: system.cpu00.interrupts: Interrupt 0xf3 sent to core. 7175754220500: External Interrupt: RIP 0xffffffff8027a0a0: vector 0xf3: #INTR 7175754220500: system.cpu00.interrupts: NEW IRR 0 NEW ISR f3. It can be seen how for both interrupts the RIP is the same. The first committed instruction after all this sequence of events is 7175754228500: system.cpu00 T0 : @handle_mm_fault+992.32768 : Microcode_ROM : slli t4, t1, 0x4 : IntAlu : D=0x0000000000000f30 which indeed is from the 0xf3 interrupt. The cpu executes all the handler and then writes to the APIC EOI register 7175754418500: system.cpu00.interrupts: Writing Local APIC register 5 at offset 0xb0 as 0. 7175754418500: system.cpu00.interrupts: WRITING TO EOI NEW ISRV IS 0xf1 Here the APIC believes it is servicing the 0xf1 interrupt. However the cpu goes back to the code it was executing right before the 0xf1 interrupt, and never services it. I was wondering if someone has found this issue before. Thanks a lot for your time. Best regards, --------------------------------------- Emilio Castillo _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
