Thanks for the clarification, actually WP needs to be reset for CPU switching to work properly. I think it is better to leave alignment check off for SE mode as there is no handler in place for unaligned access exception.
Alex -----Original Message----- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 2:28 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev < gem5-dev@gem5.org> wrote: > Thank you for all the clarifications. I see that for SE to work on vmx > the TSS limit had to be extended, am and wp bits in CR0 had to be > reset and *_EFF_BASE registers had be set in addition to *_BASE > registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that > the contents of TR register (which gets passed to kvm) are the same if > with or without *_EFF_BASE registers set. > > Thank you, > Alex > > -----Original Message----- > From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe > Black via gem5-dev > Sent: Wednesday, December 10, 2014 1:21 AM > To: gem5 Developer List > Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) > > Ok, I got SE working too. I'll clean up my patch and send that out in > a bit. > > Gabe > > On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black <gabebl...@google.com> wrote: > > > I figured out what the other problem was, so here's the review. > > > > http://reviews.gem5.org/r/2557/ > > > > Gabe > > > > On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black <gabebl...@google.com> wrote: > > > >> It was attached in my sent mail. Maybe it's being blocked by something? > >> I'm hunting down another problem so I don't want to move my tree > >> around too much, but once that's done I'll post it as a review. > >> > >> Gabe > >> > >> On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev < > >> gem5-dev@gem5.org> wrote: > >> > >>> I haven't received any attachment to your email. So I don't have > >>> your patch. > >>> > >>> Alex > >>> > >>> -----Original Message----- > >>> From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of > >>> Gabe Black via gem5-dev > >>> Sent: Tuesday, December 09, 2014 6:42 PM > >>> To: gem5 Developer List > >>> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) > >>> > >>> And... it turns out the KVM change wasn't necessary. If you're > >>> working from my patch, get rid of where the segment limit is > >>> divided > by PageBytes. > >>> That was only necessary because I wasn't adding 0xFFF to the limit > >>> when the granularity bit was set. > >>> > >>> Gabe > >>> > >>> On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black <gabebl...@google.com> > wrote: > >>> > >>> > Oh, also segment limits weren't being computed correctly in the > >>> > installSegDesc function, although I don't think that was from > >>> > the KVM stuff. Once it was fixed it required adjusting the KVM > >>> > stuff a little, though. > >>> > > >>> > Gabe > >>> > > >>> > On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black > >>> > <gabebl...@google.com> > >>> wrote: > >>> > > >>> >> Here is my patch so far. There were a few things wrong, > >>> >> although I didn't really keep notes. The limits were mixed up, > >>> >> the long mode bit was set on all descriptors when it's only > >>> >> valid for the code segment, privilege level > >>> >> 0 is the OS and 3 is for applications and not the other way > >>> >> around, and I think the type was being set wrong for one of the > segments. > >>> >> Also, the syscall and sysenter registers (star and friends) > >>> >> require the segments in the GDT to be in a particular order > >>> >> which I don't > >>> think they were. > >>> >> > >>> >> Gabe > >>> >> > >>> >> On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev < > >>> >> gem5-dev@gem5.org> wrote: > >>> >> > >>> >>> So, I am doing this on an AMD system and I have SE working and > >>> >>> am able to get FS entering into virtualized mode. However, in > >>> >>> FS I get an early exception while the kernel is booting. This > >>> >>> seems a bit different from what Nilay and Adrian observed for FS. > >>> >>> Could you please share the diffs that got FS working? > >>> >>> > >>> >>> Thanks, > >>> >>> Alex > >>> >>> > >>> >>> -----Original Message----- > >>> >>> From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of > >>> >>> Gabe Black via gem5-dev > >>> >>> Sent: Tuesday, December 09, 2014 6:07 PM > >>> >>> To: gem5 Developer List > >>> >>> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs > >>> >>> Intel) > >>> >>> > >>> >>> Oh, I see you have FS working again and not SE. NM, I'll keep > >>> looking. > >>> >>> > >>> >>> Gabe > >>> >>> > >>> >>> On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black > >>> >>> <gabebl...@google.com> > >>> wrote: > >>> >>> > >>> >>> > I have FS working again which is good, but I'm still having > >>> >>> > problems with SE. If you could let me know what you did to > >>> >>> > get things going that would be very helpful. > >>> >>> > > >>> >>> > Gabe > >>> >>> > > >>> >>> > On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via > >>> >>> > gem5-dev < gem5-dev@gem5.org> wrote: > >>> >>> > > >>> >>> >> Hi Adrian, > >>> >>> >> > >>> >>> >> Sorry for missing your first email. I do see the > >>> >>> >> interchanged segment limits for full system mode, though I > >>> >>> >> get a different behaviour on my system. The simulation > >>> >>> >> seems to hang in the > >>> following manner: > >>> >>> >> > >>> >>> >> Processor #0 (Bootup-CPU) > >>> >>> >> I/O APIC #1 at 0xFEC00000. > >>> >>> >> Setting APIC routing to flat > >>> >>> >> Processors: 1 > >>> >>> >> PANIC: early exception rip ffffffff807909a9 error 9 cr2 > >>> >>> >> ffffffffff5fd020 > >>> >>> >> > >>> >>> >> Can please provide a patch with all the modifications that > >>> >>> >> fixed the issue on your system? > >>> >>> >> > >>> >>> >> Thank you, > >>> >>> >> Alex > >>> >>> >> ________________________________________ > >>> >>> >> From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of > >>> >>> >> Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] > >>> >>> >> Sent: Tuesday, December 09, 2014 2:09 AM > >>> >>> >> To: gem5 Developer List > >>> >>> >> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs > >>> >>> >> Intel) > >>> >>> >> > >>> >>> >> You are right Nilay. I sent an email last week but nobody > >>> >>> >> has > >>> replied. > >>> >>> >> > >>> >>> >> It seems that descriptors (cdDesc, dsDesc and tssDesc) > >>> >>> >> located in src/arch/x86/system.cc file are not > >>> >>> >> well-initialized and as a consequence kvm does not work > >>> >>> >> when > running in full-system mode. > >>> >>> >> > >>> >>> >> Segment limits values (limitHigh and limitLow) are > >>> >>> >> interchanged and several segment descriptor values are > >>> >>> >> wrong too. If these values are corrected kvm works again as before. > >>> >>> >> > >>> >>> >> Adrian > >>> >>> >> > >>> >>> >> El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via > >>> >>> >> gem5-dev > >>> >>> escribió: > >>> >>> >> > I also faced problem in getting KVM CPU to run in FS mode. > >>> >>> >> > I figured > >>> >>> >> that > >>> >>> >> > the following changeset causes problems: > >>> >>> >> > > >>> >>> >> > author Alexandru Dutu <alexandru.d...@amd.com> > >>> >>> >> > Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) > >>> >>> >> > changeset 10554 fe2e2f06a7c8 > >>> >>> >> > > >>> >>> >> > I saw the hardware reason 0x80000021, but did not try to > >>> >>> >> > figure what was going on wrong. > >>> >>> >> > > >>> >>> >> > -- > >>> >>> >> > Nilay > >>> >>> >> > > >>> >>> >> > On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: > >>> >>> >> > > >>> >>> >> > > I'm pretty sure entering 64 bit mode is the same > >>> >>> >> > > between AMD and Intel CPUs. I vaguely remember there > >>> >>> >> > > being some subtle page table difference though, and > >>> >>> >> > > gem5 is building the page tables in SE mode instead of the > >>> >>> >> > > kernel. > >>> >>> >> > > > >>> >>> >> > > Gabe > >>> >>> >> > > > >>> >>> >> > > On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via > >>> >>> >> > > gem5-dev < gem5-dev@gem5.org> wrote: > >>> >>> >> > > > >>> >>> >> > >> Hi Mike, > >>> >>> >> > >> > >>> >>> >> > >> trace-cmd is a very handy tool to get an overview of > >>> >>> >> > >> what the kvm > >>> >>> >> kernel > >>> >>> >> > >> module is doing before going into gdb. In extreme > >>> >>> >> > >> cases ftrace can be useful as well. > >>> >>> >> > >> What is the error that you are seeing? Is it still > >>> >>> >> > >> failing to enter virtualized mode? > >>> >>> >> > >> > >>> >>> >> > >> If that is the case and the hardware reason is > >>> >>> >> > >> 0x80000021, that > >>> >>> >> seems to > >>> >>> >> > >> be an unrecoverable exception > >>> >>> >> > >> (drivers/hv/hyperv_vmbus.h in linux > >>> >>> >> kernel > >>> >>> >> > >> source code). When running in SE mode, we are trying > >>> >>> >> > >> to bring the > >>> >>> >> machine > >>> >>> >> > >> state to full 64bit mode without going through legacy > modes. > >>> >>> >> > >> It > >>> >>> >> might be > >>> >>> >> > >> that Intel machines have a different way of going to > >>> >>> >> > >> 64bit mode than > >>> >>> >> AMD > >>> >>> >> > >> machines (different CR4, different way of enabling > >>> >>> >> > >> 64bit mode page > >>> >>> >> tables > >>> >>> >> > >> etc.). I remember dealing with these issue for AMD > >>> >>> >> > >> platforms by going through System Programming manual > >>> >>> >> > >> and making sure > >>> >>> >> > >> gem5 gets all the > >>> >>> >> bits > >>> >>> >> > >> right as there is not much the KVM kernel model will > >>> >>> >> > >> tell about the > >>> >>> >> cause > >>> >>> >> > >> of failure. > >>> >>> >> > >> > >>> >>> >> > >> Best regards, > >>> >>> >> > >> Alex > >>> >>> >> > >> ________________________________________ > >>> >>> >> > >> From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf > >>> >>> >> > >> of Gabe Black > >>> >>> >> via > >>> >>> >> > >> gem5-dev [gem5-dev@gem5.org] > >>> >>> >> > >> Sent: Monday, December 08, 2014 7:08 PM > >>> >>> >> > >> To: gem5 Developer List > >>> >>> >> > >> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD > >>> >>> >> > >> vs > >>> >>> >> > >> Intel) > >>> >>> >> > >> > >>> >>> >> > >> I'm not an expert either, but I did have problems > >>> >>> >> > >> running KVM in SE > >>> >>> >> mode on > >>> >>> >> > >> an Intel CPU. I didn't look into it that much, but I > >>> >>> >> > >> think things > >>> >>> >> failed in > >>> >>> >> > >> the kernel somewhere. What might be happening is that > >>> >>> >> > >> the different > >>> >>> >> vendors > >>> >>> >> > >> hardware virtualization mechanisms are more or less > >>> >>> >> > >> picky about > >>> >>> >> various > >>> >>> >> > >> things. Something might be set up incorrectly, and one > >>> >>> >> implementation gets > >>> >>> >> > >> more upset about it than the other. I believe there > >>> >>> >> > >> are tools which > >>> >>> >> will > >>> >>> >> > >> help you determine whether your VM state is legal. > >>> >>> >> > >> Perhaps Andreas > >>> >>> >> can tell > >>> >>> >> > >> you more about those? > >>> >>> >> > >> > >>> >>> >> > >> Gabe > >>> >>> >> > >> > >>> >>> >> > >> On Mon, Dec 8, 2014 at 4:29 PM, mike upton via > >>> >>> >> > >> gem5-dev < > >>> >>> >> gem5-dev@gem5.org > >>> >>> >> > >>> > >>> >>> >> > >> wrote: > >>> >>> >> > >> > >>> >>> >> > >>> I have verified that x86 kvm works fine on AMD > >>> >>> >> > >>> platforms, but fails > >>> >>> >> on > >>> >>> >> > >>> Intel platforms. > >>> >>> >> > >>> > >>> >>> >> > >>> Any hints about how to narrow down the cause (other > >>> >>> >> > >>> than diving > >>> >>> >> into gdb, > >>> >>> >> > >>> which I will do). > >>> >>> >> > >>> > >>> >>> >> > >>> I am not an expert in KVM or how gem5 hooks up to libkvm. > >>> >>> >> > >>> _______________________________________________ > >>> >>> >> > >>> gem5-dev mailing list gem5-dev@gem5.org > >>> >>> >> > >>> http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> >> > >>> > >>> >>> >> > >> _______________________________________________ > >>> >>> >> > >> gem5-dev mailing list > >>> >>> >> > >> gem5-dev@gem5.org > >>> >>> >> > >> http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> >> > >> _______________________________________________ > >>> >>> >> > >> gem5-dev mailing list > >>> >>> >> > >> gem5-dev@gem5.org > >>> >>> >> > >> http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> >> > >> > >>> >>> >> > > _______________________________________________ > >>> >>> >> > > gem5-dev mailing list > >>> >>> >> > > gem5-dev@gem5.org > >>> >>> >> > > http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> >> > > > >>> >>> >> > _______________________________________________ > >>> >>> >> > gem5-dev mailing list > >>> >>> >> > gem5-dev@gem5.org > >>> >>> >> > http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> >> > >>> >>> >> > >>> >>> >> _______________________________________________ > >>> >>> >> gem5-dev mailing list > >>> >>> >> gem5-dev@gem5.org > >>> >>> >> http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> >> _______________________________________________ > >>> >>> >> gem5-dev mailing list > >>> >>> >> gem5-dev@gem5.org > >>> >>> >> http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> >> > >>> >>> > > >>> >>> > > >>> >>> _______________________________________________ > >>> >>> gem5-dev mailing list > >>> >>> gem5-dev@gem5.org > >>> >>> http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> _______________________________________________ > >>> >>> gem5-dev mailing list > >>> >>> gem5-dev@gem5.org > >>> >>> http://m5sim.org/mailman/listinfo/gem5-dev > >>> >>> > >>> >> > >>> >> > >>> > > >>> _______________________________________________ > >>> gem5-dev mailing list > >>> gem5-dev@gem5.org > >>> http://m5sim.org/mailman/listinfo/gem5-dev > >>> _______________________________________________ > >>> gem5-dev mailing list > >>> gem5-dev@gem5.org > >>> http://m5sim.org/mailman/listinfo/gem5-dev > >>> > >> > >> > > > _______________________________________________ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > _______________________________________________ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
