Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using?
Thanks, Alex -----Original Message----- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev <gem5-dev@gem5.org > wrote: > That's not actually extending the TSS limit, that's what it works out > to be with the granularity bit set. The AM and WP bits were set to the > wrong thing according to the comments next to them I'm pretty sure. If > we wanted the other behavior, we might be able to change them back and have > it work. > The _BASE registers hold the base of segments as they're specified by > the ISA. The _EFF_BASE registers hold the base that will actually be > used in address calculations based on the mode of the CPU. For > instance, if you're in 64 bit mode, the _BASE of DS might still be > 0xFFF from when you were in another mode. The _EFF_BASE would be 0 > though, since the DS base is ignored in that case. _EFF_BASE may not > be used by the KVM CPU, but it should be set up anyway in case we switch back > to a regular CPU. > > Gabe > > On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev < > gem5-dev@gem5.org> wrote: > > > Thank you for all the clarifications. I see that for SE to work on > > vmx > the > > TSS limit had to be extended, am and wp bits in CR0 had to be reset > > and *_EFF_BASE registers had be set in addition to *_BASE registers > > for TR > TSG > > IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR > > register (which gets passed to kvm) are the same if with or without > > *_EFF_BASE registers set. > > > > Thank you, > > Alex > > > > -----Original Message----- > > From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe > Black > > via gem5-dev > > Sent: Wednesday, December 10, 2014 1:21 AM > > To: gem5 Developer List > > Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) > > > > Ok, I got SE working too. I'll clean up my patch and send that out > > in a bit. > > > > Gabe > > > > On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black <gabebl...@google.com> wrote: > > > > > I figured out what the other problem was, so here's the review. > > > > > > http://reviews.gem5.org/r/2557/ > > > > > > Gabe > > > > > > On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black <gabebl...@google.com> > wrote: > > > > > >> It was attached in my sent mail. Maybe it's being blocked by > something? > > >> I'm hunting down another problem so I don't want to move my tree > > >> around too much, but once that's done I'll post it as a review. > > >> > > >> Gabe > > >> > > >> On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev < > > >> gem5-dev@gem5.org> wrote: > > >> > > >>> I haven't received any attachment to your email. So I don't have > > >>> your patch. > > >>> > > >>> Alex > > >>> > > >>> -----Original Message----- > > >>> From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of > > >>> Gabe Black via gem5-dev > > >>> Sent: Tuesday, December 09, 2014 6:42 PM > > >>> To: gem5 Developer List > > >>> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) > > >>> > > >>> And... it turns out the KVM change wasn't necessary. If you're > > >>> working from my patch, get rid of where the segment limit is > > >>> divided > > by PageBytes. > > >>> That was only necessary because I wasn't adding 0xFFF to the > > >>> limit when the granularity bit was set. > > >>> > > >>> Gabe > > >>> > > >>> On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black > > >>> <gabebl...@google.com> > > wrote: > > >>> > > >>> > Oh, also segment limits weren't being computed correctly in > > >>> > the installSegDesc function, although I don't think that was > > >>> > from the KVM stuff. Once it was fixed it required adjusting > > >>> > the KVM stuff a little, though. > > >>> > > > >>> > Gabe > > >>> > > > >>> > On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black > > >>> > <gabebl...@google.com> > > >>> wrote: > > >>> > > > >>> >> Here is my patch so far. There were a few things wrong, > > >>> >> although I didn't really keep notes. The limits were mixed > > >>> >> up, the long mode bit was set on all descriptors when it's > > >>> >> only valid for the code segment, privilege level > > >>> >> 0 is the OS and 3 is for applications and not the other way > > >>> >> around, and I think the type was being set wrong for one of > > >>> >> the > > segments. > > >>> >> Also, the syscall and sysenter registers (star and friends) > > >>> >> require the segments in the GDT to be in a particular order > > >>> >> which I don't > > >>> think they were. > > >>> >> > > >>> >> Gabe > > >>> >> > > >>> >> On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev > > >>> >> < gem5-dev@gem5.org> wrote: > > >>> >> > > >>> >>> So, I am doing this on an AMD system and I have SE working > > >>> >>> and am able to get FS entering into virtualized mode. > > >>> >>> However, in FS I get an early exception while the kernel is > > >>> >>> booting. This seems a bit different from what Nilay and Adrian > > >>> >>> observed for FS. > > >>> >>> Could you please share the diffs that got FS working? > > >>> >>> > > >>> >>> Thanks, > > >>> >>> Alex > > >>> >>> > > >>> >>> -----Original Message----- > > >>> >>> From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf > > >>> >>> Of Gabe Black via gem5-dev > > >>> >>> Sent: Tuesday, December 09, 2014 6:07 PM > > >>> >>> To: gem5 Developer List > > >>> >>> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs > > >>> >>> Intel) > > >>> >>> > > >>> >>> Oh, I see you have FS working again and not SE. NM, I'll > > >>> >>> keep > > >>> looking. > > >>> >>> > > >>> >>> Gabe > > >>> >>> > > >>> >>> On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black > > >>> >>> <gabebl...@google.com> > > >>> wrote: > > >>> >>> > > >>> >>> > I have FS working again which is good, but I'm still > > >>> >>> > having problems with SE. If you could let me know what you > > >>> >>> > did to get things going that would be very helpful. > > >>> >>> > > > >>> >>> > Gabe > > >>> >>> > > > >>> >>> > On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via > > >>> >>> > gem5-dev < gem5-dev@gem5.org> wrote: > > >>> >>> > > > >>> >>> >> Hi Adrian, > > >>> >>> >> > > >>> >>> >> Sorry for missing your first email. I do see the > > >>> >>> >> interchanged segment limits for full system mode, though > > >>> >>> >> I get a different behaviour on my system. The simulation > > >>> >>> >> seems to hang in the > > >>> following manner: > > >>> >>> >> > > >>> >>> >> Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC00000. > > >>> >>> >> Setting APIC routing to flat > > >>> >>> >> Processors: 1 > > >>> >>> >> PANIC: early exception rip ffffffff807909a9 error 9 cr2 > > >>> >>> >> ffffffffff5fd020 > > >>> >>> >> > > >>> >>> >> Can please provide a patch with all the modifications > > >>> >>> >> that fixed the issue on your system? > > >>> >>> >> > > >>> >>> >> Thank you, > > >>> >>> >> Alex > > >>> >>> >> ________________________________________ > > >>> >>> >> From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of > > >>> >>> >> Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] > > >>> >>> >> Sent: Tuesday, December 09, 2014 2:09 AM > > >>> >>> >> To: gem5 Developer List > > >>> >>> >> Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs > > >>> >>> >> Intel) > > >>> >>> >> > > >>> >>> >> You are right Nilay. I sent an email last week but nobody > > >>> >>> >> has > > >>> replied. > > >>> >>> >> > > >>> >>> >> It seems that descriptors (cdDesc, dsDesc and tssDesc) > > >>> >>> >> located in src/arch/x86/system.cc file are not > > >>> >>> >> well-initialized and as a consequence kvm does not work > > >>> >>> >> when > > running in full-system mode. > > >>> >>> >> > > >>> >>> >> Segment limits values (limitHigh and limitLow) are > > >>> >>> >> interchanged and several segment descriptor values are > > >>> >>> >> wrong too. If these values are corrected kvm works again as > > >>> >>> >> before. > > >>> >>> >> > > >>> >>> >> Adrian > > >>> >>> >> > > >>> >>> >> El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via > > >>> >>> >> gem5-dev > > >>> >>> escribió: > > >>> >>> >> > I also faced problem in getting KVM CPU to run in FS mode. > > >>> >>> >> > I figured > > >>> >>> >> that > > >>> >>> >> > the following changeset causes problems: > > >>> >>> >> > > > >>> >>> >> > author Alexandru Dutu <alexandru.d...@amd.com> > > >>> >>> >> > Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) > > >>> >>> >> > changeset 10554 fe2e2f06a7c8 > > >>> >>> >> > > > >>> >>> >> > I saw the hardware reason 0x80000021, but did not try > > >>> >>> >> > to figure what was going on wrong. > > >>> >>> >> > > > >>> >>> >> > -- > > >>> >>> >> > Nilay > > >>> >>> >> > > > >>> >>> >> > On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: > > >>> >>> >> > > > >>> >>> >> > > I'm pretty sure entering 64 bit mode is the same > > >>> >>> >> > > between AMD and Intel CPUs. I vaguely remember there > > >>> >>> >> > > being some subtle page table difference though, and > > >>> >>> >> > > gem5 is building the page tables in SE mode instead of the > > >>> >>> >> > > kernel. > > >>> >>> >> > > > > >>> >>> >> > > Gabe > > >>> >>> >> > > > > >>> >>> >> > > On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via > > >>> >>> >> > > gem5-dev < gem5-dev@gem5.org> wrote: > > >>> >>> >> > > > > >>> >>> >> > >> Hi Mike, > > >>> >>> >> > >> > > >>> >>> >> > >> trace-cmd is a very handy tool to get an overview of > > >>> >>> >> > >> what the kvm > > >>> >>> >> kernel > > >>> >>> >> > >> module is doing before going into gdb. In extreme > > >>> >>> >> > >> cases ftrace can be useful as well. > > >>> >>> >> > >> What is the error that you are seeing? Is it still > > >>> >>> >> > >> failing to enter virtualized mode? > > >>> >>> >> > >> > > >>> >>> >> > >> If that is the case and the hardware reason is > > >>> >>> >> > >> 0x80000021, that > > >>> >>> >> seems to > > >>> >>> >> > >> be an unrecoverable exception > > >>> >>> >> > >> (drivers/hv/hyperv_vmbus.h in linux > > >>> >>> >> kernel > > >>> >>> >> > >> source code). When running in SE mode, we are trying > > >>> >>> >> > >> to bring the > > >>> >>> >> machine > > >>> >>> >> > >> state to full 64bit mode without going through > > >>> >>> >> > >> legacy > > modes. > > >>> >>> >> > >> It > > >>> >>> >> might be > > >>> >>> >> > >> that Intel machines have a different way of going to > > >>> >>> >> > >> 64bit mode than > > >>> >>> >> AMD > > >>> >>> >> > >> machines (different CR4, different way of enabling > > >>> >>> >> > >> 64bit mode page > > >>> >>> >> tables > > >>> >>> >> > >> etc.). I remember dealing with these issue for AMD > > >>> >>> >> > >> platforms by going through System Programming manual > > >>> >>> >> > >> and making sure > > >>> >>> >> > >> gem5 gets all the > > >>> >>> >> bits > > >>> >>> >> > >> right as there is not much the KVM kernel model will > > >>> >>> >> > >> tell about the > > >>> >>> >> cause > > >>> >>> >> > >> of failure. > > >>> >>> >> > >> > > >>> >>> >> > >> Best regards, > > >>> >>> >> > >> Alex > > >>> >>> >> > >> ________________________________________ > > >>> >>> >> > >> From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf > > >>> >>> >> > >> of Gabe Black > > >>> >>> >> via > > >>> >>> >> > >> gem5-dev [gem5-dev@gem5.org] > > >>> >>> >> > >> Sent: Monday, December 08, 2014 7:08 PM > > >>> >>> >> > >> To: gem5 Developer List > > >>> >>> >> > >> Subject: Re: [gem5-dev] x86 SE kvm functionality > > >>> >>> >> > >> (AMD vs > > >>> >>> >> > >> Intel) > > >>> >>> >> > >> > > >>> >>> >> > >> I'm not an expert either, but I did have problems > > >>> >>> >> > >> running KVM in SE > > >>> >>> >> mode on > > >>> >>> >> > >> an Intel CPU. I didn't look into it that much, but I > > >>> >>> >> > >> think things > > >>> >>> >> failed in > > >>> >>> >> > >> the kernel somewhere. What might be happening is > > >>> >>> >> > >> that the different > > >>> >>> >> vendors > > >>> >>> >> > >> hardware virtualization mechanisms are more or less > > >>> >>> >> > >> picky about > > >>> >>> >> various > > >>> >>> >> > >> things. Something might be set up incorrectly, and > > >>> >>> >> > >> one > > >>> >>> >> implementation gets > > >>> >>> >> > >> more upset about it than the other. I believe there > > >>> >>> >> > >> are tools which > > >>> >>> >> will > > >>> >>> >> > >> help you determine whether your VM state is legal. > > >>> >>> >> > >> Perhaps Andreas > > >>> >>> >> can tell > > >>> >>> >> > >> you more about those? > > >>> >>> >> > >> > > >>> >>> >> > >> Gabe > > >>> >>> >> > >> > > >>> >>> >> > >> On Mon, Dec 8, 2014 at 4:29 PM, mike upton via > > >>> >>> >> > >> gem5-dev < > > >>> >>> >> gem5-dev@gem5.org > > >>> >>> >> > >>> > > >>> >>> >> > >> wrote: > > >>> >>> >> > >> > > >>> >>> >> > >>> I have verified that x86 kvm works fine on AMD > > >>> >>> >> > >>> platforms, but fails > > >>> >>> >> on > > >>> >>> >> > >>> Intel platforms. > > >>> >>> >> > >>> > > >>> >>> >> > >>> Any hints about how to narrow down the cause (other > > >>> >>> >> > >>> than diving > > >>> >>> >> into gdb, > > >>> >>> >> > >>> which I will do). > > >>> >>> >> > >>> > > >>> >>> >> > >>> I am not an expert in KVM or how gem5 hooks up to > libkvm. > > >>> >>> >> > >>> _______________________________________________ > > >>> >>> >> > >>> gem5-dev mailing list gem5-dev@gem5.org > > >>> >>> >> > >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > >>> > > >>> >>> >> > >> _______________________________________________ > > >>> >>> >> > >> gem5-dev mailing list gem5-dev@gem5.org > > >>> >>> >> > >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > >> _______________________________________________ > > >>> >>> >> > >> gem5-dev mailing list gem5-dev@gem5.org > > >>> >>> >> > >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > >> > > >>> >>> >> > > _______________________________________________ > > >>> >>> >> > > gem5-dev mailing list gem5-dev@gem5.org > > >>> >>> >> > > http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > > > > >>> >>> >> > _______________________________________________ > > >>> >>> >> > gem5-dev mailing list > > >>> >>> >> > gem5-dev@gem5.org > > >>> >>> >> > http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > > >>> >>> >> > > >>> >>> >> _______________________________________________ > > >>> >>> >> gem5-dev mailing list > > >>> >>> >> gem5-dev@gem5.org > > >>> >>> >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> _______________________________________________ > > >>> >>> >> gem5-dev mailing list > > >>> >>> >> gem5-dev@gem5.org > > >>> >>> >> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> >> > > >>> >>> > > > >>> >>> > > > >>> >>> _______________________________________________ > > >>> >>> gem5-dev mailing list > > >>> >>> gem5-dev@gem5.org > > >>> >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> _______________________________________________ > > >>> >>> gem5-dev mailing list > > >>> >>> gem5-dev@gem5.org > > >>> >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> >>> > > >>> >> > > >>> >> > > >>> > > > >>> _______________________________________________ > > >>> gem5-dev mailing list > > >>> gem5-dev@gem5.org > > >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> _______________________________________________ > > >>> gem5-dev mailing list > > >>> gem5-dev@gem5.org > > >>> http://m5sim.org/mailman/listinfo/gem5-dev > > >>> > > >> > > >> > > > > > _______________________________________________ > > gem5-dev mailing list > > gem5-dev@gem5.org > > http://m5sim.org/mailman/listinfo/gem5-dev > > _______________________________________________ > > gem5-dev mailing list > > gem5-dev@gem5.org > > http://m5sim.org/mailman/listinfo/gem5-dev > > > _______________________________________________ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev > _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
