Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Alex, You are absolutely right (and mercurial is not that great in tracking change sets :-). You need the patch before it: http://reviews.gem5.org/r/2572/ I am hoping to get all these pushed before Christmas. Andreas On 17/12/2014 16:38, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Andreas, I've tried applying this patch on top of revision 8fc6e7a835d1 and I get bunch of rejects. It seems dram_ctrl.cc is a bit different in this patch it has all sorts of extra code to deal with ranks. So I wondering this patch requires another one to apply properly. Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson via gem5-dev Sent: Saturday, December 13, 2014 2:12 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) This patch should hopefully solve the issue with the refresh event: http://reviews.gem5.org/r/2573/ Andreas On 11/12/2014 15:52, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Andreas, Your patches solve the issue, I don't see spurious kvm exits anymore. Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson via gem5-dev Sent: Friday, December 19, 2014 2:43 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Hi Alex, You are absolutely right (and mercurial is not that great in tracking change sets :-). You need the patch before it: http://reviews.gem5.org/r/2572/ I am hoping to get all these pushed before Christmas. Andreas On 17/12/2014 16:38, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Andreas, I've tried applying this patch on top of revision 8fc6e7a835d1 and I get bunch of rejects. It seems dram_ctrl.cc is a bit different in this patch it has all sorts of extra code to deal with ranks. So I wondering this patch requires another one to apply properly. Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson via gem5-dev Sent: Saturday, December 13, 2014 2:12 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) This patch should hopefully solve the issue with the refresh event: http://reviews.gem5.org/r/2573/ Andreas On 11/12/2014 15:52, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Alex, That is great news. Thanks for checking. Now we just have to work out a sensible API for the memory mode changes to avoid putting all this code in the controller. :-) Andreas On 19/12/2014 15:48, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Andreas, Your patches solve the issue, I don't see spurious kvm exits anymore. Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson via gem5-dev Sent: Friday, December 19, 2014 2:43 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Hi Alex, You are absolutely right (and mercurial is not that great in tracking change sets :-). You need the patch before it: http://reviews.gem5.org/r/2572/ I am hoping to get all these pushed before Christmas. Andreas On 17/12/2014 16:38, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Andreas, I've tried applying this patch on top of revision 8fc6e7a835d1 and I get bunch of rejects. It seems dram_ctrl.cc is a bit different in this patch it has all sorts of extra code to deal with ranks. So I wondering this patch requires another one to apply properly. Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson via gem5-dev Sent: Saturday, December 13, 2014 2:12 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) This patch should hopefully solve the issue with the refresh event: http://reviews.gem5.org/r/2573/ Andreas On 11/12/2014 15:52, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Andreas, I've tried applying this patch on top of revision 8fc6e7a835d1 and I get bunch of rejects. It seems dram_ctrl.cc is a bit different in this patch it has all sorts of extra code to deal with ranks. So I wondering this patch requires another one to apply properly. Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson via gem5-dev Sent: Saturday, December 13, 2014 2:12 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) This patch should hopefully solve the issue with the refresh event: http://reviews.gem5.org/r/2573/ Andreas On 11/12/2014 15:52, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Gabe, is that an AMD system? The intel side works fine, it is failing on an AMD system. I will try to run some regression tests and see if I can find a failure in the standard set of tests. The AMD system I am on has a pretty old OS, which might be part of my issue. I don't want to block the fix if it is working fine for others. Getting the intel functionality is what I needed. On Sun, Dec 14, 2014 at 9:53 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: I just tried running bzip2 approximately like the regressions would but with the KVM CPU, and it seemed to work just fine. The only thing I changed was I hacked se.py to set the cwd to what bzip2 expects. Can you please provide more specific instructions how to reproduce the hang/crash you're seeing? If this is working as expected, it would be good to get it checked in and get KVM working again. Gabe $ M5_CPU2000=/usr/local/google/home/gabeblack/gem5/dist/m5/cpu2000/ ./build/X86/gem5.opt configs/example/se.py -c /usr/local/google/home/gabeblack/gem5/dist/m5/cpu2000/binaries/x86/linux/bzip2 --cpu-type=kvm -o 'input.source 1' gem5 Simulator System. http://gem5.org gem5 is copyrighted software; use the --copyright option for details. gem5 compiled Dec 14 2014 21:18:54 gem5 started Dec 14 2014 21:49:12 gem5 executing on gabeblackz620.mtv.corp.google.com command line: ./build/X86/gem5.opt configs/example/se.py -c /usr/local/google/home/gabeblack/gem5/dist/m5/cpu2000/binaries/x86/linux/bzip2 --cpu-type=kvm -o input.source 1 Global frequency set at 1 ticks per second warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes) 0: system.remote_gdb.listener: listening for remote gdb #0 on port 7000 REAL SIMULATION info: KVM: Coalesced MMIO disabled by config. info: Entering event queue @ 0. Starting simulation... warn: kvm-x86: MSR (0x12) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x11) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d01) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d00) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4000) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4001) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4073) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d02) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d03) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d04) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x3a) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x3b) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x6e0) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x1a0) unsupported by gem5. Skipping. spec_init Loading Input Data Input data 1048576 bytes in length Compressing Input Data, level 7 info: Increasing stack size by one page. info: Increasing stack size by one page. info: Increasing stack size by one page. Compressed data 198546 bytes in length Uncompressing Data Uncompressed data 1048576 bytes in length Uncompressed data compared correctly Compressing Input Data, level 9 Compressed data 198677 bytes in length Uncompressing Data Uncompressed data 1048576 bytes in length Uncompressed data compared correctly Tested 1MB buffer: OK! Exiting @ tick 13987682791500 because target called exit() hack: Pretending totalOps is equivalent to totalInsts() On Sat, Dec 13, 2014 at 12:12 AM, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: This patch should hopefully solve the issue with the refresh event: http://reviews.gem5.org/r/2573/ Andreas On 11/12/2014 15:52, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
-Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I just tried running bzip2 approximately like the regressions would but with the KVM CPU, and it seemed to work just fine. The only thing I changed was I hacked se.py to set the cwd to what bzip2 expects. Can you please provide more specific instructions how to reproduce the hang/crash you're seeing? If this is working as expected, it would be good to get it checked in and get KVM working again. Gabe $ M5_CPU2000=/usr/local/google/home/gabeblack/gem5/dist/m5/cpu2000/ ./build/X86/gem5.opt configs/example/se.py -c /usr/local/google/home/gabeblack/gem5/dist/m5/cpu2000/binaries/x86/linux/bzip2 --cpu-type=kvm -o 'input.source 1' gem5 Simulator System. http://gem5.org gem5 is copyrighted software; use the --copyright option for details. gem5 compiled Dec 14 2014 21:18:54 gem5 started Dec 14 2014 21:49:12 gem5 executing on gabeblackz620.mtv.corp.google.com command line: ./build/X86/gem5.opt configs/example/se.py -c /usr/local/google/home/gabeblack/gem5/dist/m5/cpu2000/binaries/x86/linux/bzip2 --cpu-type=kvm -o input.source 1 Global frequency set at 1 ticks per second warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes) 0: system.remote_gdb.listener: listening for remote gdb #0 on port 7000 REAL SIMULATION info: KVM: Coalesced MMIO disabled by config. info: Entering event queue @ 0. Starting simulation... warn: kvm-x86: MSR (0x12) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x11) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d01) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d00) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4000) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4001) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4073) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d02) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d03) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x4b564d04) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x3a) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x3b) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x6e0) unsupported by gem5. Skipping. warn: kvm-x86: MSR (0x1a0) unsupported by gem5. Skipping. spec_init Loading Input Data Input data 1048576 bytes in length Compressing Input Data, level 7 info: Increasing stack size by one page. info: Increasing stack size by one page. info: Increasing stack size by one page. Compressed data 198546 bytes in length Uncompressing Data Uncompressed data 1048576 bytes in length Uncompressed data compared correctly Compressing Input Data, level 9 Compressed data 198677 bytes in length Uncompressing Data Uncompressed data 1048576 bytes in length Uncompressed data compared correctly Tested 1MB buffer: OK! Exiting @ tick 13987682791500 because target called exit() hack: Pretending totalOps is equivalent to totalInsts() On Sat, Dec 13, 2014 at 12:12 AM, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: This patch should hopefully solve the issue with the refresh event: http://reviews.gem5.org/r/2573/ Andreas On 11/12/2014 15:52, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
This patch should hopefully solve the issue with the refresh event: http://reviews.gem5.org/r/2573/ Andreas On 11/12/2014 15:52, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I was using SE, with just se.py --cpu-type=kvm On Thu, Dec 11, 2014 at 7:52 AM, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
with --mem-type=SimpleMemory it panics and dies. this is using bzip2, I am going to try another benchmark as well. without SimpleMemory, bzip2 hangs sometime between the 20M and 30M instruction. Time is advancing, but instructions are not retiring. On Thu, Dec 11, 2014 at 11:42 AM, mike upton michaelup...@gmail.com wrote: I was using SE, with just se.py --cpu-type=kvm On Thu, Dec 11, 2014 at 7:52 AM, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
libquantum runs fine. I will do a sweep of all the apps and post the results. On Thu, Dec 11, 2014 at 11:58 AM, mike upton michaelup...@gmail.com wrote: with --mem-type=SimpleMemory it panics and dies. this is using bzip2, I am going to try another benchmark as well. without SimpleMemory, bzip2 hangs sometime between the 20M and 30M instruction. Time is advancing, but instructions are not retiring. On Thu, Dec 11, 2014 at 11:42 AM, mike upton michaelup...@gmail.com wrote: I was using SE, with just se.py --cpu-type=kvm On Thu, Dec 11, 2014 at 7:52 AM, Andreas Hansson via gem5-dev gem5-dev@gem5.org wrote: Hi Alex, Mike, I¹ll try and fix this whole issue of the refresh event once and for all. SimpleMemory should only really be used for fast-forwarding and high-level sweeps, and I would like to ensure there are really no reasons to move away from the DRAM controller. It seems the sensible thing to do is to use startup and drainResume as the points where we check the mode of the memory system and either disable/enable the refresh event of the DRAM controller. Hopefully I will have something working before the weekend. Andreas On 11/12/2014 15:32, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, Are you running with SimpleMemory, SE or FS? On my AMD platform, for SE, I get very similar execution times with old implementation, for SimpleMemory and classic memory with detailed memory controller. Also what linux kernel are you using? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Wednesday, December 10, 2014 3:59 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I was testing this on both Intel and AMD platforms. The new code does seem to work for Intel platforms. The new code also seems to clean up a bunch of runtime warnings I was getting on AMD platforms. However the new code on AMD is either much slower, or it is stuck in a loop. A test that runs for 30 sec with the old code is running for more than 10 mins, and still has a long way to go. On Wed, Dec 10, 2014 at 12:27 PM, Gabe Black via gem5-dev gem5-dev@gem5.org wrote: That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Thanks for the clarification, actually WP needs to be reset for CPU switching to work properly. I think it is better to leave alignment check off for SE mode as there is no handler in place for unaligned access exception. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 2:28 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) That's not actually extending the TSS limit, that's what it works out to be with the granularity bit set. The AM and WP bits were set to the wrong thing according to the comments next to them I'm pretty sure. If we wanted the other behavior, we might be able to change them back and have it work. The _BASE registers hold the base of segments as they're specified by the ISA. The _EFF_BASE registers hold the base that will actually be used in address calculations based on the mode of the CPU. For instance, if you're in 64 bit mode, the _BASE of DS might still be 0xFFF from when you were in another mode. The _EFF_BASE would be 0 though, since the DS base is ignored in that case. _EFF_BASE may not be used by the KVM CPU, but it should be set up anyway in case we switch back to a regular CPU. Gabe On Wed, Dec 10, 2014 at 10:20 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Thank you for all the clarifications. I see that for SE to work on vmx the TSS limit had to be extended, am and wp bits in CR0 had to be reset and *_EFF_BASE registers had be set in addition to *_BASE registers for TR TSG IDTR. I wonder what is TR_EFF_BASE. It seems that the contents of TR register (which gets passed to kvm) are the same if with or without *_EFF_BASE registers set. Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Wednesday, December 10, 2014 1:21 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Will someone be providing a patch for this? I am happy to test it. From Adrian's description it seems there are a bunch of issues. On Tue, Dec 9, 2014 at 12:09 AM, Adrián Colaso Diego via gem5-dev gem5-dev@gem5.org wrote: You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Adrian, Sorry for missing your first email. I have solved the interchanged segment limits and other bits in segment descriptors for full system mode, though I get a different behavior on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can you please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Tuesday, December 09, 2014 11:52 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Will someone be providing a patch for this? I am happy to test it. From Adrian's description it seems there are a bunch of issues. On Tue, Dec 9, 2014 at 12:09 AM, Adrián Colaso Diego via gem5-dev gem5-dev@gem5.org wrote: You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I'm working on a patch to overhaul the segment setup stuff. Gabe On Tue, Dec 9, 2014 at 11:36 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I have solved the interchanged segment limits and other bits in segment descriptors for full system mode, though I get a different behavior on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can you please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of mike upton via gem5-dev Sent: Tuesday, December 09, 2014 11:52 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Will someone be providing a patch for this? I am happy to test it. From Adrian's description it seems there are a bunch of issues. On Tue, Dec 9, 2014 at 12:09 AM, Adrián Colaso Diego via gem5-dev gem5-dev@gem5.org wrote: You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Ok, I got SE working too. I'll clean up my patch and send that out in a bit. Gabe On Tue, Dec 9, 2014 at 9:41 PM, Gabe Black gabebl...@google.com wrote: I figured out what the other problem was, so here's the review. http://reviews.gem5.org/r/2557/ Gabe On Tue, Dec 9, 2014 at 5:00 PM, Gabe Black gabebl...@google.com wrote: It was attached in my sent mail. Maybe it's being blocked by something? I'm hunting down another problem so I don't want to move my tree around too much, but once that's done I'll post it as a review. Gabe On Tue, Dec 9, 2014 at 4:51 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: I haven't received any attachment to your email. So I don't have your patch. Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:42 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) And... it turns out the KVM change wasn't necessary. If you're working from my patch, get rid of where the segment limit is divided by PageBytes. That was only necessary because I wasn't adding 0xFFF to the limit when the granularity bit was set. Gabe On Tue, Dec 9, 2014 at 4:31 PM, Gabe Black gabebl...@google.com wrote: Oh, also segment limits weren't being computed correctly in the installSegDesc function, although I don't think that was from the KVM stuff. Once it was fixed it required adjusting the KVM stuff a little, though. Gabe On Tue, Dec 9, 2014 at 4:29 PM, Gabe Black gabebl...@google.com wrote: Here is my patch so far. There were a few things wrong, although I didn't really keep notes. The limits were mixed up, the long mode bit was set on all descriptors when it's only valid for the code segment, privilege level 0 is the OS and 3 is for applications and not the other way around, and I think the type was being set wrong for one of the segments. Also, the syscall and sysenter registers (star and friends) require the segments in the GDT to be in a particular order which I don't think they were. Gabe On Tue, Dec 9, 2014 at 4:22 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: So, I am doing this on an AMD system and I have SE working and am able to get FS entering into virtualized mode. However, in FS I get an early exception while the kernel is booting. This seems a bit different from what Nilay and Adrian observed for FS. Could you please share the diffs that got FS working? Thanks, Alex -Original Message- From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe Black via gem5-dev Sent: Tuesday, December 09, 2014 6:07 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) Oh, I see you have FS working again and not SE. NM, I'll keep looking. Gabe On Tue, Dec 9, 2014 at 4:04 PM, Gabe Black gabebl...@google.com wrote: I have FS working again which is good, but I'm still having problems with SE. If you could let me know what you did to get things going that would be very helpful. Gabe On Tue, Dec 9, 2014 at 10:06 AM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Adrian, Sorry for missing your first email. I do see the interchanged segment limits for full system mode, though I get a different behaviour on my system. The simulation seems to hang in the following manner: Processor #0 (Bootup-CPU) I/O APIC #1 at 0xFEC0. Setting APIC routing to flat Processors: 1 PANIC: early exception rip 807909a9 error 9 cr2 ff5fd020 Can please provide a patch with all the modifications that fixed the issue on your system? Thank you, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Adrián Colaso Diego via gem5-dev [gem5-dev@gem5.org] Sent: Tuesday, December 09, 2014 2:09 AM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) You are right Nilay. I sent an email last week but nobody has replied. It seems that descriptors (cdDesc, dsDesc and tssDesc) located in src/arch/x86/system.cc file are not well-initialized and as a consequence kvm does not work when running in full-system mode. Segment limits values (limitHigh and limitLow) are interchanged and several segment descriptor values are wrong too. If these values are corrected kvm works again as before. Adrian El lun, 08-12-2014 a las 22:50 -0600, Nilay Vaish via gem5-dev escribió: I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: authorAlexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware
[gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel)
I also faced problem in getting KVM CPU to run in FS mode. I figured that the following changeset causes problems: author Alexandru Dutu alexandru.d...@amd.com Sun Nov 23 18:01:08 2014 -0800 (2 weeks ago) changeset 10554 fe2e2f06a7c8 I saw the hardware reason 0x8021, but did not try to figure what was going on wrong. -- Nilay On Mon, 8 Dec 2014, Gabe Black via gem5-dev wrote: I'm pretty sure entering 64 bit mode is the same between AMD and Intel CPUs. I vaguely remember there being some subtle page table difference though, and gem5 is building the page tables in SE mode instead of the kernel. Gabe On Mon, Dec 8, 2014 at 7:44 PM, Dutu, Alexandru via gem5-dev gem5-dev@gem5.org wrote: Hi Mike, trace-cmd is a very handy tool to get an overview of what the kvm kernel module is doing before going into gdb. In extreme cases ftrace can be useful as well. What is the error that you are seeing? Is it still failing to enter virtualized mode? If that is the case and the hardware reason is 0x8021, that seems to be an unrecoverable exception (drivers/hv/hyperv_vmbus.h in linux kernel source code). When running in SE mode, we are trying to bring the machine state to full 64bit mode without going through legacy modes. It might be that Intel machines have a different way of going to 64bit mode than AMD machines (different CR4, different way of enabling 64bit mode page tables etc.). I remember dealing with these issue for AMD platforms by going through System Programming manual and making sure gem5 gets all the bits right as there is not much the KVM kernel model will tell about the cause of failure. Best regards, Alex From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Gabe Black via gem5-dev [gem5-dev@gem5.org] Sent: Monday, December 08, 2014 7:08 PM To: gem5 Developer List Subject: Re: [gem5-dev] x86 SE kvm functionality (AMD vs Intel) I'm not an expert either, but I did have problems running KVM in SE mode on an Intel CPU. I didn't look into it that much, but I think things failed in the kernel somewhere. What might be happening is that the different vendors hardware virtualization mechanisms are more or less picky about various things. Something might be set up incorrectly, and one implementation gets more upset about it than the other. I believe there are tools which will help you determine whether your VM state is legal. Perhaps Andreas can tell you more about those? Gabe On Mon, Dec 8, 2014 at 4:29 PM, mike upton via gem5-dev gem5-dev@gem5.org wrote: I have verified that x86 kvm works fine on AMD platforms, but fails on Intel platforms. Any hints about how to narrow down the cause (other than diving into gdb, which I will do). I am not an expert in KVM or how gem5 hooks up to libkvm. ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
