Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Gary Ching-Pang Lin]

On Sat, Aug 31, 2013 at 12:16:21AM -0700, Jordan Justen wrote:
> On Thu, Aug 29, 2013 at 8:28 PM, Gary Ching-Pang Lin  wrote:
> > On Fri, Aug 30, 2013 at 02:04:40AM +1000, Bruce Rogers wrote:
> >> I tried this out, and I get the black screen as well when ept=n, but it 
> >> boots
> >> successfully to the efi shell when ept=y.
> >>
> >> Gary, what does 'cat /sys/module/kvm_intel/parameters/ept' report on your
> >> failing machine?
> >>
> > I think this is the root cause. My machine doesn't support ept.
> 
> Gary, Bruce,
> 
> Can one of you try this OVMF branch to see if it boots when EPT is not 
> enabled?
> 
> https://github.com/jljusten/edk2.git ovmf-vm-vtf0
> 
Hi Jordan,

The black screen issue was gone after applying the patches.

Thanks,

Gary Lin

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Jordan Justen]

On Thu, Aug 29, 2013 at 8:28 PM, Gary Ching-Pang Lin  wrote:
> On Fri, Aug 30, 2013 at 02:04:40AM +1000, Bruce Rogers wrote:
>> I tried this out, and I get the black screen as well when ept=n, but it boots
>> successfully to the efi shell when ept=y.
>>
>> Gary, what does 'cat /sys/module/kvm_intel/parameters/ept' report on your
>> failing machine?
>>
> I think this is the root cause. My machine doesn't support ept.

Gary, Bruce,

Can one of you try this OVMF branch to see if it boots when EPT is not enabled?

https://github.com/jljusten/edk2.git ovmf-vm-vtf0

Thanks for your time,

-Jordan

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Paolo Bonzini]

Il 30/08/2013 21:05, Jordan Justen ha scritto:
> I meant your proposed update to VTF0 is not quite what is needed
> within EDK II. The current ROM based tables need to stay because that
> is what real firmware would need to use to enable a 64-bit PEI.

Ok, understood.

> OVMF will just have to do something different to work around the KVM
> issue.

I don't think anything special is needed, we can just document that KVM
+ a new QEMU requires a new kernel.

Paolo

> I did some work on this, but ended up going with the simpler
> change of r14494 because it appeared to be enough to work around the
> issue.

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Paolo Bonzini]

Il 30/08/2013 19:39, Jordan Justen ha scritto:
>> However, if you guys can figure out a patch for
>> > UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm so that it builds
>> > the page tables in RAM (and removing the page table Python code), that
>> > would also work.
> That path is not quite how it needs to be fixed, but I'll look into it.

Yes, KVM needs to be fixed as well.

> I had done some work to entirely eliminate the ROM based tables, but
> hit some complications. I'll work on it some more... (It has another
> large benefit of saving nearly 28k of ROM space.)

I also tried to prototype the change, but I have no idea which parts of
low memory are available and/or how to reserve 24k there.

Paolo

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Jordan Justen]

On Fri, Aug 30, 2013 at 11:44 AM, Paolo Bonzini  wrote:
> Il 30/08/2013 19:39, Jordan Justen ha scritto:
>>> However, if you guys can figure out a patch for
>>> > UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm so that it builds
>>> > the page tables in RAM (and removing the page table Python code), that
>>> > would also work.
>> That path is not quite how it needs to be fixed, but I'll look into it.
>
> Yes, KVM needs to be fixed as well.

I meant your proposed update to VTF0 is not quite what is needed
within EDK II. The current ROM based tables need to stay because that
is what real firmware would need to use to enable a 64-bit PEI.

OVMF will just have to do something different to work around the KVM
issue. I did some work on this, but ended up going with the simpler
change of r14494 because it appeared to be enough to work around the
issue.

-Jordan

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Jordan Justen]

On Fri, Aug 30, 2013 at 4:58 AM, Paolo Bonzini  wrote:
> However, if you guys can figure out a patch for
> UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm so that it builds
> the page tables in RAM (and removing the page table Python code), that
> would also work.

That path is not quite how it needs to be fixed, but I'll look into it.

I had done some work to entirely eliminate the ROM based tables, but
hit some complications. I'll work on it some more... (It has another
large benefit of saving nearly 28k of ROM space.)

-Jordan

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Jordan Justen]

On Fri, Aug 30, 2013 at 5:42 AM, Paolo Bonzini  wrote:
> Il 30/08/2013 14:10, Gleb Natapov ha scritto:
>> On Fri, Aug 30, 2013 at 01:58:59PM +0200, Paolo Bonzini wrote:
>>> Il 30/08/2013 11:37, Laszlo Ersek ha scritto:
 Disclaimer: I don't know what I'm talking about.
>>>
>>> No problem. :)
>>>
 So, Jordan's patch for OVMF (SVN r14494) builds the page tables (and
 finally writes the root to CR3) in a phase when paging is not enabled
 yet in the VM.

 Again, I have no clue, but if the guest hasn't even enabled paging yet,
 then the hypervisor (without EPT?) might have no idea that what the
 guest is writing to memory are its pagetables-to-be. The first notice
 the hypervisor might take is the store to CR3. At which point (or maybe
 even later, when paging is enabled?) the hypervisor would have to walk
 the guest's tables all at once, and build the shadow tables "in batch".
>>>
>>> The hypervisor builds shadow page tables lazily; as soon as CR0.PG is
>>> set the next instruction will pagefault and shadow page tables will
>>> start to get populated.
>>>
>>> However, surprise!  There is another set of "flat" page tables for X64,
>>> built by UefiCpuPkg/ResetVector/Vtf0/Tools/FixupForRawSection.py when
>>> you run UefiCpuPkg/ResetVector/Vtf0/Build.py.  These are always in ROM.
>>>
>>> As in Jordan's patches, the problem is that the hypervisor is expecting
>>> to be able to write to the page tables, but this is not the case because
>>> the page tables are in a read-only memory slot.
>>>
>> Only when setting dirty/accessed bits, are they not set in ROM version
>> of page tables?
>
> They are set, but the hypervisor is using gfn_to_hva unconditionally.
> See the patch I just posted.

We should be building the tables in ROM such that they are marked as
present, accessed and dirty. Thus, the processor should never want to
write to them.

My first pass 'fix' to this was to transition the tables to RAM very
early, but I think they ought to have worked from the ROM. (I just
assumed we were dealing with some KVM limitation, or was it a KVM
bug?)

-Jordan

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Paolo Bonzini]

Il 30/08/2013 14:10, Gleb Natapov ha scritto:
> On Fri, Aug 30, 2013 at 01:58:59PM +0200, Paolo Bonzini wrote:
>> Il 30/08/2013 11:37, Laszlo Ersek ha scritto:
>>> Disclaimer: I don't know what I'm talking about.
>>
>> No problem. :)
>>
>>> So, Jordan's patch for OVMF (SVN r14494) builds the page tables (and
>>> finally writes the root to CR3) in a phase when paging is not enabled
>>> yet in the VM.
>>>
>>> Again, I have no clue, but if the guest hasn't even enabled paging yet,
>>> then the hypervisor (without EPT?) might have no idea that what the
>>> guest is writing to memory are its pagetables-to-be. The first notice
>>> the hypervisor might take is the store to CR3. At which point (or maybe
>>> even later, when paging is enabled?) the hypervisor would have to walk
>>> the guest's tables all at once, and build the shadow tables "in batch".
>>
>> The hypervisor builds shadow page tables lazily; as soon as CR0.PG is
>> set the next instruction will pagefault and shadow page tables will
>> start to get populated.
>>
>> However, surprise!  There is another set of "flat" page tables for X64,
>> built by UefiCpuPkg/ResetVector/Vtf0/Tools/FixupForRawSection.py when
>> you run UefiCpuPkg/ResetVector/Vtf0/Build.py.  These are always in ROM.
>>
>> As in Jordan's patches, the problem is that the hypervisor is expecting
>> to be able to write to the page tables, but this is not the case because
>> the page tables are in a read-only memory slot.
>>
> Only when setting dirty/accessed bits, are they not set in ROM version
> of page tables?

They are set, but the hypervisor is using gfn_to_hva unconditionally.
See the patch I just posted.

Paolo

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Gleb Natapov]

On Fri, Aug 30, 2013 at 01:58:59PM +0200, Paolo Bonzini wrote:
> Il 30/08/2013 11:37, Laszlo Ersek ha scritto:
> > Disclaimer: I don't know what I'm talking about.
> 
> No problem. :)
> 
> > So, Jordan's patch for OVMF (SVN r14494) builds the page tables (and
> > finally writes the root to CR3) in a phase when paging is not enabled
> > yet in the VM.
> > 
> > Again, I have no clue, but if the guest hasn't even enabled paging yet,
> > then the hypervisor (without EPT?) might have no idea that what the
> > guest is writing to memory are its pagetables-to-be. The first notice
> > the hypervisor might take is the store to CR3. At which point (or maybe
> > even later, when paging is enabled?) the hypervisor would have to walk
> > the guest's tables all at once, and build the shadow tables "in batch".
> 
> The hypervisor builds shadow page tables lazily; as soon as CR0.PG is
> set the next instruction will pagefault and shadow page tables will
> start to get populated.
> 
> However, surprise!  There is another set of "flat" page tables for X64,
> built by UefiCpuPkg/ResetVector/Vtf0/Tools/FixupForRawSection.py when
> you run UefiCpuPkg/ResetVector/Vtf0/Build.py.  These are always in ROM.
> 
> As in Jordan's patches, the problem is that the hypervisor is expecting
> to be able to write to the page tables, but this is not the case because
> the page tables are in a read-only memory slot.
> 
Only when setting dirty/accessed bits, are they not set in ROM version
of page tables?

> Making shadow and EPT behave similarly is probably a good thing, so I'm
> sending an RFC patch to k...@vger.kernel.org that fixes the bug.
> 
> However, if you guys can figure out a patch for
> UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm so that it builds
> the page tables in RAM (and removing the page table Python code), that
> would also work.
> 
> Paolo

--
Gleb.

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Paolo Bonzini]

Il 30/08/2013 11:37, Laszlo Ersek ha scritto:
> Disclaimer: I don't know what I'm talking about.

No problem. :)

> So, Jordan's patch for OVMF (SVN r14494) builds the page tables (and
> finally writes the root to CR3) in a phase when paging is not enabled
> yet in the VM.
> 
> Again, I have no clue, but if the guest hasn't even enabled paging yet,
> then the hypervisor (without EPT?) might have no idea that what the
> guest is writing to memory are its pagetables-to-be. The first notice
> the hypervisor might take is the store to CR3. At which point (or maybe
> even later, when paging is enabled?) the hypervisor would have to walk
> the guest's tables all at once, and build the shadow tables "in batch".

The hypervisor builds shadow page tables lazily; as soon as CR0.PG is
set the next instruction will pagefault and shadow page tables will
start to get populated.

However, surprise!  There is another set of "flat" page tables for X64,
built by UefiCpuPkg/ResetVector/Vtf0/Tools/FixupForRawSection.py when
you run UefiCpuPkg/ResetVector/Vtf0/Build.py.  These are always in ROM.

As in Jordan's patches, the problem is that the hypervisor is expecting
to be able to write to the page tables, but this is not the case because
the page tables are in a read-only memory slot.

Making shadow and EPT behave similarly is probably a good thing, so I'm
sending an RFC patch to k...@vger.kernel.org that fixes the bug.

However, if you guys can figure out a patch for
UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm so that it builds
the page tables in RAM (and removing the page table Python code), that
would also work.

Paolo

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Laszlo Ersek]

On 08/30/13 05:28, Gary Ching-Pang Lin wrote:
> On Fri, Aug 30, 2013 at 02:04:40AM +1000, Bruce Rogers wrote:
>>  >>> On 8/29/2013 at 02:23 AM, Gary Ching-Pang Lin  wrote: 
>>> On Wed, Aug 28, 2013 at 02:55:26PM +0200, Andreas Färber wrote:
 Am 28.08.2013 14:10, schrieb Laszlo Ersek:
> On 08/28/13 13:49, Andreas Färber wrote:
>> Am 28.08.2013 13:45, schrieb Laszlo Ersek:
>>> (qemu-devel CC'd)
>>>
>>> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
 Hi,

 I recently updated qemu to 1.6.0 and found OVMF just showed a blank
 screen when kvm was enabled. I tried to dump OVMF log with the
 following commond but nothing was stored in debug.log.

 qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon 
 file:debug.log -global 
>>> isa-debugcon.iobase=0x402

 The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
 and uploaded to the following link:

>>> https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing

 I found a similar case with kernel < 3.9, but I already upgraded linux
 kernel to 3.10.5, so this may be another bug.
>>>
>>> Well, the usual first response in cases like this is...
>>>
>>> Can you bisect qemu? :)
>>
>> We had a similar report:
>> https://bugzilla.novell.com/show_bug.cgi?id=835895
>
> Well that's sorta the same report, considering you and Gary both work
> for SUSE, and the Novell BZ seems to imply the build in question was 
>>> Gary's:
>
>> qemu 1.6.0 fails to run the tianocore firmware
>> (home:gary_lin:UEFI/OVMF) properly. This worked with previous qemu
> 
>> versions:
>
> :)

 Different reporters, so who knows if the setups are the same. ;)

>> git-bisect said:
>> 235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
>> commit 235e8982ad393e5611cb892df54881c872eea9e1
>> Author: Jordan Justen 
>> Date:   Wed May 29 01:27:26 2013 -0700
>>
>> kvm: support using KVM_MEM_READONLY flag for regions
>>
>> For readonly memory regions and rom devices in romd_mode,
>> we make use of the KVM_MEM_READONLY. A slot that uses
>> KVM_MEM_READONLY can be read from and code can execute from the
>> region, but writes will exit to qemu.
>>
>> For rom devices with !romd_mode, we force the slot to be
>> removed so reads or writes to the region will exit to qemu.
>> (Note that a memory region in this state is not executable
>> within kvm.)
>>
>> v7:
>>  * Update for readable => romd_mode rename (5f9a5ea1)
>>
>> Signed-off-by: Jordan Justen 
>> Reviewed-by: Xiao Guangrong  (v4)
>> Reviewed-by: Paolo Bonzini  (v5)
>> Message-id: 
>> 1369816047-16384-4-git-send-email-jordan.l.jus...@intel.com
>> Signed-off-by: Anthony Liguori 
>>
>>
>> Any hints or patches welcome. :)
>
> Hm. LP 1212402  probably
> concerns the "similar case with kernel < 3.9" mentioned by Gary, and is
> likely not revelant here.
>
>
> Gary & Ludwig, can you confirm that your OVMF build includes SVN r14494?
>
> Author: Jordan Justen 
> Date:   Thu Jul 18 22:51:27 2013 +
>
> OvmfPkg/Sec: Build identity mapped pages in RAM for X64
>
> This is based on MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c.
>
> Previously we would run using page tables built into the
> firmware device.
>
> If a flash memory is available, it is unsafe for the page
> tables to be stored in memory since the processor may try
> to write to the page table data structures.
>
> Additionally, when KVM ROM support is enabled for the
> firmware device, then PEI fails to boot when the page
> tables are in the firmware device.

 https://build.opensuse.org/package/show/Virtualization/OVMF
 is at r14547 and that one works for me.

 Gary/Ludwig, can you confirm that this is resolved?

>>> I still got the black screen even with OVMF r14547 or r14608. I also
>>> tried kernel 3.10.9 and 3.11-rc7, but no luck.
>>>
>>> Reverting the KVM_MEM_READONLY patch works for me, but the OVMF patch
>>> somehow didn't work.
>>>
>>> Thanks,
>>>
>>> Gary Lin
>>
>> I tried this out, and I get the black screen as well when ept=n, but it boots
>> successfully to the efi shell when ept=y.
>>
>> Gary, what does 'cat /sys/module/kvm_intel/parameters/ept' report on your
>> failing machine?
>>
> I think this is the root cause. My machine doesn't support ept.

Disclaimer: I don't know what I'm talking about.

So, Jordan's patch for OVMF (SVN r14494) builds the page tables (and
finally writes the root to CR3) in a phase when paging is not enabled

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Paolo Bonzini]

Il 30/08/2013 05:28, Gary Ching-Pang Lin ha scritto:
>> > I tried this out, and I get the black screen as well when ept=n, but it 
>> > boots
>> > successfully to the efi shell when ept=y.
>> > 
>> > Gary, what does 'cat /sys/module/kvm_intel/parameters/ept' report on your
>> > failing machine?
>> >
> I think this is the root cause. My machine doesn't support ept.

Thanks for the report!

Paolo

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Gary Ching-Pang Lin]

On Fri, Aug 30, 2013 at 02:04:40AM +1000, Bruce Rogers wrote:
>  >>> On 8/29/2013 at 02:23 AM, Gary Ching-Pang Lin  wrote: 
> > On Wed, Aug 28, 2013 at 02:55:26PM +0200, Andreas Färber wrote:
> >> Am 28.08.2013 14:10, schrieb Laszlo Ersek:
> >> > On 08/28/13 13:49, Andreas Färber wrote:
> >> >> Am 28.08.2013 13:45, schrieb Laszlo Ersek:
> >> >>> (qemu-devel CC'd)
> >> >>>
> >> >>> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
> >>  Hi,
> >> 
> >>  I recently updated qemu to 1.6.0 and found OVMF just showed a blank
> >>  screen when kvm was enabled. I tried to dump OVMF log with the
> >>  following commond but nothing was stored in debug.log.
> >> 
> >>  qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon 
> >>  file:debug.log -global 
> > isa-debugcon.iobase=0x402
> >> 
> >>  The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
> >>  and uploaded to the following link:
> >>  
> > https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing
> >> 
> >>  I found a similar case with kernel < 3.9, but I already upgraded linux
> >>  kernel to 3.10.5, so this may be another bug.
> >> >>>
> >> >>> Well, the usual first response in cases like this is...
> >> >>>
> >> >>> Can you bisect qemu? :)
> >> >>
> >> >> We had a similar report:
> >> >> https://bugzilla.novell.com/show_bug.cgi?id=835895
> >> > 
> >> > Well that's sorta the same report, considering you and Gary both work
> >> > for SUSE, and the Novell BZ seems to imply the build in question was 
> > Gary's:
> >> > 
> >> >> qemu 1.6.0 fails to run the tianocore firmware
> >> >> (home:gary_lin:UEFI/OVMF) properly. This worked with previous qemu
> >> > 
> >> >> versions:
> >> > 
> >> > :)
> >> 
> >> Different reporters, so who knows if the setups are the same. ;)
> >> 
> >> >> git-bisect said:
> >> >> 235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
> >> >> commit 235e8982ad393e5611cb892df54881c872eea9e1
> >> >> Author: Jordan Justen 
> >> >> Date:   Wed May 29 01:27:26 2013 -0700
> >> >>
> >> >> kvm: support using KVM_MEM_READONLY flag for regions
> >> >>
> >> >> For readonly memory regions and rom devices in romd_mode,
> >> >> we make use of the KVM_MEM_READONLY. A slot that uses
> >> >> KVM_MEM_READONLY can be read from and code can execute from the
> >> >> region, but writes will exit to qemu.
> >> >>
> >> >> For rom devices with !romd_mode, we force the slot to be
> >> >> removed so reads or writes to the region will exit to qemu.
> >> >> (Note that a memory region in this state is not executable
> >> >> within kvm.)
> >> >>
> >> >> v7:
> >> >>  * Update for readable => romd_mode rename (5f9a5ea1)
> >> >>
> >> >> Signed-off-by: Jordan Justen 
> >> >> Reviewed-by: Xiao Guangrong  (v4)
> >> >> Reviewed-by: Paolo Bonzini  (v5)
> >> >> Message-id: 
> >> >> 1369816047-16384-4-git-send-email-jordan.l.jus...@intel.com
> >> >> Signed-off-by: Anthony Liguori 
> >> >>
> >> >>
> >> >> Any hints or patches welcome. :)
> >> > 
> >> > Hm. LP 1212402  probably
> >> > concerns the "similar case with kernel < 3.9" mentioned by Gary, and is
> >> > likely not revelant here.
> >> > 
> >> > 
> >> > Gary & Ludwig, can you confirm that your OVMF build includes SVN r14494?
> >> > 
> >> > Author: Jordan Justen 
> >> > Date:   Thu Jul 18 22:51:27 2013 +
> >> > 
> >> > OvmfPkg/Sec: Build identity mapped pages in RAM for X64
> >> > 
> >> > This is based on MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c.
> >> > 
> >> > Previously we would run using page tables built into the
> >> > firmware device.
> >> > 
> >> > If a flash memory is available, it is unsafe for the page
> >> > tables to be stored in memory since the processor may try
> >> > to write to the page table data structures.
> >> > 
> >> > Additionally, when KVM ROM support is enabled for the
> >> > firmware device, then PEI fails to boot when the page
> >> > tables are in the firmware device.
> >> 
> >> https://build.opensuse.org/package/show/Virtualization/OVMF
> >> is at r14547 and that one works for me.
> >> 
> >> Gary/Ludwig, can you confirm that this is resolved?
> >>
> > I still got the black screen even with OVMF r14547 or r14608. I also
> > tried kernel 3.10.9 and 3.11-rc7, but no luck.
> > 
> > Reverting the KVM_MEM_READONLY patch works for me, but the OVMF patch
> > somehow didn't work.
> > 
> > Thanks,
> > 
> > Gary Lin
> 
> I tried this out, and I get the black screen as well when ept=n, but it boots
> successfully to the efi shell when ept=y.
> 
> Gary, what does 'cat /sys/module/kvm_intel/parameters/ept' report on your
> failing machine?
>
I think this is the root cause. My machine doesn't support ept.

Gary Lin

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Bruce Rogers]

 >>> On 8/29/2013 at 02:23 AM, Gary Ching-Pang Lin  wrote: 
> On Wed, Aug 28, 2013 at 02:55:26PM +0200, Andreas Färber wrote:
>> Am 28.08.2013 14:10, schrieb Laszlo Ersek:
>> > On 08/28/13 13:49, Andreas Färber wrote:
>> >> Am 28.08.2013 13:45, schrieb Laszlo Ersek:
>> >>> (qemu-devel CC'd)
>> >>>
>> >>> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
>>  Hi,
>> 
>>  I recently updated qemu to 1.6.0 and found OVMF just showed a blank
>>  screen when kvm was enabled. I tried to dump OVMF log with the
>>  following commond but nothing was stored in debug.log.
>> 
>>  qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon 
>>  file:debug.log -global 
> isa-debugcon.iobase=0x402
>> 
>>  The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
>>  and uploaded to the following link:
>>  
> https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing
>> 
>>  I found a similar case with kernel < 3.9, but I already upgraded linux
>>  kernel to 3.10.5, so this may be another bug.
>> >>>
>> >>> Well, the usual first response in cases like this is...
>> >>>
>> >>> Can you bisect qemu? :)
>> >>
>> >> We had a similar report:
>> >> https://bugzilla.novell.com/show_bug.cgi?id=835895
>> > 
>> > Well that's sorta the same report, considering you and Gary both work
>> > for SUSE, and the Novell BZ seems to imply the build in question was 
> Gary's:
>> > 
>> >> qemu 1.6.0 fails to run the tianocore firmware
>> >> (home:gary_lin:UEFI/OVMF) properly. This worked with previous qemu
>> > 
>> >> versions:
>> > 
>> > :)
>> 
>> Different reporters, so who knows if the setups are the same. ;)
>> 
>> >> git-bisect said:
>> >> 235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
>> >> commit 235e8982ad393e5611cb892df54881c872eea9e1
>> >> Author: Jordan Justen 
>> >> Date:   Wed May 29 01:27:26 2013 -0700
>> >>
>> >> kvm: support using KVM_MEM_READONLY flag for regions
>> >>
>> >> For readonly memory regions and rom devices in romd_mode,
>> >> we make use of the KVM_MEM_READONLY. A slot that uses
>> >> KVM_MEM_READONLY can be read from and code can execute from the
>> >> region, but writes will exit to qemu.
>> >>
>> >> For rom devices with !romd_mode, we force the slot to be
>> >> removed so reads or writes to the region will exit to qemu.
>> >> (Note that a memory region in this state is not executable
>> >> within kvm.)
>> >>
>> >> v7:
>> >>  * Update for readable => romd_mode rename (5f9a5ea1)
>> >>
>> >> Signed-off-by: Jordan Justen 
>> >> Reviewed-by: Xiao Guangrong  (v4)
>> >> Reviewed-by: Paolo Bonzini  (v5)
>> >> Message-id: 
>> >> 1369816047-16384-4-git-send-email-jordan.l.jus...@intel.com
>> >> Signed-off-by: Anthony Liguori 
>> >>
>> >>
>> >> Any hints or patches welcome. :)
>> > 
>> > Hm. LP 1212402  probably
>> > concerns the "similar case with kernel < 3.9" mentioned by Gary, and is
>> > likely not revelant here.
>> > 
>> > 
>> > Gary & Ludwig, can you confirm that your OVMF build includes SVN r14494?
>> > 
>> > Author: Jordan Justen 
>> > Date:   Thu Jul 18 22:51:27 2013 +
>> > 
>> > OvmfPkg/Sec: Build identity mapped pages in RAM for X64
>> > 
>> > This is based on MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c.
>> > 
>> > Previously we would run using page tables built into the
>> > firmware device.
>> > 
>> > If a flash memory is available, it is unsafe for the page
>> > tables to be stored in memory since the processor may try
>> > to write to the page table data structures.
>> > 
>> > Additionally, when KVM ROM support is enabled for the
>> > firmware devi
ce, then PEI fails to boot when the page
>> > tables are in the firmware device.
>> 
>> https://build.opensuse.org/package/show/Virtualization/OVMF
>> is at r14547 and that one works for me.
>> 
>> Gary/Ludwig, can you confirm that this is resolved?
>>
> I still got the black screen even with OVMF r14547 or r14608. I also
> tried kernel 3.10.9 and 3.11-rc7, but no luck.
> 
> Reverting the KVM_MEM_READONLY patch works for me, but the OVMF patch
> somehow didn't work.
> 
> Thanks,
> 
> Gary Lin

I tried this out, and I get the black screen as well when ept=n, but it boots
successfully to the efi shell when ept=y.

Gary, what does 'cat /sys/module/kvm_intel/parameters/ept' report on your
failing machine?

Bruce

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Gary Ching-Pang Lin]

On Wed, Aug 28, 2013 at 02:55:26PM +0200, Andreas Färber wrote:
> Am 28.08.2013 14:10, schrieb Laszlo Ersek:
> > On 08/28/13 13:49, Andreas Färber wrote:
> >> Am 28.08.2013 13:45, schrieb Laszlo Ersek:
> >>> (qemu-devel CC'd)
> >>>
> >>> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
>  Hi,
> 
>  I recently updated qemu to 1.6.0 and found OVMF just showed a blank
>  screen when kvm was enabled. I tried to dump OVMF log with the
>  following commond but nothing was stored in debug.log.
> 
>  qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon file:debug.log 
>  -global isa-debugcon.iobase=0x402
> 
>  The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
>  and uploaded to the following link:
>  https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing
> 
>  I found a similar case with kernel < 3.9, but I already upgraded linux
>  kernel to 3.10.5, so this may be another bug.
> >>>
> >>> Well, the usual first response in cases like this is...
> >>>
> >>> Can you bisect qemu? :)
> >>
> >> We had a similar report:
> >> https://bugzilla.novell.com/show_bug.cgi?id=835895
> > 
> > Well that's sorta the same report, considering you and Gary both work
> > for SUSE, and the Novell BZ seems to imply the build in question was Gary's:
> > 
> >> qemu 1.6.0 fails to run the tianocore firmware
> >> (home:gary_lin:UEFI/OVMF) properly. This worked with previous qemu
> > 
> >> versions:
> > 
> > :)
> 
> Different reporters, so who knows if the setups are the same. ;)
> 
> >> git-bisect said:
> >> 235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
> >> commit 235e8982ad393e5611cb892df54881c872eea9e1
> >> Author: Jordan Justen 
> >> Date:   Wed May 29 01:27:26 2013 -0700
> >>
> >> kvm: support using KVM_MEM_READONLY flag for regions
> >>
> >> For readonly memory regions and rom devices in romd_mode,
> >> we make use of the KVM_MEM_READONLY. A slot that uses
> >> KVM_MEM_READONLY can be read from and code can execute from the
> >> region, but writes will exit to qemu.
> >>
> >> For rom devices with !romd_mode, we force the slot to be
> >> removed so reads or writes to the region will exit to qemu.
> >> (Note that a memory region in this state is not executable
> >> within kvm.)
> >>
> >> v7:
> >>  * Update for readable => romd_mode rename (5f9a5ea1)
> >>
> >> Signed-off-by: Jordan Justen 
> >> Reviewed-by: Xiao Guangrong  (v4)
> >> Reviewed-by: Paolo Bonzini  (v5)
> >> Message-id: 1369816047-16384-4-git-send-email-jordan.l.jus...@intel.com
> >> Signed-off-by: Anthony Liguori 
> >>
> >>
> >> Any hints or patches welcome. :)
> > 
> > Hm. LP 1212402  probably
> > concerns the "similar case with kernel < 3.9" mentioned by Gary, and is
> > likely not revelant here.
> > 
> > 
> > Gary & Ludwig, can you confirm that your OVMF build includes SVN r14494?
> > 
> > Author: Jordan Justen 
> > Date:   Thu Jul 18 22:51:27 2013 +
> > 
> > OvmfPkg/Sec: Build identity mapped pages in RAM for X64
> > 
> > This is based on MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c.
> > 
> > Previously we would run using page tables built into the
> > firmware device.
> > 
> > If a flash memory is available, it is unsafe for the page
> > tables to be stored in memory since the processor may try
> > to write to the page table data structures.
> > 
> > Additionally, when KVM ROM support is enabled for the
> > firmware device, then PEI fails to boot when the page
> > tables are in the firmware device.
> 
> https://build.opensuse.org/package/show/Virtualization/OVMF
> is at r14547 and that one works for me.
> 
> Gary/Ludwig, can you confirm that this is resolved?
>
I still got the black screen even with OVMF r14547 or r14608. I also
tried kernel 3.10.9 and 3.11-rc7, but no luck.

Reverting the KVM_MEM_READONLY patch works for me, but the OVMF patch
somehow didn't work.

Thanks,

Gary Lin

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Ludwig Nussel]

Andreas Färber wrote:

https://build.opensuse.org/package/show/Virtualization/OVMF
is at r14547 and that one works for me.

Gary/Ludwig, can you confirm that this is resolved?


Yes, that version doesn't have the black screen with qemu 1.6.0
indeed and boots into the efi shell just fine, thanks!

cu
Ludwig

--
 (o_   Ludwig Nussel
 //\
 V_/_  http://www.suse.de/
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 
16746 (AG Nürnberg)

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Andreas Färber]

Am 28.08.2013 14:10, schrieb Laszlo Ersek:
> On 08/28/13 13:49, Andreas Färber wrote:
>> Am 28.08.2013 13:45, schrieb Laszlo Ersek:
>>> (qemu-devel CC'd)
>>>
>>> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
 Hi,

 I recently updated qemu to 1.6.0 and found OVMF just showed a blank
 screen when kvm was enabled. I tried to dump OVMF log with the
 following commond but nothing was stored in debug.log.

 qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon file:debug.log 
 -global isa-debugcon.iobase=0x402

 The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
 and uploaded to the following link:
 https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing

 I found a similar case with kernel < 3.9, but I already upgraded linux
 kernel to 3.10.5, so this may be another bug.
>>>
>>> Well, the usual first response in cases like this is...
>>>
>>> Can you bisect qemu? :)
>>
>> We had a similar report:
>> https://bugzilla.novell.com/show_bug.cgi?id=835895
> 
> Well that's sorta the same report, considering you and Gary both work
> for SUSE, and the Novell BZ seems to imply the build in question was Gary's:
> 
>> qemu 1.6.0 fails to run the tianocore firmware
>> (home:gary_lin:UEFI/OVMF) properly. This worked with previous qemu
> 
>> versions:
> 
> :)

Different reporters, so who knows if the setups are the same. ;)

>> git-bisect said:
>> 235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
>> commit 235e8982ad393e5611cb892df54881c872eea9e1
>> Author: Jordan Justen 
>> Date:   Wed May 29 01:27:26 2013 -0700
>>
>> kvm: support using KVM_MEM_READONLY flag for regions
>>
>> For readonly memory regions and rom devices in romd_mode,
>> we make use of the KVM_MEM_READONLY. A slot that uses
>> KVM_MEM_READONLY can be read from and code can execute from the
>> region, but writes will exit to qemu.
>>
>> For rom devices with !romd_mode, we force the slot to be
>> removed so reads or writes to the region will exit to qemu.
>> (Note that a memory region in this state is not executable
>> within kvm.)
>>
>> v7:
>>  * Update for readable => romd_mode rename (5f9a5ea1)
>>
>> Signed-off-by: Jordan Justen 
>> Reviewed-by: Xiao Guangrong  (v4)
>> Reviewed-by: Paolo Bonzini  (v5)
>> Message-id: 1369816047-16384-4-git-send-email-jordan.l.jus...@intel.com
>> Signed-off-by: Anthony Liguori 
>>
>>
>> Any hints or patches welcome. :)
> 
> Hm. LP 1212402  probably
> concerns the "similar case with kernel < 3.9" mentioned by Gary, and is
> likely not revelant here.
> 
> 
> Gary & Ludwig, can you confirm that your OVMF build includes SVN r14494?
> 
> Author: Jordan Justen 
> Date:   Thu Jul 18 22:51:27 2013 +
> 
> OvmfPkg/Sec: Build identity mapped pages in RAM for X64
> 
> This is based on MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c.
> 
> Previously we would run using page tables built into the
> firmware device.
> 
> If a flash memory is available, it is unsafe for the page
> tables to be stored in memory since the processor may try
> to write to the page table data structures.
> 
> Additionally, when KVM ROM support is enabled for the
> firmware device, then PEI fails to boot when the page
> tables are in the firmware device.

https://build.opensuse.org/package/show/Virtualization/OVMF
is at r14547 and that one works for me.

Gary/Ludwig, can you confirm that this is resolved?

Thanks,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Laszlo Ersek]

On 08/28/13 13:49, Andreas Färber wrote:
> Am 28.08.2013 13:45, schrieb Laszlo Ersek:
>> (qemu-devel CC'd)
>>
>> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
>>> Hi,
>>>
>>> I recently updated qemu to 1.6.0 and found OVMF just showed a blank
>>> screen when kvm was enabled. I tried to dump OVMF log with the
>>> following commond but nothing was stored in debug.log.
>>>
>>> qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon file:debug.log 
>>> -global isa-debugcon.iobase=0x402
>>>
>>> The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
>>> and uploaded to the following link:
>>> https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing
>>>
>>> I found a similar case with kernel < 3.9, but I already upgraded linux
>>> kernel to 3.10.5, so this may be another bug.
>>
>> Well, the usual first response in cases like this is...
>>
>> Can you bisect qemu? :)
> 
> We had a similar report:
> https://bugzilla.novell.com/show_bug.cgi?id=835895

Well that's sorta the same report, considering you and Gary both work
for SUSE, and the Novell BZ seems to imply the build in question was Gary's:

> qemu 1.6.0 fails to run the tianocore firmware
> (home:gary_lin:UEFI/OVMF) properly. This worked with previous qemu

> versions:

:)

> 
> git-bisect said:
> 235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
> commit 235e8982ad393e5611cb892df54881c872eea9e1
> Author: Jordan Justen 
> Date:   Wed May 29 01:27:26 2013 -0700
> 
> kvm: support using KVM_MEM_READONLY flag for regions
> 
> For readonly memory regions and rom devices in romd_mode,
> we make use of the KVM_MEM_READONLY. A slot that uses
> KVM_MEM_READONLY can be read from and code can execute from the
> region, but writes will exit to qemu.
> 
> For rom devices with !romd_mode, we force the slot to be
> removed so reads or writes to the region will exit to qemu.
> (Note that a memory region in this state is not executable
> within kvm.)
> 
> v7:
>  * Update for readable => romd_mode rename (5f9a5ea1)
> 
> Signed-off-by: Jordan Justen 
> Reviewed-by: Xiao Guangrong  (v4)
> Reviewed-by: Paolo Bonzini  (v5)
> Message-id: 1369816047-16384-4-git-send-email-jordan.l.jus...@intel.com
> Signed-off-by: Anthony Liguori 
> 
> 
> Any hints or patches welcome. :)

Hm. LP 1212402  probably
concerns the "similar case with kernel < 3.9" mentioned by Gary, and is
likely not revelant here.


Gary & Ludwig, can you confirm that your OVMF build includes SVN r14494?

Author: Jordan Justen 
Date:   Thu Jul 18 22:51:27 2013 +

OvmfPkg/Sec: Build identity mapped pages in RAM for X64

This is based on MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c.

Previously we would run using page tables built into the
firmware device.

If a flash memory is available, it is unsafe for the page
tables to be stored in memory since the processor may try
to write to the page table data structures.

Additionally, when KVM ROM support is enabled for the
firmware device, then PEI fails to boot when the page
tables are in the firmware device.

Thanks
Laszlo

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Andreas Färber]

Am 28.08.2013 13:45, schrieb Laszlo Ersek:
> (qemu-devel CC'd)
> 
> On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
>> Hi,
>>
>> I recently updated qemu to 1.6.0 and found OVMF just showed a blank
>> screen when kvm was enabled. I tried to dump OVMF log with the
>> following commond but nothing was stored in debug.log.
>>
>> qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon file:debug.log 
>> -global isa-debugcon.iobase=0x402
>>
>> The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
>> and uploaded to the following link:
>> https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing
>>
>> I found a similar case with kernel < 3.9, but I already upgraded linux
>> kernel to 3.10.5, so this may be another bug.
> 
> Well, the usual first response in cases like this is...
> 
> Can you bisect qemu? :)

We had a similar report:
https://bugzilla.novell.com/show_bug.cgi?id=835895

git-bisect said:
235e8982ad393e5611cb892df54881c872eea9e1 is the first bad commit
commit 235e8982ad393e5611cb892df54881c872eea9e1
Author: Jordan Justen 
Date:   Wed May 29 01:27:26 2013 -0700

kvm: support using KVM_MEM_READONLY flag for regions

For readonly memory regions and rom devices in romd_mode,
we make use of the KVM_MEM_READONLY. A slot that uses
KVM_MEM_READONLY can be read from and code can execute from the
region, but writes will exit to qemu.

For rom devices with !romd_mode, we force the slot to be
removed so reads or writes to the region will exit to qemu.
(Note that a memory region in this state is not executable
within kvm.)

v7:
 * Update for readable => romd_mode rename (5f9a5ea1)

Signed-off-by: Jordan Justen 
Reviewed-by: Xiao Guangrong  (v4)
Reviewed-by: Paolo Bonzini  (v5)
Message-id: 1369816047-16384-4-git-send-email-jordan.l.jus...@intel.com
Signed-off-by: Anthony Liguori 


Any hints or patches welcome. :)

Regards,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [edk2] OVMF hung on qemu 1.6.0 with KVM

[Laszlo Ersek]

(qemu-devel CC'd)

On 08/28/13 12:35, Gary Ching-Pang Lin wrote:
> Hi,
> 
> I recently updated qemu to 1.6.0 and found OVMF just showed a blank
> screen when kvm was enabled. I tried to dump OVMF log with the
> following commond but nothing was stored in debug.log.
> 
> qemu-system-x86_64 -s -enable-kvm -bios OVMF.fd -debugcon file:debug.log 
> -global isa-debugcon.iobase=0x402
> 
> The kvm trace was recorded with "trace-cmd record -b 2 -e kvm"
> and uploaded to the following link:
> https://docs.google.com/file/d/0B9hbtlc_aK_gcGh2TDZLUVlzWWc/edit?usp=sharing
> 
> I found a similar case with kernel < 3.9, but I already upgraded linux
> kernel to 3.10.5, so this may be another bug.

Well, the usual first response in cases like this is...

Can you bisect qemu? :)

Laszlo