Re: Slow TCP performance from Win2k8R2 guests under Linux KVM

2013-07-22 Thread Gordon Messmer

On 07/21/2013 03:32 AM, Yan Vugenfirer wrote:

Can you provide results with iperf or netperf?


Actually, iperf seems to operate at expected speeds, while some other 
applications don't.  I'm working on getting a list of some of the 
affected programs.  I believe my users first reported the problem with 
ftp.  I'll follow up as soon as possible.


Thanks for the suggestion.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Slow TCP performance from Win2k8R2 guests under Linux KVM

2013-07-15 Thread Gordon Messmer
Would anyone be so kind as to test the performance of TCP streams on 
Windows 2k8R2 guests of KVM virt servers?


Red Hat's virtio-win-1.6.3 package was published to address a problem 
described as "low performance" on Win2k8 guests.  Actual performance was 
not recorded in the bug report.


On my KVM servers, Windows 2k8R2 guests are able to send data at less 
than 4Mbps.  I test this by using netcat on Windows and piping a file to 
its input.  The receiving end of the transfer is an un-virtualized 
CentOS system.  Transfer rate is measured by selecting the stream in 
iptraf while it is active.


UDP transfers, such as by CIFS, will send data at around 400Mbps.

TCP transfers from Windows 2012 or Windows 8 are not similarly affected.

The same driver for Win2k8 was published by Red Hat in virtio-win 1.6.3, 
1.6.4, and 1.6.5.


Does anyone see TCP transfer rates from Win2k8R2 that don't look 
ridiculously slow?


https://bugzilla.redhat.com/show_bug.cgi?id=859882
http://rhn.redhat.com/errata/RHBA-2013-0441.html
http://joncraton.org/blog/46/netcat-for-windows

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Bad ROM contents on pass-through card

2012-10-07 Thread Gordon Messmer

On 10/07/2012 06:43 AM, Alex Williamson wrote:

On Sat, 2012-10-06 at 18:23 -0700, Gordon Messmer wrote:


If I don't pass an emulated VGA device (and modify seabios to not run
the VGA ROM), the guess won't read anything from the rom at all.

Can anyone point me at which parts of the code I might review to see
where things are getting mapped and what's going wrong?


The comments in linux:/drivers/pci/rom.c:pci_map_rom are probably
relevant.  Linux decides based on the VGA enable bit whether to return
the PCI option ROM or legacy VGA ROM.  In your case, it's giving you the
legacy ROM from the cirrus device.


Interesting.  There are two things that seem odd about that.  First, the 
Linux guest doesn't seem to read the correct ROM either.  During boot, 
dmesg prints:

[1.047559] [drm] radeon defaulting to kernel modesetting.
[1.047561] [drm] radeon kernel modesetting enabled.
[1.047595] checking generic (f000 16) vs hw (e000 1000)
[1.047832] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11
[1.048270] [drm] initializing kernel modesetting (RV620 
0x1002:0x95C0 0x1028:0x3243).

[1.048288] [drm] register mmio base: 0xF203
[1.048289] [drm] register mmio size: 65536
[1.048395] radeon :00:05.0: Expecting atombios for R600 GPU
[1.048924] radeon :00:05.0: Fatal error during GPU init
[1.048925] [drm] radeon: finishing device.
[1.048927] [TTM] Memory type 2 has not been initialized

Second, if I don't provide a -vga argument, the guest will hang during 
boot.  If I modify seabios not to run the VGA ROM, the guest boots but 
the radeon driver output is the same and I can't read any ROM at all 
from /sys



The next step is probably looking at
generating a PCI topology in the guest similar to a physical system when
multiple VGA devices are present.  That means bridges and/or root ports
with VGA routing control.


Can you point me toward the documentation?  I don't see anything that 
looks relevant in the man page.



What's the size of Radeon3470-5.rom?  When using romfile the size of the
option ROM BAR is made to be big enough to contain the ROM.  Without
that, we match the physical device and expose the BAR as potentially
being bigger than the ROM it contains.  Thanks,


It's 64512 bytes.

Thanks, Alex.  I really appreciate your time.


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Bad ROM contents on pass-through card

2012-10-06 Thread Gordon Messmer
I started trying to pass through a PCI-E card to a KVM guest early this 
year.  Other hardware can be passed through successfully, but the guest 
sees the wrong option ROM for this card.


My system is an AMD CPU and GA-970A-D3 motherboard.  There are currently 
two Radeon HD 3470 cards installed, partially so that I could read their 
ROMs into a file.


The host OS is CentOS 6.3, and I've updated seabios and qemu to current 
versions.


My guest is started with:

/usr/local/bin/qemu-system-x86_64 ... -vga cirrus \
-device pci-assign,host=05:00.0,id=hostdev1,configfd=27,\
bus=pci.0,addr=0x5,romfile=/var/lib/libvirt/images/Radeon3470-5.rom

However, when the guest tries to read the ROM for the ATI card, it's 
seeing the ROM for the emulated cirrus card instead:


# strings /sys/devices/pci:00/:00:05.0/rom | head -4
Plex86/Bochs VGABios (PCI)
current-cvs 23 Sep 2012
(C) 2008 the LGPL VGABios developers Team

If I don't pass an emulated VGA device (and modify seabios to not run 
the VGA ROM), the guess won't read anything from the rom at all.


Can anyone point me at which parts of the code I might review to see 
where things are getting mapped and what's going wrong?




On the host system, lspci describes the device:

05:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI 
RV620 PRO [Radeon HD 3470] (prog-if 00 [VGA controller])

Subsystem: Dell Device 3243
Flags: fast devsel, IRQ 26
Memory at c000 (64-bit, prefetchable) [size=256M]
Memory at fd9f (64-bit, non-prefetchable) [size=64K]
I/O ports at ae00 [size=256]
Expansion ROM at fd90 [disabled] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information 
Kernel driver in use: pci-stub
Kernel modules: radeon

and in the guest (Fedora 17):

00:05.0 VGA compatible controller: ATI Technologies Inc Mobility Radeon 
HD 3470 (prog-if 00 [VGA controller])

Subsystem: Dell Device 3243
Physical Slot: 5
Flags: fast devsel, IRQ 11
Memory at e000 (32-bit, non-prefetchable) [size=256M]
Memory at f203 (32-bit, non-prefetchable) [size=64K]
I/O ports at c100 [size=256]
Expansion ROM at f204 [disabled] [size=64K]
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [50] Power Management version 3
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit-

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Windows XP + Virtio

2012-05-19 Thread Gordon Messmer

On 05/01/2012 04:33 PM, Sean Kennedy wrote:

I am getting crashes (BSoD) when using Virtio for the disk driver in Windows XP.


I was getting the same thing on Windows Server 2008 guests.  The systems 
would crash and some of them corrupted their filesystems pretty badly.  
I ended up reverting to running them with IDE emulation so that the 
virtio block driver wasn't used.


The odd thing was that I benchmarked disk IO using iozone and found that 
there was no consistent behavior difference between virtio and emulated 
IDE.  In fact, emulated IDE sometimes performed better than virtio.


Who else is benchmarking the virtio block devices to see what 
performance benefits they provide?  What's the best way for users to 
measure the difference between the two block IO systems?


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Device pass-through

2012-01-15 Thread Gordon Messmer

I still poke at this when I get time.

I notice that pci_update_mappings calls pci_bar_address, and the latter 
will always return PCI_BAR_UNMAPPED for this device.  For the emulated 
devices, pci_bar_address will get an address with PCI_ROM_ADDRESS_ENABLE 
at some point, and pci_map_option_rom will be called.  With the card 
that I'm passing in, that never happens.  Could anyone point me in the 
direction I should go from here?  What sets that bit for the emulated 
devices that might not happen for the PCI card?



if (type & PCI_BASE_ADDRESS_MEM_TYPE_64) {
new_addr = pci_get_quad(d->config + bar);
} else {
new_addr = pci_get_long(d->config + bar);
}
/* the ROM slot has a specific enable bit */
if (reg == PCI_ROM_SLOT && !(new_addr & PCI_ROM_ADDRESS_ENABLE)) {
return PCI_BAR_UNMAPPED;
}
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Device pass-through

2012-01-08 Thread Gordon Messmer

On 01/07/2012 02:21 PM, Gordon Messmer wrote:

I'm certain that qemu-kvm loads both
/usr/share/qemu-kvm/vgabios-cirrus.bin and
/var/lib/libvirt/images/Radeon3470.rom, which seems correct.  However,
in the guest, both the virtual VGA card and the real PCI one have the
same ROM, from vgabios-cirrus.bin.


I've learned slightly more.  First, I built a version of seabios that 
never calls vga_setup.


Once I disabled vga_setup, I was able to boot my guest with no emulated 
VGA device.  Now, the PCI device passed through no longer has the Cirrus 
ROM.  Instead it has the etherboot rom attached to the virtio ethernet 
device.  When I remove that device and boot the guest, the VGA card has 
no ROM at all.


I get the same results on qemu-kvm-0.12.1.2 from CentOS 6.2 and with 
qemu-0.15.0 from Fedora 16, rebuilt for CentOS.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Device pass-through

2012-01-07 Thread Gordon Messmer

On 01/06/2012 06:07 PM, Gordon Messmer wrote:


/usr/libexec/qemu-kvm\
-vnc 127.0.0.1:0 -vga cirrus \
-device
pci-assign,host=05:00.0,id=hostdev1,configfd=27,bus=pci.0,addr=0x7,romfile=/var/lib/libvirt/images/Radeon3470.rom


Snipped a bunch of other args...  I'm certain that qemu-kvm loads both 
/usr/share/qemu-kvm/vgabios-cirrus.bin and 
/var/lib/libvirt/images/Radeon3470.rom, which seems correct.  However, 
in the guest, both the virtual VGA card and the real PCI one have the 
same ROM, from vgabios-cirrus.bin.


I've looked through the qemu-kvm source and don't see anything that 
would obviously cause that sort of problem.  Should I be looking into 
seabios?


00:02.0 0300: 1013:00b8 (prog-if 00 [VGA controller])
Subsystem: 1af4:1100
Physical Slot: 2
Flags: fast devsel
Memory at f000 (32-bit, prefetchable) [size=32M]
Memory at f200 (32-bit, non-prefetchable) [size=4K]
Expansion ROM at f201 [disabled] [size=64K]
Kernel modules: cirrusfb

00:07.0 0300: 1002:95c0 (prog-if 00 [VGA controller])
Subsystem: 1028:3243
Physical Slot: 7
Flags: fast devsel, IRQ 11
Memory at e000 (32-bit, prefetchable) [size=256M]
Memory at f205 (32-bit, non-prefetchable) [size=64K]
I/O ports at c100 [size=256]
Expansion ROM at f206 [disabled] [size=64K]
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [50] Power Management version 3
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit-
Kernel modules: radeon


# strings /sys/bus/pci/devices/\:00\:02.0/rom | head -2
Plex86/Bochs VGABios (PCI)
current-cvs 19 Jul 2011
# strings /sys/bus/pci/devices/\:00\:07.0/rom | head -2
Plex86/Bochs VGABios (PCI)
current-cvs 19 Jul 2011
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Device pass-through

2012-01-06 Thread Gordon Messmer

On 01/06/2012 01:31 AM, André Weidemann wrote:

On 06.01.2012 08:25, Gordon Messmer wrote:

Well, I finally figured out that I have to enable the reading of roms
from the device by writing "1" to the "rom" node in /sys/. Now the
problem is that the rom is 64k, and only 32k are making it into the
guest. I saw a reference to this problem here:


Do you have the ROM as a file on your HDD perhaps? If so, you can try
the following: -device pci-assign,host=05:00.0,romfile=${ROMFILE}.
This used to work fine for me even with ROM file of around 130k.


I do, and I'm actually using that.  I'm using a shell script wrapper to 
"fix" the argument for the PCI video card, since libvirt XML doesn't 
seem to provide an option for rom files.  The command that gets run is 
included below.


I've learned that I was actually incorrect about the problem.  The card 
in the guest *does* have a 32K rom, but it's not a truncated version of 
the ATI rom.  Instead, it's the "Plex86/Bochs VGABios (PCI)."  I've 
tried starting the guest with no graphics or video hardware specified in 
the libvirt XML, but qemu-kvm just eats a bunch of CPU time.  I don't 
see any serial console output (I do see grub and kernel output on the 
serial console when I have virtual video hardware specified), so I'm 
pretty sure the guest isn't actually starting.




/usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 4096 -smp 
4,sockets=4,cores=1,threads=1 -name herald -uuid 
556638be-ee50-e2f6-1d22-37b98b63b8d1 -nodefconfig -nodefaults -chardev 
socket,id=charmonitor,path=/var/lib/libvirt/qemu/herald.monitor,server,nowait 
-mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -drive 
file=/dev/mapper/VolGroup-lv_vm_herald,if=none,id=drive-virtio-disk0,format=raw,cache=none,aio=native 
-device 
virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 
-device 
virtio-net-pci,vlan=0,id=net0,mac=52:54:00:21:e7:9e,bus=pci.0,addr=0x3 
-net tap,fd=25,vlan=0,name=hostnet0 -chardev pty,id=charserial0 -device 
isa-serial,chardev=charserial0,id=serial0 -usb -device 
usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device 
pci-assign,host=00:14.2,id=hostdev0,configfd=26,bus=pci.0,addr=0x5 
-device 
pci-assign,host=05:00.0,id=hostdev1,configfd=27,bus=pci.0,addr=0x7,romfile=/var/lib/libvirt/images/Radeon3470.rom 
-device usb-host,hostbus=5,hostaddr=3,id=hostdev2 -device 
usb-host,hostbus=7,hostaddr=3,id=hostdev3 -device 
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Device pass-through

2012-01-05 Thread Gordon Messmer

On 01/05/2012 11:07 AM, Gordon Messmer wrote:

I started with an update to seabios, from the bundled version
0.6.1.2-8.el6 to a rebuilt package from F16, 0.6.2-3.el6. That's enough
to get the guest to boot with the pass-through video card. It doesn't
work, currently, and I'm pretty sure that's because I can't read the ROM
from the card. I'm going to look around for a solution to that and will
update again.


Well, I finally figured out that I have to enable the reading of roms 
from the device by writing "1" to the "rom" node in /sys/.  Now the 
problem is that the rom is 64k, and only 32k are making it into the 
guest.  I saw a reference to this problem here:

http://www.spinics.net/lists/kvm/msg49946.html

I've updated seabios to 0.6.3, and I've rebuilt the qemu-kvm packages 
from F16 on CentOS 6.2 to try those.  It took a while, but didn't change 
the results.  I'm still only able to read 32k of rom from the node in 
/sys/ in the guest.


Perhaps I'd have better luck with a different card.

In the meantime, thanks for your advice.  I think I'm close, and I'll 
keep working on the project.  If I get it working, I'll send along 
further details with the steps I had to take.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Device pass-through

2012-01-05 Thread Gordon Messmer

On 01/03/2012 07:44 PM, Alex Williamson wrote:

Yep, that's what I would have guessed, there's a 256MB resource.  I'm
not sure if the seabios you're using is mapping the MMIO hole
efficiently enough to handle that.  Can you test on new upstream
qemu-kvm?  Thanks,


I'm not done poking the system, but I thought I'd send an update.

I started with an update to seabios, from the bundled version 
0.6.1.2-8.el6 to a rebuilt package from F16, 0.6.2-3.el6.  That's enough 
to get the guest to boot with the pass-through video card.  It doesn't 
work, currently, and I'm pretty sure that's because I can't read the ROM 
from the card.  I'm going to look around for a solution to that and will 
update again.


lspci in the guest shows the device:

00:07.0 VGA compatible controller: ATI Technologies Inc Mobility Radeon
HD 3470 (prog-if 00 [VGA controller])
Subsystem: Dell Device 3243
Physical Slot: 7
Flags: fast devsel, IRQ 11
Memory at e000 (32-bit, prefetchable) [size=256M]
Memory at f205 (32-bit, non-prefetchable) [size=64K]
I/O ports at c100 [size=256]
Expansion ROM at  [disabled]
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [50] Power Management version 3
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit-
Kernel modules: radeon


dmesg reports some errors:

[1.048791] [drm] Initialized drm 1.1.0 20060810
[1.066131] [drm] radeon defaulting to kernel modesetting.
[1.066133] [drm] radeon kernel modesetting enabled.
[1.066220] radeon :00:07.0: PCI INT A -> Link[LNKC] ->
GSI 11 (level, high) -> IRQ 11
[1.066267] radeon :00:07.0: setting latency timer to 64
[1.066611] [drm] initializing kernel modesetting
(RV620 0x1002:0x95C0 0x1028:0x3243).
[1.066774] [drm] register mmio base: 0xF205
[1.066776] [drm] register mmio size: 65536
[1.066884] radeon :00:07.0: Expecting atombios for R600 GPU
[1.066886] radeon :00:07.0: Fatal error during GPU init
[1.066888] [drm] radeon: finishing device.
[1.066890] [TTM] Memory type 2 has not been initialized.
[1.067924] vga_switcheroo: disabled
[1.068070] radeon :00:07.0: PCI INT A disabled
[1.068075] radeon: probe of :00:07.0 failed with error -22



--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Device pass-through

2012-01-03 Thread Gordon Messmer

Thanks, Alex.  I really appreciate the reply.

On 01/03/2012 01:34 PM, Alex Williamson wrote:

This is actually the GART aperture, which Linux will try to use as an
IOMMU.  You can pretty much ignore this, but it may be wasting memory
for no good reason if it's really hiding usable memory and allocating a
bounce buffer area.


It does look that way, but I doubt I'll miss it.  Would you suggest that 
I boot with swiotlb=0 ?



The host is potentially exposed to a malicious guest trying to trigger
MSI interrupts for denial of service or exploit probing.  If you trust
your guest, it's safe to allow this.


I do.  It's a Fedora guest that I'm personally using.  Suppose I didn't. 
 My kernel was built with CONFIG_INTR_REMAP=y but the kernel said 
"kvm_iommu_map_guest: No interrupt remapping support, disallowing device 
assignment."  Does that mean that the motherboard doesn't support 
interrupt remapping?



4: What do these errors mean, and is there any way I can correct the
problem?

create_userspace_phys_mem: File exists
assigned_dev_iomem_map: Error: create new mapping failed


Might mean the resources for the graphics card are getting mapped to
overlap with guest memory.  What does lspci -v for the graphics card
show?  More recent changes upstream might make better use of the space
we have for devices, might be worth a test.


05:00.0 VGA compatible controller: ATI Technologies Inc RV620 PRO
[Radeon HD 3470] (prog-if 00 [VGA controller])
Subsystem: Dell Device 3243
Flags: fast devsel, IRQ 17
Memory at c000 (64-bit, prefetchable) [disabled] [size=256M]
Memory at fd9f (64-bit, non-prefetchable)
[disabled] [size=64K]
I/O ports at ae00 [disabled] [size=256]
Expansion ROM at fd90 [disabled by cmd] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information 
Kernel modules: radeon


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Device pass-through

2012-01-02 Thread Gordon Messmer
I'm going to summarize my questions first and write the long explanation 
afterward...


1: What happens when Linux rejects the aperture allocated by the BIOS 
for an AMD IOMMU?
2: What is unsafe about the allow_unsafe_assigned_interrupts option? 
What are the potential consequences of its use?

3: How can I get the ROM from a graphics card?
4: What do these errors mean, and is there any way I can correct the 
problem?

create_userspace_phys_mem: File exists
assigned_dev_iomem_map: Error: create new mapping failed


My goal is to replace two machines that I previously used with one 
machine and a KVM guest.  I previously had one system that was always on 
and ran a stable OS, and a second system for my desktop.  I could afford 
to experiment or format the desktop system without interrupting access 
to my data, all of which is on the stable system.


I'd like to use KVM to accomplish something more or less equivalent by 
creating a guest and passing the keyboard, mouse, audio device, and a 
video card to it.  To this end, I have a new AMD system with a Gigabyte 
GA-970A-D3 motherboard and two Radeon video cards.  Based on my reading, 
I should probably be able to pass one of those to the guest as a PCI 
device.  It isn't in use by the host, and I'm not trying to make it the 
guest's primary display, so I don't have to worry about early boot 
output.  The guest will still have a VNC console as its primary display. 
 I'm mostly there now.  I can pass through the host's keyboard, mouse, 
and audio device.


Naturally, there are a couple of challenges.  First, the IOMMU on this 
board is a little wonky.  The kernel currently reports:



kernel: Checking aperture...
kernel: No AGP bridge found
kernel: Node 0: aperture @ 2000 size 32 MB
kernel: Aperture pointing to e820 RAM. Ignoring.
kernel: Your BIOS doesn't leave a aperture memory hole
kernel: Please enable the IOMMU option in the BIOS setup
kernel: This costs you 64 MB of RAM
kernel: Mapping aperture over 65536 KB of RAM @ 2000
kernel: PM: Registered nosave memory: 2000 -
2400
kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
kernel: Placing 64MB software IO TLB between 880037ff -
88003bff
kernel: software IO TLB at phys 0x37ff - 0x3bff


additionally:


kernel: BIOS-provided physical RAM map:
kernel: BIOS-e820:  - 00098800 (usable)
kernel: BIOS-e820: 0009f800 - 000a (reserved)
kernel: BIOS-e820: 000f - 0010 (reserved)
kernel: BIOS-e820: 0010 - bfda (usable)
kernel: BIOS-e820: bfda - bfdd1000 (ACPI NVS)
kernel: BIOS-e820: bfdd1000 - bfe0 (ACPI data)
kernel: BIOS-e820: bfe0 - bff0 (reserved)
kernel: BIOS-e820: e000 - f000 (reserved)
kernel: BIOS-e820: fec0 - 0001 (reserved)
kernel: BIOS-e820: 0001 - 00044000 (usable)
...
kernel: ACPI: IVRS bfddff00 000D0
(v01  AMD RD890S 00202031 AMD  )


Initially the kernel reported "AMD-Vi disabled by default: pass 
amd_iommu=on to enable"  After booting with that option, I seem to be 
able to boot guests with devices passed through.  That leads me to the 
first question above.  How bad is the state of the IOMMU on this board? 
 Where should the IVRS be mapped instead?


In addition to the amd_iommu boot option, I had to add:
options kvm allow_unsafe_assigned_interrupts=1
in order to pass through the audio device.  That leads me to the second 
question.  What's unsafe about this?


When I try to boot the guest and pass in the video card, I get a warning 
about the ROM:



pci-assign: Cannot read from host /sys/bus/pci/devices/:05:00.0/rom
Device option ROM contents are probably invalid (check dmesg).
Skip option ROM probe with rombar=0, or load from file with
romfile=
create_userspace_phys_mem: File exists
assigned_dev_iomem_map: Error: create new mapping failed


Third question, when I try to read that path, I get an IO error and the 
kernel says "pci :05:00.0: Invalid ROM contents".  Is there any 
other way to get the ROM from the card?


I've tried the other option.  I added  to the libvirt 
definition for this guest.  When I try to boot the guest with that 
option, the kernel doesn't add anything to dmesg that looks relevant, 
but the libvirt log says:



LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none 
/usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 4096 -smp 
4,sockets=4,cores=1,threads=1 -name herald -uuid 
1929f6a4-b2dd-a4fd-fe07-747b7fdf9fae -nodefconfig -nodefaults -chardev 
socket,id=charmonitor,path=/var/lib/libvirt/qemu/herald.monitor,server,nowait 
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown 
-drive 
file=/dev/mapper/VolGroup-lv_vm_herald,if=none,id=drive-virtio-disk0,format=raw,cache=none,aio