Re: GPU Pass-through need help.
Prasad Joshi wrote: I was previously seeing a problem with the ROM BIOS, but I could solve it by passing a correct BIOS file in function assigned_initfn. For now I am using hardcoded file name in the code. Did you recheck video bios inside guest? (for example in sysfs) If this is bootstrap vga, guest linux kernel will use bios copy placed at 0xc. But in my setup qemu can only pass first 32k of option-roms, due to curruption at seabios shadow-copying stage. (seems like because improper i440fx PAM registers emulation) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: GPU Pass-through need help.
On Wed, Feb 9, 2011 at 4:33 PM, Konstantin Khlebnikov khlebni...@openvz.org wrote: Prasad Joshi wrote: I was previously seeing a problem with the ROM BIOS, but I could solve it by passing a correct BIOS file in function assigned_initfn. For now I am using hardcoded file name in the code. Did you recheck video bios inside guest? (for example in sysfs) For GPU cards on my machine, reading from the /sys/bus/pci/devices/device/ rom always give error 'Invalid Argument'. I had to extract the BIOS using some utilities. The BIOS seems to be correct as I have been able to pass-through a old Nvidia (NVS 290) and old ATI (RV370 FireGL V3100) card to Ubuntu Guest OS. If this is bootstrap vga, guest linux kernel will use bios copy placed at 0xc. But in my setup qemu can only pass first 32k of option-roms, due to curruption at seabios shadow-copying stage. (seems like because improper i440fx PAM registers emulation) I am not sure if it is related to the problem you are facing. The current SeaBIOS has very limited pci memory space from 0xf000 to 0xfec0 (236MB). This includes everything, prefectable and non-prefecthable memory. The range is very small for VGA cards to work. I have modified the memory range on my machine for prefectable memory. There is one bug in qemu-kvm code, which acidently detects the prefectable memory as non-prefectable. I had sent a patch to correct this behavior yesterday. Hope it gets accepted. Thanks and Regards, Prasad -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: GPU Pass-through need help.
Prasad Joshi wrote: On Wed, Feb 9, 2011 at 4:33 PM, Konstantin Khlebnikov khlebni...@openvz.org mailto:khlebni...@openvz.org wrote: Prasad Joshi wrote: I was previously seeing a problem with the ROM BIOS, but I could solve it by passing a correct BIOS file in function assigned_initfn. For now I am using hardcoded file name in the code. Did you recheck video bios inside guest? (for example in sysfs) For GPU cards on my machine, reading from the /sys/bus/pci/devices/device/rom always give error 'Invalid Argument'. Yep, rom in sysfs must be enabled before via 'echo 1 rom' If rom was corrupted you will see warning in dmesg. I had to extract the BIOS using some utilities. The BIOS seems to be correct as I have been able to pass-through a old Nvidia (NVS 290) and old ATI (RV370 FireGL V3100) card to Ubuntu Guest OS. If this is bootstrap vga, guest linux kernel will use bios copy placed at 0xc. But in my setup qemu can only pass first 32k of option-roms, due to curruption at seabios shadow-copying stage. (seems like because improper i440fx PAM registers emulation) I am not sure if it is related to the problem you are facing. The current SeaBIOS has very limited pci memory space from 0xf000 to 0xfec0 (236MB). This includes everything, prefectable and non-prefecthable memory. The range is very small for VGA cards to work. I have modified the memory range on my machine for prefectable memory. But if in your guest /sys/bus/pci/devices/device/boot_vga == 1 linux will use shadow copy from 0xc instead of pci rom bar There is one bug in qemu-kvm code, which acidently detects the prefectable memory as non-prefectable. I had sent a patch to correct this behavior yesterday. Hope it gets accepted. Yes, I saw him, IORESOURCE_PREFETCH was accidentally changed in linux kernel. Maybe better to detect it via opening and mapping /resorceid_wc instad of /resourceid, it exist only for prefechable resources. Plus qemu will get write-combined mapping. Thanks and Regards, Prasad -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
GPU Pass-through need help.
Hello All,
I am trying to pass-through a GPU PCIe ATI Radeon to VM. Here is a log on VM.
pra...@prasad-virtual-machine:~$ dmesg | grep -i -e drm -e radeon
[2.162294] [drm] Initialized drm 1.1.0 20060810
[2.459594] [drm] radeon defaulting to kernel modesetting.
[2.459596] [drm] radeon kernel modesetting enabled.
[2.766698] radeon :00:04.0: PCI INT A - Link[LNKD] - GSI 10
(level, high) - IRQ 10
[2.766734] radeon :00:04.0: setting latency timer to 64
[2.783512] [drm] initializing kernel modesetting (RV380 0x1002:0x5B64).
[2.792407] [drm] register mmio base: 0x4000
[2.792408] [drm] register mmio size: 65536
[2.797177] [drm] Generation 2 PCI interface, using max accessible memory
[2.797275] radeon :00:04.0: VRAM: 128M 0xF800 -
0x (128M used)
[2.797284] radeon :00:04.0: GTT: 512M 0xD800 -
0xF7FF
[2.798370] radeon :00:04.0: irq 40 for MSI/MSI-X
[2.870703] radeon :00:04.0: radeon: using MSI.
[3.151162] [drm] radeon: irq initialized.
[3.151539] [drm] Detected VRAM RAM=128M, BAR=128M
[3.151541] [drm] RAM width 128bits DDR
[3.151610] [drm] radeon: 128M of VRAM memory ready
[3.151611] [drm] radeon: 512M of GTT memory ready.
[3.151627] [drm] GART: num cpu pages 131072, num gpu pages 131072
[3.152701] [drm] radeon: 1 quad pipes, 1 Z pipes initialized.
[3.544943] [drm] PCIE GART of 512M enabled (table at 0xF804).
[3.548479] radeon :00:04.0: WB enabled
[3.549020] [drm] Loading R300 Microcode
[3.554278] [drm] radeon: ring at 0xD8001000
[3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
(sracth(0x15E4)=0xCAFEDEAD)
[3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
[3.782542] radeon :00:04.0: failled initializing CP (-22).
[3.784006] radeon :00:04.0: Disabling GPU acceleration
[3.793709] [drm] radeon: cp finalized
[3.834875] radeon :00:04.0: 8800320b4c00 unpin not necessary
[3.841759] [drm] Radeon Display Connectors
[3.841766] [drm] Connector 0:
[3.841771] [drm] VGA
[3.841777] [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60
[3.841781] [drm] Encoders:
[3.841785] [drm] CRT1: INTERNAL_DAC1
[3.841788] [drm] Connector 1:
[3.841791] [drm] DVI-I
[3.841794] [drm] HPD1
[3.841799] [drm] DDC: 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64
[3.841803] [drm] Encoders:
[3.841806] [drm] CRT2: INTERNAL_DAC2
[3.841809] [drm] DFP1: INTERNAL_TMDS1
[6.158753] [drm] fb mappable at 0xF804
[6.158756] [drm] vram apper at 0xF800
[6.158757] [drm] size 5242880
[6.158758] [drm] fb depth is 24
[6.158759] [drm]pitch is 5120
[ 13.914196] fb0: radeondrmfb frame buffer device
[ 13.914198] drm: registered panic notifier
[ 13.914683] [drm] Initialized radeon 2.7.0 20080528 for
:00:04.0 on minor 0
I was previously seeing a problem with the ROM BIOS, but I could solve
it by passing a correct BIOS file in function assigned_initfn. For now
I am using hardcoded file name in the code.
Can anyone help me with debugging of following errors?
[3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
(sracth(0x15E4)=0xCAFEDEAD)
[3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
3444 WREG32(scratch, 0xCAFEDEAD);
3445 r = radeon_ring_lock(rdev, 2);
3446 if (r) {
3447 DRM_ERROR(radeon: cp failed to lock ring (%d).\n, r);
3448 radeon_scratch_free(rdev, scratch);
3449 return r;
3450 }
3451 radeon_ring_write(rdev, PACKET0(scratch, 0));
3452 radeon_ring_write(rdev, 0xDEADBEEF);
3453 radeon_ring_unlock_commit(rdev);
3454 for (i = 0; i rdev-usec_timeout; i++) {
3455 tmp = RREG32(scratch);
3456 if (tmp == 0xDEADBEEF) {
3457 break;
3458 }
3459 DRM_UDELAY(1);
3460 }
3461 if (i rdev-usec_timeout) {
3462 DRM_INFO(ring test succeeded in %d usecs\n, i);
3463 } else {
3464 DRM_ERROR(radeon: ring test failed (sracth(0x%04X)=0x%08X)\n,
3465 scratch, tmp);
3466 r = -EINVAL;
3467 }
It seems like, on line 3452 code writes DEADBEEF ins some ring and on
line 3455-3456 it reads whatever has been written on ring and it fails
in this case as it reads back 0xCAFEDEAD
How can I debug this VM problem?
BTW, the assigned device is listed in lspci output
pra...@prasad-virtual-machine:~$ lspci | grep ATI
00:04.0 Display controller: ATI Technologies Inc RV370 5B64 [FireGL
V3100 (PCIE)] (rev 80)
I also noticed following log on the host machine.
[ 6042.798880] kvm: 1566: cpu0 unimplemented perfctr wrmsr:
0xc0010004data 0xabcd
[ 6104.970165] pci-stub :02:00.0: restoring config space at
offset0x1 (was 0x100400, writing 0x17)
[ 6105.530290] assign device 0:2:0.0
[ 6105.530351] pci-stub :02:00.0: irq 88 for MSI/MSI-X
[
