[Xen-devel] xen does not support the 8G large bar

[Gaofeng (GaoFeng, Euler)]

Hi George,

I found that you have submitted a patch about "libxl,hvmloader: Don't relocate 
memory for MMIO hole". So, I have a "8G large bar PCI device(NVIDIA Tesla M60)" 
passthrough question to ask you.



Host passthrough PCI Device info:
lspci -vs 06:00.0
06:00.0 3D controller: NVIDIA Corporation Device 13f2 (rev a1)
        Subsystem: NVIDIA Corporation Device 115e
        Flags: fast devsel, IRQ 40
        Memory at 93000000 (32-bit, non-prefetchable) [disabled] [size=16M]
        Memory at 23c00000000 (64-bit, prefetchable) [disabled] [size=8G]
        Memory at 23e00000000 (64-bit, prefetchable) [disabled] [size=32M]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Endpoint, MSI 00
        Capabilities: [100] Virtual Channel
        Capabilities: [250] Latency Tolerance Reporting
        Capabilities: [258] #1e
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [420] Advanced Error Reporting
        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 
<?>
        Capabilities: [900] #19
        Kernel driver in use: pciback
        Kernel modules: nvidiafb

Guest passthrough PCI Device info:
lspci -vs 00:05.0
00:05.0 3D controller: NVIDIA Corporation Device 13f2 (rev a1)
        Subsystem: NVIDIA Corporation Device 115e
        Physical Slot: 5
        Flags: fast devsel, IRQ 36
        Memory at f5000000 (32-bit, non-prefetchable) [size=16M]
        Memory at 200000000 (64-bit, prefetchable) [size=4G]
        Memory at f2000000 (64-bit, prefetchable) [size=32M]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Endpoint, MSI 00
        Kernel modules: nouveau, nvidiafb


passthrough to the guest, the large bar size only has 4G. so, the passthrough 
pci device does not work, windows 7/2008 R2 64bit vm has the same problem.

.

According to the analysis hvmloader code, find a problem:



if (is_64bar) {

                bar_data_upper = pci_readl(devfn, bar_reg + 4);

                pci_writel(devfn, bar_reg + 4, ~0);

                bar_sz_upper = pci_readl(devfn, bar_reg + 4);

                pci_writel(devfn, bar_reg + 4, bar_data_upper);

                bar_sz = (bar_sz_upper << 32) | bar_sz;

            }

            bar_sz &= ~(bar_sz - 1);



read from the pci device, bar_sz_upper is 0xffffffff, if the bar size is 8G, 
the bar_sz_upper should be 0xfffffffe.

Have you ever encountered a similar problem?



Env info:

device_model_version: qemu-xen

xen version: xen-4.6.1

Guest: RedHat-6.4-64

Passthrough PCI Device: NVIDIA Tesla M60(compute mode)

CPU: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz



Thanks

Feng

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel