On Aug 9, 2011, at 5:13 AM, Avi Kivity wrote: > On 08/08/2011 10:18 PM, John Paul Walters wrote: >> On Jul 21, 2011, at 2:10 AM, Avi Kivity wrote: >> >> > On 07/21/2011 02:20 AM, John Paul Walters wrote: >> >> Hi, >> >> >> >> We have a 256 core SGI Ultraviolet machine running RHEL 6.1 with >> >> qemu-kvm 0.13, and we'd like to be able to start large guest VMs of up to >> >> 256 cores. I see that x86 guests are currently limited to 64 VCPUs. Is >> >> there any reason for this hard limitation? It appears that we can't get >> >> around this limitation by simply redefining the kernel's KVM_MAX_VCPUS to >> >> 256. Qemu-kvm and possibly SeaBIOS seem to require changes as well. Can >> >> anyone offer any suggestions as to how straightforward it would be to >> >> increase the number of CPUs that we can allocate to KVM guests? >> >> >> > >> > And here I am on record saying no one wants this... >> > >> > kvm.git has patches increasing the limit to 254 (256 is not possible due >> > to the APIC ID being 8 bits and two IDs being reserved). >> > >> > Latest seabios appears to have no cpu limits; qemu is limited to 255. >> > >> >> >> Hi again, >> >> I've applied the 254 core patches (below) from kvm.git on a RHEL 6.1 kernel. >> The new modules build and insert fine. >> >> https://git.kernel.org/?p=virt/kvm/kvm.git;a=commit;h=052fa7f4c5e79262cffcdc90bdd94172e00d45e3 >> https://git.kernel.org/?p=virt/kvm/kvm.git;a=commit;h=29a07f8e31980599c586ea7d1f84957bc7fe98ed >> >> However, whenever I try to boot a system with more than 83 CPUs, the system >> fails to boot with: >> >> Booting from Hard Disk... >> Boot failed: could not read the boot disk >> >> I'm using qemu-kvm.git with the following command line: >> /opt/qemu.git/bin/qemu-system-x86_64 -smp 84 -hda big_image_2.qcow2 -m 8388 >> -redir tcp:52109::22 >> >> Does anyone have any suggestions? >> >> > > Most likely a seabios failure. Suggest you enable debugging in seabios and > see what's going on; also copy the seabios mailing list. > Hi Avi,
I've enabled debugging in seabios (#define DEBUG_BIOS) and get the output below. Note that with the help of folks in the KVM irc channel I'm able to start a 254 core instance using the KVM tool, so the problem seems to be limited to qemu/seabios. best, JP jwalters@uv /tmp/qemu_test_jp $ /opt/qemu.git/bin/qemu-system-x86_64 -smp 84 -drive file=big_image_2.qcow2,if=virtio -m 8388 warning: subregion collision fffe0000/20000 vs 0/12c400000 VNC server running on `::1:5901' Start bios (version pre-0.6.3-20110315_112143-titi) Ram Size=0xe0000000 (0x000000012c400000 high) Relocating init from 0x000e49d0 to 0xdffe1880 (size 58968) CPU Mhz=2002 PCI: pci_bios_init_bus_rec bus = 0x0 PIIX3/PIIX4 init: elcr=00 0c PCI: bus=0 devfn=0x00: vendor_id=0x8086 device_id=0x1237 PCI: bus=0 devfn=0x08: vendor_id=0x8086 device_id=0x7000 PCI: bus=0 devfn=0x09: vendor_id=0x8086 device_id=0x7010 region 4: 0x0000c000 PCI: bus=0 devfn=0x0b: vendor_id=0x8086 device_id=0x7113 PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8 region 0: 0xf0000000 region 1: 0xf2000000 region 6: 0xf2010000 PCI: bus=0 devfn=0x18: vendor_id=0x10ec device_id=0x8139 region 0: 0x0000c100 region 1: 0xf2020000 region 6: 0xf2030000 PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1001 region 0: 0x0000c200 region 1: 0xf2040000 Found 84 cpu(s) max supported 84 cpu(s) MP table addr=0x000fd4b0 MPC table addr=0x000fd4c0 size=1892 SMBIOS ptr=0x000fd490 table=0xdffff030 ACPI tables: RSDP=0x000fd460 RSDT=0xdfffa5c0 Scan for VGA option rom Running option rom at c000:0003 VGABios $Id$ Turning on vga text mode console SeaBIOS (version pre-0.6.3-20110315_112143-titi) Found 1 lpt ports Found 1 serial ports ATA controller 0 at 1f0/3f4/0 (irq 14 dev 9) ATA controller 1 at 170/374/0 (irq 15 dev 9) found virtio-blk at 0:4 ebda moved from 9fc00 to 9dc00 WARNING - Unable to allocate resource at init_virtio_blk:107! WARNING - Unable to allocate resource at init_atadrive:740! PS2 keyboard initialized All threads complete. Scan for option roms Running option rom at c900:0003 pmm call arg1=1 pmm call arg1=0 pmm call arg1=1 pmm call arg1=0 Searching bootorder for: /pci@i0cf8/*@3 Searching bootorder for: /rom@genroms/vapic.bin Running option rom at ca00:0003 Returned 40960 bytes of ZoneHigh e820 map has 8 items: 0: 0000000000000000 - 000000000009dc00 = 1 1: 000000000009dc00 - 00000000000a0000 = 2 2: 00000000000f0000 - 0000000000100000 = 2 3: 0000000000100000 - 00000000dfffa000 = 1 4: 00000000dfffa000 - 00000000e0000000 = 2 5: 00000000feffc000 - 00000000ff000000 = 2 6: 00000000fffc0000 - 0000000100000000 = 2 7: 0000000100000000 - 000000022c400000 = 1 enter handle_19: NULL Booting from ROM... Booting from c900:0372 enter handle_18: NULL Booting from Hard Disk... Boot failed: could not read the boot disk enter handle_18: NULL Booting from Floppy... Boot failed: could not read the boot disk enter handle_18: NULL No bootable device. > -- > error compiling committee.c: too many arguments to function > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
