On 2014-11-12 17:31, Neel Natu wrote: > Hi Tinker, > > On Wed, Nov 12, 2014 at 2:08 PM, Tinker <ti...@openmailbox.org> wrote: >> On 2014-11-12 23:55, Allan Jude wrote: >>> >>> On 2014-11-12 16:39, ti...@openmailbox.org wrote: >>>> >>>> Hi! >>>> >>>> In order justify giving energy to BHyVe, I need to know if it's >>>> "future-proof" in that the 16 vCPU limit can be increased? >>>> >>>> Please let the world know if BHyVe's 16 vCPU limit can be lifted in some >>>> way, by configuration, patch, etc. (and if you want to, why this limit >>>> is in place by default today). >>>> >>>> Thanks!! >>>> Tinker >>>> >>>> _______________________________________________ >>>> freebsd-virtualization@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>>> To unsubscribe, send any mail to >>>> "freebsd-virtualization-unsubscr...@freebsd.org" >>> >>> >>> You can increase the limit by editing sys/amd64/include/vmm.h >>> >>> #define VM_MAXCPU 16 >>> >>> From what I've been told, things scale badly above 24 CPUs. They plan to >>> solve this issue, but have not yet. If you system has enough cores to >>> support using more than 16 per VM, you can modify the file and recompile >>> the kernel and use as many CPUs as you want, but not much testing has >>> happened with bigger numbers. >> >> >> Dear Allan, >> >> Thank you very much for responding. >> >> Aha, great! >> >> >> Only for completeness, do you have any particular idea about >> * When the scaling above 24 vCPU:s will be optimized, like approx how many >> years away is it (like 1 or more than 1)?, and > > bhyve allocates memory for the maximum number of vcpus when the > virtual machine is created. This is simple to implement and also fits > in nicely with bhyve's loader model where the guest loader > (bhyveload/grub-bhyve) is separate from bhyve. > > The downside is that if VM_MAXCPU is a large number then there is a > large memory penalty for virtual machines that only use 1 or 2 vcpus. > Hence the desire to cap it at a smallish number. > > There are a few ways to deal with this: > - patch the code to change VM_MAXCPU (this is what you need to do today) > - allow the maximum vcpus to be changed via a tunable (this can be > done in short order) > - the limit can go away when bhyve moves to a single process model > because we'll know the number of vcpus at VM creation time. > >> * What the technological reason for the scaling is, is it that somehow the >> BHyVe instances on the different cores need to inter-communicate, for >> instance that all disk and network IO is done via one single core currently? >> > > Actually, the performance depends more on the workload and the guest > OS so you should definitely try how it goes for your application. > We'll be happy to help fix any issues that come up. > > best > Neel > >> >> In all cases, your response is great news, as your baseline answer that it's >> doable and only a question of optimization and tweaking of present >> functionality - >> >> Thanks again! >> >> Tinker >>
Right, I forgot that at MeetBSD it was decided that making the max cpus a sysctl was a good stop-gap measure to hold us over until the redesign of bhyveload. -- Allan Jude
signature.asc
Description: OpenPGP digital signature