Hello, guys:
I has a server running oi_151.1.8, the server have 32G memory . root@oi01:~# prtconf | grep Memory Memory size: 32760 Megabytes root@oi01:~# echo "::memstat" | mdb -k Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 451736 1764 5% ZFS File Data 78429 306 1% Anon 19849 77 0% Exec and libs 586 2 0% Page cache 2170 8 0% Free (cachelist) 10432 40 0% Free (freelist) 7821076 30551 93% Total 8384278 32751 Physical 8384277 32751 root@oi01:~# I lauch a KVM VM by #!/bin/sh qemu-kvm \ -vnc 0.0.0.0:2 \ -cpu host \ -smp 2 \ -m 8192 \ -no-hpet \ -localtime \ -drive file=/dev/zvol/rdsk/rpool/svrinit,if=virtio,index=0 \ -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:01 \ -net vnic,vlan=0,name=e1000g0,ifname=vnic01,macaddr=2:8:20:bd:ae:01 \ -vga std \ -daemonize root@oi01:~# echo "::memstat" | mdb -k Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 461061 1801 5% ZFS File Data 78446 306 1% Anon 2131265 8325 25% Exec and libs 918 3 0% Page cache 2255 8 0% Free (cachelist) 10015 39 0% Free (freelist) 5700318 22266 68% Total 8384278 32751 Physical 8384277 32751 root@oi01:~# then I launch second KVM VM, #!/bin/sh qemu-kvm \ -enable-kvm \ -vnc 0.0.0.0:3 \ -cpu host \ -smp 2 \ -m 8192 \ -no-hpet \ -localtime \ -drive file=/dev/zvol/rdsk/rpool/svr03,if=virtio,index=0 \ -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:03 \ -net vnic,vlan=0,name=e1000g0,ifname=vnic03,macaddr=2:8:20:bd:ae:03 \ -vga std \ -daemonize I got err message : Failed to allocate memory: Resource temporarily unavailable -bash: fork: Not enough space . the same ietmes happen on OmniOS , the following is a OmniOS-discuss mail . That caused by availrmem not enough for memory locked . Is that mean I can only use about 14G memory for KVM VM in a 32G memory installed server? OS can only locked about 14G memory . ——————FOLLOWING-------------- Hello, guys: I has a server running OmniOS v11 r151016. the server have 32G memory . I star tow kvm virtual machines by running the following commands: qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu host -smp 4 -m 8192 -no-hpe qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:11 -cpu host -smp 2 -m 4096 -no-hpe one use 8G memory and the other one use 4G memory. now the memory usage of the system as following: root@BLCC01:/root# prtconf | grep Memory Memory size: 32760 Megabytes root@BLCC01:/root# echo "::memstat" | mdb -k Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 549618 2146 7% ZFS File Data 668992 2613 8% Anon 3198732 12495 38% Exec and libs 1411 5 0% Page cache 4402 17 0% Free (cachelist) 10578 41 0% Free (freelist) 3950545 15431 47% Total 8384278 32751 Physical 8384277 32751 root@BLCC01:/root# swap -sh total: 12G allocated + 35M reserved = 12G used, 6.8G available root@BLCC01:/root# swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 263,2 8 8388600 8388600 root@BLCC01:/root# root@BLCC01:/root# prctl $$ project.max-locked-memory usage 12.0GB system 16.0EB max deny - project.max-port-ids privileged 8.19K - deny - system 65.5K max deny - project.max-shm-memory privileged 8.00GB - deny - system 16.0EB max deny - #prstat -J PROJID NPROC SWAP RSS MEMORY TIME CPU PROJECT 1 5 12G 12G 38% 1:07:23 5.6% user.root 0 43 72M 76M 0.2% 0:00:59 0.0% system 3 5 4392K 14M 0.0% 0:00:00 0.0% default then I start the third vm (4G memory), it got the following error : qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:2 -cpu host -smp 2 -m 4096 -no-hpet qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying… I got 15G free memory in the system, why qemu-system-x86_64 can not locked enough memory ? Thanks for your help ! sorry for my poor english ! ----------------------------------- fcliang Thank for your help! when the server boot up, it has 7989066 pages availrmem. after I launch one VM (8Gmemory), availrmem decrease to 4756624 . 7989066-4756624 = 3232442 3232442/256 = 12626.7265625 / 1024 = 12.3G root@BLCC01:/root# mdb -ke 'availrmem/D ; pages_pp_maximum/D' availrmem: availrmem: 7989066 pages_pp_maximum: pages_pp_maximum: 325044 root@BLCC01:/root# qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu host -smp 4 -m 8192 -no-hpe root@BLCC01:/root# mdb -ke 'availrmem/D ; pages_pp_maximum/D' availrmem: availrmem: 4756624 pages_pp_maximum: pages_pp_maximum: 325044 root@BLCC01:/root# That mean the VM use 12.3G availrmem , how it happens ? Thank ! ------------------------------ fcliang On Dec 2, 2015, at 1:37, Joshua M. Clulow <j...@sysmgr.org> wrote: > On 1 December 2015 at 09:11, Dan McDonald <dan...@omniti.com> wrote: >>> On Dec 1, 2015, at 12:03 PM, Fucai.Liang <fcli...@baolict.com> wrote: >>> then I start the third vm (4G memory), it got the following error : >>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:2 -cpu host -smp 2 -m 4096 >>> -no-hpet >>> >>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... >>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... >>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... >>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... >>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying... >>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying… >>> >>> I got 15G free memory in the system, why qemu-system-x86_64 can not locked >>> enough memory ? >> What does "vmstat 1 5" say prior to your launch of the third VM? > > I suspect it will show you have free memory available, but that what > is really happening is getting here: > > > https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/seg_vn.c#L7989-L8002 > > This is likely failing in page_pp_lock() because "availrmem" has > fallen below "pages_pp_maximum": > > > https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/vm_page.c#L3817-L3818 > > We set this value here, though it can be overridden in "/etc/system": > > > https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/vm_page.c#L423-L436 > > You can look at the current values with mdb: > > mdb -ke 'availrmem/D ; pages_pp_maximum/D' > > Increasing this value doesn't seem to be without risk: I believe that > it can lead to memory exhaustion deadlocks, amongst other things. I > don't know if it's expected to be tuneable without a reboot. > > > Cheers. > > -- > Joshua M. Clulow > UNIX Admin/Developer > http://blog.sysmgr.org _______________________________________________ OmniOS-discuss mailing list omnios-disc...@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ------------------------------ Fucai Liang _______________________________________________ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss