Thanks for your help! root@oi01:~# swap -sh total: 8.1G allocated + 21M reserved = 8.1G used, 16G available root@oi01:~# swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 100,2 8 12574712 12574712 root@oi01:~#
16G swap , so KVM can locked 16G memory . thanks again . ------------------------------ Fucai Liang On Dec 10, 2015, at 17:24, Jonathan Adams <t12nsloo...@gmail.com> wrote: > how much swap do you have? > > KVM automatically allocates the same amount of swap as memory when you > start an instance, if you don't have the free swap it won't work. > > Jon > > On 10 December 2015 at 04:04, Fucai Liang (BLCT) <fcli...@baolict.com> > wrote: > >> >> >> Hello, guys: >> >> I has a server running oi_151.1.8, the server have 32G memory . >> >> >> root@oi01:~# prtconf | grep Memory >> Memory size: 32760 Megabytes >> root@oi01:~# echo "::memstat" | mdb -k >> Page Summary Pages MB %Tot >> ------------ ---------------- ---------------- ---- >> Kernel 451736 1764 5% >> ZFS File Data 78429 306 1% >> Anon 19849 77 0% >> Exec and libs 586 2 0% >> Page cache 2170 8 0% >> Free (cachelist) 10432 40 0% >> Free (freelist) 7821076 30551 93% >> >> Total 8384278 32751 >> Physical 8384277 32751 >> root@oi01:~# >> >> >> I lauch a KVM VM by >> >> >> #!/bin/sh >> >> qemu-kvm \ >> -vnc 0.0.0.0:2 \ >> -cpu host \ >> -smp 2 \ >> -m 8192 \ >> -no-hpet \ >> -localtime \ >> -drive file=/dev/zvol/rdsk/rpool/svrinit,if=virtio,index=0 \ >> -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:01 \ >> -net >> vnic,vlan=0,name=e1000g0,ifname=vnic01,macaddr=2:8:20:bd:ae:01 \ >> -vga std \ >> -daemonize >> >> >> root@oi01:~# echo "::memstat" | mdb -k >> Page Summary Pages MB %Tot >> ------------ ---------------- ---------------- ---- >> Kernel 461061 1801 5% >> ZFS File Data 78446 306 1% >> Anon 2131265 8325 25% >> Exec and libs 918 3 0% >> Page cache 2255 8 0% >> Free (cachelist) 10015 39 0% >> Free (freelist) 5700318 22266 68% >> >> Total 8384278 32751 >> Physical 8384277 32751 >> root@oi01:~# >> >> >> then I launch second KVM VM, >> >> #!/bin/sh >> >> qemu-kvm \ >> -enable-kvm \ >> -vnc 0.0.0.0:3 \ >> -cpu host \ >> -smp 2 \ >> -m 8192 \ >> -no-hpet \ >> -localtime \ >> -drive file=/dev/zvol/rdsk/rpool/svr03,if=virtio,index=0 \ >> -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:03 \ >> -net >> vnic,vlan=0,name=e1000g0,ifname=vnic03,macaddr=2:8:20:bd:ae:03 \ >> -vga std \ >> -daemonize >> >> >> >> I got err message : >> Failed to allocate memory: Resource temporarily unavailable >> >> -bash: fork: Not enough space >> >> . >> >> the same ietmes happen on OmniOS , the following is a OmniOS-discuss mail >> . >> >> That caused by availrmem not enough for memory locked . >> >> Is that mean I can only use about 14G memory for KVM VM in a 32G memory >> installed server? >> OS can only locked about 14G memory . >> >> ——————FOLLOWING-------------- >> >> Hello, guys: >> >> I has a server running OmniOS v11 r151016. the server have 32G memory . >> I star tow kvm virtual machines by running the following commands: >> >> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu host -smp 4 -m 8192 >> -no-hpe >> >> >> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:11 -cpu host -smp 2 -m 4096 >> -no-hpe >> >> one use 8G memory and the other one use 4G memory. >> >> now the memory usage of the system as following: >> >> root@BLCC01:/root# prtconf | grep Memory >> Memory size: 32760 Megabytes >> root@BLCC01:/root# echo "::memstat" | mdb -k >> Page Summary Pages MB %Tot >> ------------ ---------------- ---------------- ---- >> Kernel 549618 2146 7% >> ZFS File Data 668992 2613 8% >> Anon 3198732 12495 38% >> Exec and libs 1411 5 0% >> Page cache 4402 17 0% >> Free (cachelist) 10578 41 0% >> Free (freelist) 3950545 15431 47% >> >> Total 8384278 32751 >> Physical 8384277 32751 >> root@BLCC01:/root# swap -sh >> total: 12G allocated + 35M reserved = 12G used, 6.8G available >> root@BLCC01:/root# swap -l >> swapfile dev swaplo blocks free >> /dev/zvol/dsk/rpool/swap 263,2 8 8388600 8388600 >> root@BLCC01:/root# >> >> >> root@BLCC01:/root# prctl $$ >> >> project.max-locked-memory >> usage 12.0GB >> system 16.0EB max deny >> - >> project.max-port-ids >> privileged 8.19K - deny >> - >> system 65.5K max deny >> - >> project.max-shm-memory >> privileged 8.00GB - deny >> - >> system 16.0EB max deny >> - >> >> >> >> >> >> >> #prstat -J >> >> PROJID NPROC SWAP RSS MEMORY TIME CPU PROJECT >> 1 5 12G 12G 38% 1:07:23 5.6% user.root >> 0 43 72M 76M 0.2% 0:00:59 0.0% system >> 3 5 4392K 14M 0.0% 0:00:00 0.0% default >> >> >> >> then I start the third vm (4G memory), it got the following error : >> >> >> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:2 -cpu host -smp 2 -m 4096 >> -no-hpet >> >> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying… >> >> >> I got 15G free memory in the system, why qemu-system-x86_64 can not locked >> enough memory ? >> >> Thanks for your help ! >> >> sorry for my poor english ! >> >> >> >> >> ----------------------------------- >> fcliang >> >> >> >> Thank for your help! >> >> when the server boot up, it has 7989066 pages availrmem. after I launch >> one VM (8Gmemory), availrmem decrease to 4756624 . >> >> >> >> 7989066-4756624 = 3232442 >> >> 3232442/256 = 12626.7265625 / 1024 = 12.3G >> >> >> >> root@BLCC01:/root# mdb -ke 'availrmem/D ; pages_pp_maximum/D' >> availrmem: >> availrmem: 7989066 >> pages_pp_maximum: >> pages_pp_maximum: 325044 >> >> root@BLCC01:/root# qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu >> host -smp 4 -m 8192 -no-hpe >> >> >> >> root@BLCC01:/root# mdb -ke 'availrmem/D ; pages_pp_maximum/D' >> availrmem: >> availrmem: 4756624 >> pages_pp_maximum: >> pages_pp_maximum: 325044 >> root@BLCC01:/root# >> >> >> That mean the VM use 12.3G availrmem , how it happens ? >> >> Thank ! >> >> >> >> >> ------------------------------ >> fcliang >> >> >> >> >> On Dec 2, 2015, at 1:37, Joshua M. Clulow <j...@sysmgr.org> wrote: >> >>> On 1 December 2015 at 09:11, Dan McDonald <dan...@omniti.com> wrote: >>>>> On Dec 1, 2015, at 12:03 PM, Fucai.Liang <fcli...@baolict.com> wrote: >>>>> then I start the third vm (4G memory), it got the following error : >>>>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:2 -cpu host -smp 2 -m >> 4096 -no-hpet >>>>> >>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying... >>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still >> trying… >>>>> >>>>> I got 15G free memory in the system, why qemu-system-x86_64 can not >> locked enough memory ? >>>> What does "vmstat 1 5" say prior to your launch of the third VM? >>> >>> I suspect it will show you have free memory available, but that what >>> is really happening is getting here: >>> >>> >> https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/seg_vn.c#L7989-L8002 >>> >>> This is likely failing in page_pp_lock() because "availrmem" has >>> fallen below "pages_pp_maximum": >>> >>> >> https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/vm_page.c#L3817-L3818 >>> >>> We set this value here, though it can be overridden in "/etc/system": >>> >>> >> https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/vm_page.c#L423-L436 >>> >>> You can look at the current values with mdb: >>> >>> mdb -ke 'availrmem/D ; pages_pp_maximum/D' >>> >>> Increasing this value doesn't seem to be without risk: I believe that >>> it can lead to memory exhaustion deadlocks, amongst other things. I >>> don't know if it's expected to be tuneable without a reboot. >>> >>> >>> Cheers. >>> >>> -- >>> Joshua M. Clulow >>> UNIX Admin/Developer >>> http://blog.sysmgr.org >> >> _______________________________________________ >> OmniOS-discuss mailing list >> omnios-disc...@lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> >> >> >> ------------------------------ >> Fucai Liang >> >> >> >> >> _______________________________________________ >> openindiana-discuss mailing list >> openindiana-discuss@openindiana.org >> http://openindiana.org/mailman/listinfo/openindiana-discuss >> > _______________________________________________ > openindiana-discuss mailing list > openindiana-discuss@openindiana.org > http://openindiana.org/mailman/listinfo/openindiana-discuss _______________________________________________ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss