I'm testing 5.3.2 with huge page enabled. Here's the setup : CONFIG proxy.config.cache.ram_cache.size = INT 137438953472 (128GB)
Normally I would need 65536 pages (2048KB in size), But I added ~12 GB extra : vm.nr_hugepages=72000 The results are good. no performance issues (on the current build I have), the PageTables size is hovering around 8000 kB (PageTables: 8440 kB). Before, this would go higher than 200,000kB creating some memory management issues every 32-33 minutes and causing high load average on the server (on code with hugepages modifications). After a weekend of load testing just so the RAM Cache is fully filled up, I see that all 72,000 pages are being used : [root@psp6cdedge03 ~]# cat /proc/meminfo | grep Huge > AnonHugePages: 0 kB > HugePages_Total: 72000 > HugePages_Free: 0 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 2048 kB If you enable Huge Pages, you'll see that traffic server doesn't report that RES memory in top anymore and can be a bit confusing... > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 7386 ats 20 0 154g 1.0g 39m S 45.0 0.4 2590:29 [ET_NET 0] This build also have the NUMA fix, before you'd see a bunch of numa_miss and numa_foreign : > # numastat > node0 node1 > numa_hit 25212626736 25457088463 > numa_miss 0 0 > numa_foreign 0 0 > interleave_hit 12464334 12464369 > local_node 25212625913 25444350193 > other_node 823 12738270 How much Huge Pages do I need extras? Is there some other parameters that would help me sizing the number of reserve huge page I need? Thanks, Steve
