Been watching this thread, but figured I'd pipe up ... (need to blog this, it's certainly long enough, and I'll probably do so shortly)
1. VM balance The kernel balances program usage against dirty buffers (unflushed writes) against cache (reads that have been cached), using all available memory. When dirty buffers are not yet flushed and read cache is sucking up the rest of free memory, and programs want more memory, the kernel must either flush buffers or drop cache. Flushing buffers is more costly, although dropping read cache still takes overhead (TLB, page changes and related, administrative operations). 2. vm.drop_caches, it's a "command," not so much a tunable The sysfs interface (and /proc/sys) vm.drop_caches is not really a tunable, but a kernel "command" if you will. It tells the kernel to drop all memory being used for read caches so it can be utilized by programs. It also causes other types of read caches to be dropped, and other things tied to the VM system for reads (such as GFS2 master locks -- although GFS v1 works differently, with its own tunables, and is not as tied to the VM). 3. vm.* related to cache (reads) If you're finding that vm.drop_caches needs to be run, then it's an indicator that you should be tuning _other_ vm.* entries, especially those related to read cache. Some tunables to consider ... - vm.swappiness - vm.vfs_cache_pressure - vm.min_free_kbytes A big one for Oracle is vm.swappiness, which is set way too high for 4GiB+ memory systems at 60. It causes programs to be swapped out, which can actually cause read caches to increase in comparison (long story, but the kernel is tuned for smaller memory sizes by default, which are the majority of installs). Oracle recommends 10. Others will recommend even lower for large memory sizes. There is also vm.vfs_cache_pressure which addresses how the cache is utilized and what "pressure" (against free memory) is allowed. This is an excellent one to play with in a benchmark. Lastly, one variable not addressed often in most documentation, but I often set, is vm.min_free_kbytes. On large memory systems, I think it's way too low. It's the amount of memory the kernel tries to keep free after all programs, dirty buffers and cache. Typically it's 64MiB or smaller. So if a program asks for a large allocation, or another program starts up, buffers will need to be flushed or cache dropped. I like to set it more on the order of 5% of memory, so about 1GiB for every 20GiB. But that's just me. 4. vm.* related to dirty buffers (unflushed writes) This is indirectly related to caches (reads), but could be a contributing factor, especially if you're only assuming it's caches that need to be dropped. If your dirty buffers are regularly high or you have high I/O writes (use vmstat, sar -b and others), then your memory could be swamped by dirty buffers. With large memory sizes, it's not uncommon for 10GiB+ (e.g., 32GiB+ at 40% defaults) of memory that could be utilized in less than a minute. And then over the next, several minutes (or even 10+), your system is swapped with I/O just trying to be flushed. It can be deadly to system performance. - vm.dirty_background_ratio - vm.dirty_expire_centisecs - vm.dirty_ratio - vm.dirty_writeback_centisecs Instead of talking about how these four (4) inter-related, I invite you to read the most excellent and public Red Hat Oracle Tuning Guide: http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Oracle_Tuning_Guide/index.html Furthermore, these items are also covered, even more extensively, in Red Hat RH442, which I highly recommend for even "expert" level Linux sysadmins/engineers: https://www.redhat.com/courses/rh442_red_hat_enterprise_system_monitoring_and_performance_tuning/ You don't have to be a RHCE to sit the RH442 class (only the EX442 exam). You will often get one of Red Hat's best instructors, typically a top-tier RHCA with consulting experience and/or on-site time at major customers. 5. Hugepages, Oracle SGA and DB + App Server on the same system Lastly, hugepages are the ultimate, efficient way to run Oracle and even Java on Linux/x86-64. It reduces overhead on the TLB and paging logic, and that's even more of a consideration when running the DB and App Server on the same system. Traditionally Oracle has recomended one not put the DB and App Server on the same system, but with today's number of cores and memory sizes in the same system, this is often unnecessary. However, you want to set the shared memory (especially the number of hugepages) to be enough for both Oracle SGA and Java heap, with no overlap, if at all possible (and should should if you have a large memory system). One of Red Hat's expert consultants at tuning Oracle and App Servers is Scott Croft, who has presented at Red Hat Summit for two (2) years straight now. I have utilized his scripts for years now to calaculate shared memory in use in order to get an accurate number of huge pages. I could toot his resume of customers that now worship him, but those who know who he is know who he is. ;) Scott's Summit Blog and Presentation (2009-2010): http://summitblog.redhat.com/2010/05/05/red-hat-oracle-tuning-what-every-administrator-should-know/ http://www.redhat.com/f/pdf/summit/scroft_240_turning_rhel_for_oracle.pdf ----- Original Message ---- From: "Langley, Morgan (GE Capital)" <[email protected]> Well they were running some instances of Oracle DB and JBoss on the machine and they tuned their memory usage so one would not eat up too much memory, preventing the other from running normally... -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Srija Sent: Friday, September 24, 2010 3:45 PM To: Red Hat Enterprise Linux 5 (Tikanga) discussion mailing-list Subject: Re: [rhelv5-list] Tuning parameter drop_caches Thanks everybody by answering my queries. I am also thinking in the same way that using "drop_caches" is not the solution rather it is dangerous. ###to "clear the memory" and found some other solution. :) Would you pl. let me know what is the solution you have :) Thanks again to all of you. -- Bryan J Smith Professional, Technical Annoyance ------------------------------------------------------------ "Now if you own an automatic ... sell it! You are totally missing out on the coolest part of driving" -- Johnny O'Connell _______________________________________________ rhelv5-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/rhelv5-list
