You will find much information if you Google "configuring Linux page stealing". This is actually the core of the problem with swap (and throwing away pages of shared libraries). Or talk to your devops team about how to avoid page stealing in systems with large memory storage footprints ... especially as you may find java heaps in the 10-15gb are practicable with tuning.
*.......* *“Life should not be a journey to the grave with the intention of arriving safely in apretty and well preserved body, but rather to skid in broadside in a cloud of smoke,thoroughly used up, totally worn out, and loudly proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA (+1) 415.501.0198London (+44) (0) 20 8144 9872* On Wed, Mar 25, 2015 at 4:14 PM, Mich Talebzadeh <m...@peridale.co.uk> wrote: > Yes I believe that is the case. > > > > This is very common from days of Max shared memory on Solaris etc. Large > applications tend to have processes with large virtual address spaces. This > is typically the result of attaching to large shared memory segments used > by applications and large copy-on-write (COW) segments that get mapped but > sometimes never actually get touched. The net effect of this is that on the > host supporting multiple applications, the virtual address space > requirements will grow to be quite large, typically exceeding the physical > memory. Consequently, a fair amount of swap disk needs to be configured to > support these applications with large virtual address space running > concurrently. In the old days this would typically be 1.2* shared memory > segment or RAM > > > > > > HTH > > > > Mich Talebzadeh > > > > http://talebzadehmich.wordpress.com > > > > *Publications due shortly:* > > *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and > Coherence Cache* > > > > NOTE: The information in this email is proprietary and confidential. This > message is for the designated recipient only, if you are not the intended > recipient, you should destroy it immediately. Any information in this > message shall not be understood as given or endorsed by Peridale Ltd, its > subsidiaries or their employees, unless expressly so stated. It is the > responsibility of the recipient to ensure that this email is virus free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > > > *From:* max scalf [mailto:oracle.bl...@gmail.com] > *Sent:* 25 March 2015 23:05 > *To:* user@hadoop.apache.org > *Subject:* Re: Swap requirements > > > > Thank you harsh. Can you please explain what you mean when u said "Just > simple virtual memory used by the process" ? Doesn't virtual memory means > swap? > > On Wednesday, March 25, 2015, Harsh J <ha...@cloudera.com> wrote: > > The suggestion (regarding swappiness) is not for disabling swap as much as > it is to 'not using swap (until really necessary)'. When you run a constant > memory-consuming service such as HBase you'd ideally want the RAM to serve > up as much as it can, which setting that swappiness value helps do (the OS > otherwise begins swapping way before its available physical RAM is nearing > full state). > > > > The vmem-pmem ratio is something entirely else. The vmem of a process does > not mean swap space usage, just simple virtual memory used by the . I'd > recommend disabling YARN's vmem checks on today's OSes (but keep pmem > checks on). You can read some more on this at > http://www.quora.com/Why-do-some-applications-use-significantly-more-virtual-memory-on-RHEL-6-compared-to-RHEL-5 > > > > On Thu, Mar 26, 2015 at 3:37 AM, Abdul I Mohammed <oracle.bl...@gmail.com> > wrote: > > Thanks Mith...any idea about Yarn.nodemanager.Vmem-pmem-ratio parameter... > > If data nodes does not require swap then what about the above parameter? > What is that used for in yarn? > > > > > > -- > > Harsh J >
