Re: Reducing Linux virtual machine size
Hi Bruce , I would recommend you the utilization of the CMM-1 and CPUplugd feature . If a monitoring system as SYSSTAT you can gathering the data and understand how your system is working and how you could get the best of it . Best Regards . Saulo Augusto Silva 2010/7/23 Bruce Furber > Can someone recommend a how to procedure for monitoring a zLinux machine > to determine how much to reduce a machines virtual memory? > > Getting permission to schedule time to log a machines off is very difficult > so I have to be confident I have it right. > > -- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or > visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > -- > For more information on Linux on System z, visit > http://wiki.linuxvm.org/ > -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
Tom Duerbusch wrote:
Hi Ray.
I tried your script on some of my images. Works fine, except when Oracle is
involved.
linux62:~ # ps -eo pmem | awk '{pmem += $1};END {print "pmem ="pmem"%"}';
pmem =1347.1%
I do have a lot of swap blocks allocated. This is due to a batch type run,
that is run off hours. During the day, when users are on, we swap very little.
So if this does include swap pages, I don't think the script would give me what
I need, during normal processing. Do you agree? Or am off track here?
Thanks
Tom Duerbusch
THD Consulting
"Mrohs, Ray" 7/23/2010 1:45 PM >>>
Start up all your Linux procs and then run this little script.
#! /bin/sh
ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
It will give you a ballpark percentage of current memory utilization.
I tuned some Apache/ftp servers down to 100M with no ill effects.
To (maybe) alleviate the problem with oracle, add a sort step:
ps -eo pmem,cmd | sort -u -k2 | awk '{pmem += $1}; END {print "pmem =
"pmem"%"}';
This sorts by the second (and subsequent) key and drops duplicates. Be
aware it has its own problem, if you have lots of different copies of a
program (bash maybe) with the same commandline arguments, it will be
counted once.
You could also experiment with start time and any other sort keys that
seem attractive.
--
Cheers
John
-- spambait
1...@coco.merseine.nu z1...@coco.merseine.nu
-- Advice
http://webfoot.com/advice/email.top.php
http://www.catb.org/~esr/faqs/smart-questions.html
http://support.microsoft.com/kb/555375
You cannot reply off-list:-)
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
You currently have 6GB defined... ~40% of that is 2.4GB. I would try dropping
the Linux memory to around 4GB, and maybe add some swap space.
Ray
> -Original Message-
> From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On
> Behalf Of Rogério Soares
> Sent: Monday, July 26, 2010 4:09 PM
> To: LINUX-390@VM.MARIST.EDU
> Subject: Re: Reducing Linux virtual machine size
>
> Ray, please check this:
>
> db2p105:~ # free -m
> total used free shared
> buffers cached
> Mem: 6025 4285 1739 0
> 16 4045
> -/+ buffers/cache:223 5801
> Swap:0 0 0
> db2p105:~ # ps -eo pmem | awk '{pmem += $1}; END {print "pmem
> = "pmem"%"}';
> pmem = 39.4%
>
> With this, i can say, this machine needs +/- 2GB ?
> this machine runs DB2... i'm a little confuse with this because system
> requeriments for db2 ask a minimum of 3GB...
>
> thanks for help!
>
>
> On Fri, Jul 23, 2010 at 3:45 PM, Mrohs, Ray
> wrote:
> > Start up all your Linux procs and then run this little script.
> >
> > #! /bin/sh
> > ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
> >
> > It will give you a ballpark percentage of current memory
> utilization.
> > I tuned some Apache/ftp servers down to 100M with no ill effects.
> >
> > Ray Mrohs
> > U.S. Department of Justice
> > 202-307-6896
> >
> >
> >
> >
> >
> >> -Original Message-
> >> From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On
> >> Behalf Of Bruce Furber
> >> Sent: Friday, July 23, 2010 11:01 AM
> >> To: LINUX-390@VM.MARIST.EDU
> >> Subject: Reducing Linux virtual machine size
> >>
> >> Can someone recommend a how to procedure for monitoring a
> >> zLinux machine to determine how much to reduce a machines
> >> virtual memory?
> >>
> >> Getting permission to schedule time to log a machines off is
> >> very difficult so I have to be confident I have it right.
> >>
> >>
> --
> >> For LINUX-390 subscribe / signoff / archive access instructions,
> >> send email to lists...@vm.marist.edu with the message: INFO
> >> LINUX-390 or visit
> >> http://www.marist.edu/htbin/wlvindex?LINUX-390
> >>
> --
> >> For more information on Linux on System z, visit
> >> http://wiki.linuxvm.org/
> >>
> >
> --
> > For LINUX-390 subscribe / signoff / archive access instructions,
> > send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390 or visit
> > http://www.marist.edu/htbin/wlvindex?LINUX-390
> >
> --
> > For more information on Linux on System z, visit
> > http://wiki.linuxvm.org/
> >
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
Ray, please check this:
db2p105:~ # free -m
total used free sharedbuffers cached
Mem: 6025 4285 1739 0 16 4045
-/+ buffers/cache:223 5801
Swap:0 0 0
db2p105:~ # ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
pmem = 39.4%
With this, i can say, this machine needs +/- 2GB ?
this machine runs DB2... i'm a little confuse with this because system
requeriments for db2 ask a minimum of 3GB...
thanks for help!
On Fri, Jul 23, 2010 at 3:45 PM, Mrohs, Ray wrote:
> Start up all your Linux procs and then run this little script.
>
> #! /bin/sh
> ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
>
> It will give you a ballpark percentage of current memory utilization.
> I tuned some Apache/ftp servers down to 100M with no ill effects.
>
> Ray Mrohs
> U.S. Department of Justice
> 202-307-6896
>
>
>
>
>
>> -Original Message-
>> From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On
>> Behalf Of Bruce Furber
>> Sent: Friday, July 23, 2010 11:01 AM
>> To: LINUX-390@VM.MARIST.EDU
>> Subject: Reducing Linux virtual machine size
>>
>> Can someone recommend a how to procedure for monitoring a
>> zLinux machine to determine how much to reduce a machines
>> virtual memory?
>>
>> Getting permission to schedule time to log a machines off is
>> very difficult so I have to be confident I have it right.
>>
>> --
>> For LINUX-390 subscribe / signoff / archive access instructions,
>> send email to lists...@vm.marist.edu with the message: INFO
>> LINUX-390 or visit
>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>> --
>> For more information on Linux on System z, visit
>> http://wiki.linuxvm.org/
>>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
My guess is that Oracle runs a few processes using shared memory. The script
just totals
all the procs, shared or not, so shared memory applications will definitely
skew the results.
Ray
> -Original Message-
> From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On
> Behalf Of Tom Duerbusch
> Sent: Friday, July 23, 2010 6:05 PM
> To: LINUX-390@VM.MARIST.EDU
> Subject: Re: Reducing Linux virtual machine size
>
> Hi Ray.
>
> I tried your script on some of my images. Works fine, except
> when Oracle is involved.
>
> linux62:~ # ps -eo pmem | awk '{pmem += $1};END {print "pmem
> ="pmem"%"}';
> pmem =1347.1%
> I do have a lot of swap blocks allocated. This is due to a
> batch type run, that is run off hours. During the day, when
> users are on, we swap very little.
>
> So if this does include swap pages, I don't think the script
> would give me what I need, during normal processing. Do you
> agree? Or am off track here?
>
> Thanks
>
> Tom Duerbusch
> THD Consulting
>
> >>> "Mrohs, Ray" 7/23/2010 1:45 PM >>>
> Start up all your Linux procs and then run this little script.
>
> #! /bin/sh
> ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
>
> It will give you a ballpark percentage of current memory utilization.
> I tuned some Apache/ftp servers down to 100M with no ill effects.
>
> Ray Mrohs
> U.S. Department of Justice
> 202-307-6896
>
>
>
>
>
> > -----Original Message-
> > From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On
> > Behalf Of Bruce Furber
> > Sent: Friday, July 23, 2010 11:01 AM
> > To: LINUX-390@VM.MARIST.EDU
> > Subject: Reducing Linux virtual machine size
> >
> > Can someone recommend a how to procedure for monitoring a
> > zLinux machine to determine how much to reduce a machines
> > virtual memory?
> >
> > Getting permission to schedule time to log a machines off is
> > very difficult so I have to be confident I have it right.
> >
> >
> --
> > For LINUX-390 subscribe / signoff / archive access instructions,
> > send email to lists...@vm.marist.edu with the message: INFO
> > LINUX-390 or visit
> > http://www.marist.edu/htbin/wlvindex?LINUX-390
> >
> --
> > For more information on Linux on System z, visit
> > http://wiki.linuxvm.org/
> >
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
I installed VMRMSVM it works and stopped VM from stealing pages. But it seems to take too many pages away from some machines while leavings others that I know are over allocated alone. Bruce -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
Sorry ... that should be "compressed _swap_ cache" Shane ... On Sat, Jul 24th, 2010 at 9:00 AM, I wrote: > ... Recently the memory manager has > started using compressed page cache to ameliorate this effect somewhat. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
I would be guessing you have lots of Oracle threads - all of which will have
the same common code mapped. And counted by the summation.
These fields only count resident memory - swap usage will be (sort of)
irrelevant. Pages that are swap cached (as distinct from only swapped out)
reside in real pages, and so can affect the pool of available page frames.
Determining this number is not simple. Recently the memory manager has
started using compressed page cache to ameliorate this effect somewhat.
Refer to my previous (flippant) comment.
Shane ...
On Sat, Jul 24th, 2010 at 8:05 AM, Tom Duerbusch wrote:
> I tried your script on some of my images. Works fine, except when Oracle
> is involved.
>
> linux62:~ # ps -eo pmem | awk '{pmem += $1};END {print "pmem ="pmem"%"}';
> pmem =1347.1%
> I do have a lot of swap blocks allocated. This is due to a batch type
> run, that is run off hours. During the day, when users are on, we swap
> very little.
>
> So if this does include swap pages, I don't think the script would give me
> what I need, during normal processing. Do you agree? Or am off track
> here?
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
This is somewhat empirical, and depending on the usage of shared libraries
will over-estimate the situation somewhat.
Not a bad thing in the context.
Determining actual accountable memory on linux has been like catching a
greased pig. With recent kernels you can work back from the pagemaps - here's
a citation from ../Documentation in the kernel source tree.
Using pagemap to do something useful:
The general procedure for using pagemap to find out about a process' memory
usage goes like this:
1. Read /proc/pid/maps to determine which parts of the memory space are
mapped to what.
2. Select the maps you are interested in -- all of them, or a particular
library, or the stack or the heap, etc.
3. Open /proc/pid/pagemap and seek to the pages you would like to examine.
4. Read a u64 for each page from pagemap.
5. Open /proc/kpagecount and/or /proc/kpageflags. For each PFN you just
read, seek to that entry in the file, and read the data you want.
For example, to find the "unique set size" (USS), which is the amount of
memory that a process is using that is not shared with any other process,
you can go through every map in the process, find the PFNs, look those up
in kpagecount, and tally up the number of pages that are only referenced
once.
I have seen some user-space code to do all this - must move the testing of
this up my "to-do" list.
Shane ...
On Sat, Jul 24th, 2010 at 4:45 AM, "Mrohs, Ray" wrote:
> Start up all your Linux procs and then run this little script.
>
> #! /bin/sh
> ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
>
> It will give you a ballpark percentage of current memory utilization.
> I tuned some Apache/ftp servers down to 100M with no ill effects.
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
Hi Ray.
I tried your script on some of my images. Works fine, except when Oracle is
involved.
linux62:~ # ps -eo pmem | awk '{pmem += $1};END {print "pmem ="pmem"%"}';
pmem =1347.1%
I do have a lot of swap blocks allocated. This is due to a batch type run,
that is run off hours. During the day, when users are on, we swap very little.
So if this does include swap pages, I don't think the script would give me what
I need, during normal processing. Do you agree? Or am off track here?
Thanks
Tom Duerbusch
THD Consulting
>>> "Mrohs, Ray" 7/23/2010 1:45 PM >>>
Start up all your Linux procs and then run this little script.
#! /bin/sh
ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
It will give you a ballpark percentage of current memory utilization.
I tuned some Apache/ftp servers down to 100M with no ill effects.
Ray Mrohs
U.S. Department of Justice
202-307-6896
> -Original Message-
> From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On
> Behalf Of Bruce Furber
> Sent: Friday, July 23, 2010 11:01 AM
> To: LINUX-390@VM.MARIST.EDU
> Subject: Reducing Linux virtual machine size
>
> Can someone recommend a how to procedure for monitoring a
> zLinux machine to determine how much to reduce a machines
> virtual memory?
>
> Getting permission to schedule time to log a machines off is
> very difficult so I have to be confident I have it right.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
Start up all your Linux procs and then run this little script.
#! /bin/sh
ps -eo pmem | awk '{pmem += $1}; END {print "pmem = "pmem"%"}';
It will give you a ballpark percentage of current memory utilization.
I tuned some Apache/ftp servers down to 100M with no ill effects.
Ray Mrohs
U.S. Department of Justice
202-307-6896
> -Original Message-
> From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On
> Behalf Of Bruce Furber
> Sent: Friday, July 23, 2010 11:01 AM
> To: LINUX-390@VM.MARIST.EDU
> Subject: Reducing Linux virtual machine size
>
> Can someone recommend a how to procedure for monitoring a
> zLinux machine to determine how much to reduce a machines
> virtual memory?
>
> Getting permission to schedule time to log a machines off is
> very difficult so I have to be confident I have it right.
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
it supposes do reduce online... my plan is turn on VMRM, monitor and then reduce memory definitive based on VMRM adjusts. 2010/7/23 Rogério Soares : > Hi Bruce, > > I have the same problem.. i'm trying now set up VMRM.. > > > Take a look this: > http://www.vm.ibm.com/sysman/vmrm/vmrmcmm.html > > and this, to active then... (chapter 17) > > * http://publibz.boulder.ibm.com/epubs/pdf/hcsi1b31.pdf > > *z/vm 5.4 > > On Fri, Jul 23, 2010 at 12:01 PM, Bruce Furber wrote: >> Can someone recommend a how to procedure for monitoring a zLinux machine to >> determine how much to reduce a machines virtual memory? >> >> Getting permission to schedule time to log a machines off is very difficult >> so I have to be confident I have it right. >> >> -- >> For LINUX-390 subscribe / signoff / archive access instructions, >> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or >> visit >> http://www.marist.edu/htbin/wlvindex?LINUX-390 >> -- >> For more information on Linux on System z, visit >> http://wiki.linuxvm.org/ >> > -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: Reducing Linux virtual machine size
Hi Bruce, I have the same problem.. i'm trying now set up VMRM.. Take a look this: http://www.vm.ibm.com/sysman/vmrm/vmrmcmm.html and this, to active then... (chapter 17) * http://publibz.boulder.ibm.com/epubs/pdf/hcsi1b31.pdf *z/vm 5.4 On Fri, Jul 23, 2010 at 12:01 PM, Bruce Furber wrote: > Can someone recommend a how to procedure for monitoring a zLinux machine to > determine how much to reduce a machines virtual memory? > > Getting permission to schedule time to log a machines off is very difficult > so I have to be confident I have it right. > > -- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > -- > For more information on Linux on System z, visit > http://wiki.linuxvm.org/ > -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Reducing Linux virtual machine size
Can someone recommend a how to procedure for monitoring a zLinux machine to determine how much to reduce a machines virtual memory? Getting permission to schedule time to log a machines off is very difficult so I have to be confident I have it right. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
