After getting PerfKit working...
We can now see what is eating our system, and it turns out that it¹s z/VM... CPU Load Vector Facility Status or PROC TYPE %CPU%CP %EMU %WT %SYS %SP %SIC %LOGLD %VTOT %VEMUREST ded. User P00 IFL99 991 1 94 0 89100 .... ... Master P01 IFL403 37 601 0 88 40 .... ... Alternate We¹re grinding CPU 0 at 99%, and it¹s all CP time. Any insightful suggestions? -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different.
After getting PerfKit working...
And, it seems that our second LPAR (on another CEC) is playing the same game... P00 IFL 100 1000 0 100 0 61100 .... ... Master P01 IFL712 69 291 0 88 71 .... ... Alternate -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different.
Re: After getting PerfKit working...
Robert, Have you looked in the bottom right corner of the display to see what guest might be using the most CPU ? Or Option 21 from the main PerfKit menu ? Bob From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of RPN01 Sent: Tuesday, March 30, 2010 2:46 PM To: IBMVM@LISTSERV.UARK.EDU Subject: After getting PerfKit working... We can now see what is eating our system, and it turns out that it's z/VM... CPU Load Vector Facility Status or PROC TYPE %CPU%CP %EMU %WT %SYS %SP %SIC %LOGLD %VTOT %VEMUREST ded. User P00 IFL99 991 1 94 0 89100 .... ... Master P01 IFL403 37 601 0 88 40 .... ... Alternate We're grinding CPU 0 at 99%, and it's all CP time. Any insightful suggestions? -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different.
Re: After getting PerfKit working...
On 03/30/2010 01:46 PM, RPN01 wrote: We can now see what is eating our system, and it turns out that it’s z/VM... CPU Load Vector Facility Status or PROC TYPE %CPU %CP %EMU %WT %SYS %SP %SIC %LOGLD %VTOT %VEMU REST ded. User P00 IFL 99 99 1 1 94 0 89 100 .. .. ... Master P01 IFL 40 3 37 60 1 0 88 40 .. .. ... Alternate We’re grinding CPU 0 at 99%, and it’s all CP time. Any insightful suggestions? Is this the old active wait key 3 issue, or doesn't VM use that anymore? -- Rich Smrcina Phone: 414-491-6001 http://www.linkedin.com/in/richsmrcina Catch the WAVV! http://www.wavv.org WAVV 2010 - Apr 9-13, 2010 Covington, KY
Re: After getting PerfKit working...
The max CPU user extreme is 14%. This is all CP overhead time. Our paging rate is 20 pgs/sec. OPERATOR on both LPARs is unusually active, and I¹m about to take a walk over to the data center to see what it thinks it¹s doing. There doesn¹t seem to be anything else unusual going on. On 3/30/10 1:55 PM, Robert J McCarthy bob.mccar...@custserv.com wrote: Robert, Have you looked in the bottom right corner of the display to see what guest might be using the most CPU ? Or Option 21 from the main PerfKit menu ? Bob From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of RPN01 Sent: Tuesday, March 30, 2010 2:46 PM To: IBMVM@LISTSERV.UARK.EDU Subject: After getting PerfKit working... We can now see what is eating our system, and it turns out that it¹s z/VM... CPU Load Vector Facility Status or PROC TYPE %CPU%CP %EMU %WT %SYS %SP %SIC %LOGLD%VTOT %VEMUREST ded. User P00 IFL99 991 1 94 0 89100 .... ... Master P01 IFL403 37 601 0 88 40 .... ... Alternate We¹re grinding CPU 0 at 99%, and it¹s all CP time. Any insightful suggestions?
Re: After getting PerfKit working...
They found the Christmas Tree program and it is running on both Operator user id's right now ;-) munson RPN01 nix.rob...@mayo.edu Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 03/30/2010 03:23 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: After getting PerfKit working... The max CPU user extreme is 14%. This is all CP overhead time. Our paging rate is 20 pgs/sec. OPERATOR on both LPARs is unusually active, and I?m about to take a walk over to the data center to see what it thinks it?s doing. There doesn?t seem to be anything else unusual going on. On 3/30/10 1:55 PM, Robert J McCarthy bob.mccar...@custserv.com wrote: Robert, Have you looked in the bottom right corner of the display to see what guest might be using the most CPU ? Or Option 21 from the main PerfKit menu ? Bob From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of RPN01 Sent: Tuesday, March 30, 2010 2:46 PM To: IBMVM@LISTSERV.UARK.EDU Subject: After getting PerfKit working... We can now see what is eating our system, and it turns out that it?s z/VM... CPU Load Vector Facility Status or PROC TYPE %CPU%CP %EMU %WT %SYS %SP %SIC %LOGLD %VTOT %VEMUREST ded. User P00 IFL99 991 1 94 0 89100 .... ... Master P01 IFL403 37 601 0 88 40 .... ... Alternate We?re grinding CPU 0 at 99%, and it?s all CP time. Any insightful suggestions? *** IMPORTANT NOTE*-- The opinions expressed in this message and/or any attachments are those of the author and not necessarily those of Brown Brothers Harriman Co., its subsidiaries and affiliates (BBH). There is no guarantee that this message is either private or confidential, and it may have been altered by unauthorized sources without your or our knowledge. Nothing in the message is capable or intended to create any legally binding obligations on either party and it is not intended to provide legal advice. BBH accepts no responsibility for loss or damage from its use, including damage from virus.
Re: After getting PerfKit working...
I always thought allowing OPERATOR to IPL CMS was a BAD thing. Its been a while since I used VM but in the past issues like this were often spool related. Is something flooding the spool.. -Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Bill Munson Sent: 30 March 2010 20:27 To: IBMVM@LISTSERV.UARK.EDU Subject: Re: After getting PerfKit working... They found the Christmas Tree program and it is running on both Operator user id's right now ;-) munson RPN01 nix.rob...@mayo.edu Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 03/30/2010 03:23 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: After getting PerfKit working... The max CPU user extreme is 14%. This is all CP overhead time. Our paging rate is 20 pgs/sec. OPERATOR on both LPARs is unusually active, and I?m about to take a walk over to the data center to see what it thinks it?s doing. There doesn?t seem to be anything else unusual going on. On 3/30/10 1:55 PM, Robert J McCarthy bob.mccar...@custserv.com wrote: Robert, Have you looked in the bottom right corner of the display to see what guest might be using the most CPU ? Or Option 21 from the main PerfKit menu ? Bob From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of RPN01 Sent: Tuesday, March 30, 2010 2:46 PM To: IBMVM@LISTSERV.UARK.EDU Subject: After getting PerfKit working... We can now see what is eating our system, and it turns out that it?s z/VM... CPU Load Vector Facility Status or PROC TYPE %CPU%CP %EMU %WT %SYS %SP %SIC %LOGLD %VTOT %VEMUREST ded. User P00 IFL99 991 1 94 0 89100 .... ... Master P01 IFL403 37 601 0 88 40 .... ... Alternate We?re grinding CPU 0 at 99%, and it?s all CP time. Any insightful suggestions? *** IMPORTANT NOTE*-- The opinions expressed in this message and/or any attachments are those of the author and not necessarily those of Brown Brothers Harriman Co., its subsidiaries and affiliates (BBH). There is no guarantee that this message is either private or confidential, and it may have been altered by unauthorized sources without your or our knowledge. Nothing in the message is capable or intended to create any legally binding obligations on either party and it is not intended to provide legal advice. BBH accepts no responsibility for loss or damage from its use, including damage from virus. ** **
Re: After getting PerfKit working...
On Tuesday, 03/30/2010 at 03:30 EDT, Dave Wade g4...@dpwade.eclipse.co.uk wrote: I always thought allowing OPERATOR to IPL CMS was a BAD thing. Its been a while since I used VM but in the past issues like this were often spool related. Is something flooding the spool.. As a general rule, the key system operation user IDs should IPL 190, not CMS in order to reduce the dependency on the spool. If you're using something like IBM Operations Manager for z/VM to create console logs, then you eliminate still another spool touchpoint. Alan Altmark z/VM Development IBM Endicott
Re: After getting PerfKit working...
-Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Alan Altmark Sent: 30 March 2010 20:40 To: IBMVM@LISTSERV.UARK.EDU Subject: Re: After getting PerfKit working... On Tuesday, 03/30/2010 at 03:30 EDT, Dave Wade g4...@dpwade.eclipse.co.uk wrote: I always thought allowing OPERATOR to IPL CMS was a BAD thing. Its been a while since I used VM but in the past issues like this were often spool related. Is something flooding the spool.. As a general rule, the key system operation user IDs should IPL 190, not CMS in order to reduce the dependency on the spool. I was thinking IPL any thing as historically they didn't have a 190 or 191 and so couldn't run anything, inlcuding games. If you're using something like IBM Operations Manager for z/VM to create console logs, then you eliminate still another spool touchpoint. Alan Altmark z/VM Development IBM Endicott
Re: After getting PerfKit working...
On Tuesday, 03/30/2010 at 03:50 EDT, Dave Wade g4...@dpwade.eclipse.co.uk wrote: I was thinking IPL any thing as historically they didn't have a 190 or 191 and so couldn't run anything, inlcuding games. Captain, you're asking me to use equipment which isn't far ahead of stone knives and bearskins. Operate CP without CMS?!? I'll pretend I didn't hear that. Lalalalalala... Alan Altmark z/VM Development IBM Endicott
Re: After getting PerfKit working...
historically might be right. Certainly since Programmable Operator came along (IIRC) in VM/SP R3, running CMS on OPERATOR has been a GOOD THING. Later, more advanced console management products (IBM Operations Manager for z/VM, VM:Operator from CA, etc.) have vastly improved automation and productivity - to say nothing of all the pretty colored messages! These here days computer operators have much better and more advanced places to play Adventure, Zork, Pong, and all the rest. There's this cool thing called an Atari, and another - a Magnavox Odyssey (just supply your own TV as a monitor)... well, we were speaking historically, right? :-) Mike Walter Hewitt Associates The opinions expressed herein are mine alone, not my employer's. Dave Wade g4...@dpwade.eclipse.co.uk Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 03/30/2010 02:49 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: After getting PerfKit working... -Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Alan Altmark Sent: 30 March 2010 20:40 To: IBMVM@LISTSERV.UARK.EDU Subject: Re: After getting PerfKit working... On Tuesday, 03/30/2010 at 03:30 EDT, Dave Wade g4...@dpwade.eclipse.co.uk wrote: I always thought allowing OPERATOR to IPL CMS was a BAD thing. Its been a while since I used VM but in the past issues like this were often spool related. Is something flooding the spool.. As a general rule, the key system operation user IDs should IPL 190, not CMS in order to reduce the dependency on the spool. I was thinking IPL any thing as historically they didn't have a 190 or 191 and so couldn't run anything, inlcuding games. If you're using something like IBM Operations Manager for z/VM to create console logs, then you eliminate still another spool touchpoint. Alan Altmark z/VM Development IBM Endicott The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited. All messages sent to and from this e-mail address may be monitored as permitted by applicable law and regulations to ensure compliance with our internal policies and to protect our business. E-mails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, lost or destroyed, or contain viruses. You are deemed to have accepted these risks if you communicate with us by e-mail.
