Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
Hi Bill; We have 295 CP volumes (3390 mod 27) shared between the two LPAR's, running CSE. There are an additional 15,652 volumes owned by z/OS, which we keep offline to z/VM. We run a script in AUTOLOG1 which goes through the list of volumes and makes the decision for each if it should stay online to z/VM, and takes action to set things into their normal state. ckofflin 0 errors 295 CP OWNED or SYSTEM disks skipped 0 PAV Aliases skipped 15652 found already offline 0 varied offline Ready; T=0.22/0.31 08:23:27 -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different. On 3/30/10 1:18 PM, Bill Munson william.mun...@bbh.com wrote: Eginhard, Yes it was trial and error - and we made it LARGE enough not to fail again In our Largest LPAR we have 80 guests running - 52 are LINUX guests. 70 mod27's and 75 mod3's for (paging) and one mod9 the RES pack. we have 5 VM lpars and all lpars can see the other dasd, though not attached to the system, only the dasd for each LPAR is attached to that system, we do not vary off anything but the MVS dasd q monitor MONITOR EVENT ACTIVEBLOCK4 PARTITION16384 MONITOR DCSS NAME - MONDCSS CONFIGURATION SIZE 800 LIMIT 1 MINUTES CONFIGURATION AREA IS FREE USERS CONNECTED TO *MONITOR - ESAWRITE LINMON PERFSVM MONITOR SAMPLE ACTIVE INTERVAL1 MINUTES RATE 1.00 SECONDS MONITOR DCSS NAME - MONDCSS CONFIGURATION SIZE 1500 LIMIT 1 MINUTES CONFIGURATION AREA IS FREE USERS CONNECTED TO *MONITOR - ESAWRITE LINMON PERFSVM munson 201-418-7588 Eginhard Jaeger e.jae...@ch.inter.net Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 03/30/2010 01:57 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small There is no single 'right' MONDCSS size for all systems: it's about performance, so 'it depends'. The MONDCSS has to be large enough to allow the CP monitor to place all the monitor records you told it to collect in that storage area. Since most users just go and enable whole domains, it's the domains generating the largest number of monitor records that one wants to watch. For sample records that is, on most systems, the I/O domain, where you could end up with tens of thousands of devices already years ago when I still worked with VM. Be aware that the monitor will create a device activity record 3 of 268 bytes and a cache activity record 4 of 264 bytes for each DASD, and they must all fit simultaneously into the MONDCSS, together with all the other monitor records. (And, as mentioned in another append, the default SAMPLE CONFIG size is often too small for so many devices and has to be made larger.) But there's one general rule that has not yet been mentioned in this thread: don't let the MONDCSS overlay the storage of the virtual machine that is doing the data collecting, in this case PerfKit, or it will not be able to use it. While your MONDCSS looks VERY large to me, I'm admittedly out of date as far as current I/O configurations are concerned, and you apparently ended up with it for a good reason, after a trial and error phase with smaller sizes. Can you tell me the number of I/O devices that your VM sees and is collecting data for? Eginhard - Original Message - From: Bill Munson william.mun...@bbh.com That does not look like it is large enough. here is my definition MONDCSS CPDCSS N/A08000 0 SC R It can work for a while but if the segment is not large enough it will soon fail. *** IMPORTANT NOTE*-- The opinions expressed in this message and/or any attachments are those of the author and not necessarily those of Brown Brothers Harriman Co., its subsidiaries and affiliates (BBH). There is no guarantee that this message is either private or confidential, and it may have been altered by unauthorized sources without your or our knowledge. Nothing in the message is capable or intended to create any legally binding obligations on either party and it is not intended to provide legal advice. BBH accepts no responsibility for loss or damage from its use, including damage from virus. ** **
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
It's very probably the 15'652 z/OS volumes that caused your problems. Even when varied offline, the CP monitor will generate a configuration record of 256 bytes for each of those volumes whenever you issue a MONITOR START command. CP's default is to reserve 1/2 of the whole MONDCSS for event data, i.e. only 50% are left for sample data. However, the monitor configuration records will be written into a separate sub-area of those 50% left for sample data, and that is what the CP monitor (and then PerfKit) complained about: the configuration data sub-area was not defined large enough. Be aware that the SAMPLE CONFIG SIZE needed just for the z/OS volumes is 15'652*256 or about 1000 page frames! I don't know what default size is currently set by CP when you set up a MONDCSS (haven't found it in the documentation) but for systems with large numbers of I/O devices I'd recommend setting the SAMPLE CONFIG SIZE manually to about 1/6 of the total MONDCSS size (i.e. 1/3 of the default sample data area) to avoid this kind of problems. If you then still get 'SAMPLE CONFIG size too small' messages it is time to define a larger MONDCSS (and adapt SAMPLE CONFIG). Eginhard - Original Message - From: RPN01 nix.rob...@mayo.edu To: IBMVM@LISTSERV.UARK.EDU Sent: Wednesday, March 31, 2010 3:28 PM Subject: Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small Hi Bill; We have 295 CP volumes (3390 mod 27) shared between the two LPAR's, running CSE. There are an additional 15,652 volumes owned by z/OS, which we keep offline to z/VM. We run a script in AUTOLOG1 which goes through the list of volumes and makes the decision for each if it should stay online to z/VM, and takes action to set things into their normal state. ckofflin 0 errors 295 CP OWNED or SYSTEM disks skipped 0 PAV Aliases skipped 15652 found already offline 0 varied offline Ready; T=0.22/0.31 08:23:27 -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different. On 3/30/10 1:18 PM, Bill Munson william.mun...@bbh.com wrote: Eginhard, Yes it was trial and error - and we made it LARGE enough not to fail again In our Largest LPAR we have 80 guests running - 52 are LINUX guests. 70 mod27's and 75 mod3's for (paging) and one mod9 the RES pack. we have 5 VM lpars and all lpars can see the other dasd, though not attached to the system, only the dasd for each LPAR is attached to that system, we do not vary off anything but the MVS dasd q monitor MONITOR EVENT ACTIVEBLOCK4 PARTITION16384 MONITOR DCSS NAME - MONDCSS CONFIGURATION SIZE 800 LIMIT 1 MINUTES CONFIGURATION AREA IS FREE USERS CONNECTED TO *MONITOR - ESAWRITE LINMON PERFSVM MONITOR SAMPLE ACTIVE INTERVAL1 MINUTES RATE 1.00 SECONDS MONITOR DCSS NAME - MONDCSS CONFIGURATION SIZE 1500 LIMIT 1 MINUTES CONFIGURATION AREA IS FREE USERS CONNECTED TO *MONITOR - ESAWRITE LINMON PERFSVM munson 201-418-7588 Eginhard Jaeger e.jae...@ch.inter.net Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 03/30/2010 01:57 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small There is no single 'right' MONDCSS size for all systems: it's about performance, so 'it depends'. The MONDCSS has to be large enough to allow the CP monitor to place all the monitor records you told it to collect in that storage area. Since most users just go and enable whole domains, it's the domains generating the largest number of monitor records that one wants to watch. For sample records that is, on most systems, the I/O domain, where you could end up with tens of thousands of devices already years ago when I still worked with VM. Be aware that the monitor will create a device activity record 3 of 268 bytes and a cache activity record 4 of 264 bytes for each DASD, and they must all fit simultaneously into the MONDCSS, together with all the other monitor records. (And, as mentioned in another append, the default SAMPLE CONFIG size is often too small for so many devices and has to be made larger.) But there's one general rule that has not yet been mentioned in this thread: don't let the MONDCSS overlay the storage of the virtual machine that is doing the data collecting, in this case PerfKit, or it will not be able to use it. While your MONDCSS looks VERY large to me, I'm admittedly out of date as far as current I/O configurations are concerned, and you apparently ended up with it for a good reason, after a trial
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
very large mondcss segments do not impact performance, only small ones do. B On Mar 30, 2010, at 11:21, RPN01 nix.rob...@mayo.edu wrote: Our MONDCSS grew, perhaps too large, while fighting this type message a long time ago. Once the problem was resolved, we didn't attempt to back off the changes we'd made, and the large size doesn't seem to hurt anything at the moment. I know that ultimately, making the segment larger was not the answer to the problem at the time, either. Also, Mr. Nunsford says hello. -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different. On 3/30/10 12:57 PM, Eginhard Jaeger e.jae...@ch.inter.net wrote: There is no single 'right' MONDCSS size for all systems: it's about performance, so 'it depends'. The MONDCSS has to be large enough to allow the CP monitor to place all the monitor records you told it to collect in that storage area. Since most users just go and enable whole domains, it's the domains generating the largest number of monitor records that one wants to watch. For sample records that is, on most systems, the I/O domain, where you could end up with tens of thousands of devices already years ago when I still worked with VM. Be aware that the monitor will create a device activity record 3 of 268 bytes and a cache activity record 4 of 264 bytes for each DASD, and they must all fit simultaneously into the MONDCSS, together with all the other monitor records. (And, as mentioned in another append, the default SAMPLE CONFIG size is often too small for so many devices and has to be made larger.) But there's one general rule that has not yet been mentioned in this thread: don't let the MONDCSS overlay the storage of the virtual machine that is doing the data collecting, in this case PerfKit, or it will not be able to use it. While your MONDCSS looks VERY large to me, I'm admittedly out of date as far as current I/O configurations are concerned, and you apparently ended up with it for a good reason, after a trial and error phase with smaller sizes. Can you tell me the number of I/O devices that your VM sees and is collecting data for? Eginhard - Original Message - From: Bill Munson william.mun...@bbh.com That does not look like it is large enough. here is my definition MONDCSS CPDCSS N/A08000 0 SC R It can work for a while but if the segment is not large enough it will soon fail.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
WAVV requirement WRIBDB12 submitted to increase the default MONDCSS size to accommodate the maximum supported configuration for a VM system. -- db On 3/30/10 2:51 PM, Barton Robinson bar...@velocitysoftware.com wrote: very large mondcss segments do not impact performance, only small ones do.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
Re: Perfkit SAMPLE CONFIG size too smallThis is usually caused by zVM accessing lots of additional I/O devices that it previously didn't see. While redefining the CONFIG size and the size of the MONDCSS can allow CP to again pack all of the available information into its monitor data, all of the additional monitor records must also be processed. The FCXPMN44E message means probably that PerfKit couldn't process the data fast enough, i.e. it needs a higher share of the CPU. But check first whether the problem is really caused by additional I/O devices. Is your VM seeing lots of devices belonging to other LPARs? Vary off the ones that are not needed, or tell the monitor via MONITOR command not to collect data for the ones you're not interested in. Eginhard My questions now are, What happened to the prior segment that caused it to fail? Could the problem have been avoided, and how? Also, now I'm getting the following two messages, repeatedly, and we're still not collecting any data: FCXPMN444E IUCV reply failed with reason code 9 HCPMOV6274I The sample data messages and corresponding records have been purged.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
There is no single 'right' MONDCSS size for all systems: it's about performance, so 'it depends'. The MONDCSS has to be large enough to allow the CP monitor to place all the monitor records you told it to collect in that storage area. Since most users just go and enable whole domains, it's the domains generating the largest number of monitor records that one wants to watch. For sample records that is, on most systems, the I/O domain, where you could end up with tens of thousands of devices already years ago when I still worked with VM. Be aware that the monitor will create a device activity record 3 of 268 bytes and a cache activity record 4 of 264 bytes for each DASD, and they must all fit simultaneously into the MONDCSS, together with all the other monitor records. (And, as mentioned in another append, the default SAMPLE CONFIG size is often too small for so many devices and has to be made larger.) But there's one general rule that has not yet been mentioned in this thread: don't let the MONDCSS overlay the storage of the virtual machine that is doing the data collecting, in this case PerfKit, or it will not be able to use it. While your MONDCSS looks VERY large to me, I'm admittedly out of date as far as current I/O configurations are concerned, and you apparently ended up with it for a good reason, after a trial and error phase with smaller sizes. Can you tell me the number of I/O devices that your VM sees and is collecting data for? Eginhard - Original Message - From: Bill Munson william.mun...@bbh.com That does not look like it is large enough. here is my definition MONDCSS CPDCSS N/A08000 0 SC R It can work for a while but if the segment is not large enough it will soon fail.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
I plan to talk to the hardware people in a moment to see if anything was added recently? We do have a large amount of DASD that is really owned by z/OS, which we immediately take offline, but they¹d still be in the config area. -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different. On 3/30/10 11:21 AM, Eginhard Jaeger e.jae...@ch.inter.net wrote: This is usually caused by zVM accessing lots of additional I/O devices that it previously didn't see. While redefining the CONFIG size and the size of the MONDCSS can allow CP to again pack all of the available information into its monitor data, all of the additional monitor records must also be processed. The FCXPMN44E message means probably that PerfKit couldn't process the data fast enough, i.e. it needs a higher share of the CPU. But check first whether the problem is really caused by additional I/O devices. Is your VM seeing lots of devices belonging to other LPARs? Vary off the ones that are not needed, or tell the monitor via MONITOR command not to collect data for the ones you're not interested in. Eginhard My questions now are, What happened to the prior segment that caused it to fail? Could the problem have been avoided, and how? Also, now I¹m getting the following two messages, repeatedly, and we¹re still not collecting any data: FCXPMN444E IUCV reply failed with reason code 9 HCPMOV6274I The sample data messages and corresponding records have been purged.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
Eginhard, Yes it was trial and error - and we made it LARGE enough not to fail again In our Largest LPAR we have 80 guests running - 52 are LINUX guests. 70 mod27's and 75 mod3's for (paging) and one mod9 the RES pack. we have 5 VM lpars and all lpars can see the other dasd, though not attached to the system, only the dasd for each LPAR is attached to that system, we do not vary off anything but the MVS dasd q monitor MONITOR EVENT ACTIVEBLOCK4 PARTITION16384 MONITOR DCSS NAME - MONDCSS CONFIGURATION SIZE 800 LIMIT 1 MINUTES CONFIGURATION AREA IS FREE USERS CONNECTED TO *MONITOR - ESAWRITE LINMON PERFSVM MONITOR SAMPLE ACTIVE INTERVAL1 MINUTES RATE 1.00 SECONDS MONITOR DCSS NAME - MONDCSS CONFIGURATION SIZE 1500 LIMIT 1 MINUTES CONFIGURATION AREA IS FREE USERS CONNECTED TO *MONITOR - ESAWRITE LINMON PERFSVM munson 201-418-7588 Eginhard Jaeger e.jae...@ch.inter.net Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 03/30/2010 01:57 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small There is no single 'right' MONDCSS size for all systems: it's about performance, so 'it depends'. The MONDCSS has to be large enough to allow the CP monitor to place all the monitor records you told it to collect in that storage area. Since most users just go and enable whole domains, it's the domains generating the largest number of monitor records that one wants to watch. For sample records that is, on most systems, the I/O domain, where you could end up with tens of thousands of devices already years ago when I still worked with VM. Be aware that the monitor will create a device activity record 3 of 268 bytes and a cache activity record 4 of 264 bytes for each DASD, and they must all fit simultaneously into the MONDCSS, together with all the other monitor records. (And, as mentioned in another append, the default SAMPLE CONFIG size is often too small for so many devices and has to be made larger.) But there's one general rule that has not yet been mentioned in this thread: don't let the MONDCSS overlay the storage of the virtual machine that is doing the data collecting, in this case PerfKit, or it will not be able to use it. While your MONDCSS looks VERY large to me, I'm admittedly out of date as far as current I/O configurations are concerned, and you apparently ended up with it for a good reason, after a trial and error phase with smaller sizes. Can you tell me the number of I/O devices that your VM sees and is collecting data for? Eginhard - Original Message - From: Bill Munson william.mun...@bbh.com That does not look like it is large enough. here is my definition MONDCSS CPDCSS N/A08000 0 SC R It can work for a while but if the segment is not large enough it will soon fail. *** IMPORTANT NOTE*-- The opinions expressed in this message and/or any attachments are those of the author and not necessarily those of Brown Brothers Harriman Co., its subsidiaries and affiliates (BBH). There is no guarantee that this message is either private or confidential, and it may have been altered by unauthorized sources without your or our knowledge. Nothing in the message is capable or intended to create any legally binding obligations on either party and it is not intended to provide legal advice. BBH accepts no responsibility for loss or damage from its use, including damage from virus.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
Our MONDCSS grew, perhaps too large, while fighting this type message a long time ago. Once the problem was resolved, we didn't attempt to back off the changes we'd made, and the large size doesn't seem to hurt anything at the moment. I know that ultimately, making the segment larger was not the answer to the problem at the time, either. Also, Mr. Nunsford says hello. -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different. On 3/30/10 12:57 PM, Eginhard Jaeger e.jae...@ch.inter.net wrote: There is no single 'right' MONDCSS size for all systems: it's about performance, so 'it depends'. The MONDCSS has to be large enough to allow the CP monitor to place all the monitor records you told it to collect in that storage area. Since most users just go and enable whole domains, it's the domains generating the largest number of monitor records that one wants to watch. For sample records that is, on most systems, the I/O domain, where you could end up with tens of thousands of devices already years ago when I still worked with VM. Be aware that the monitor will create a device activity record 3 of 268 bytes and a cache activity record 4 of 264 bytes for each DASD, and they must all fit simultaneously into the MONDCSS, together with all the other monitor records. (And, as mentioned in another append, the default SAMPLE CONFIG size is often too small for so many devices and has to be made larger.) But there's one general rule that has not yet been mentioned in this thread: don't let the MONDCSS overlay the storage of the virtual machine that is doing the data collecting, in this case PerfKit, or it will not be able to use it. While your MONDCSS looks VERY large to me, I'm admittedly out of date as far as current I/O configurations are concerned, and you apparently ended up with it for a good reason, after a trial and error phase with smaller sizes. Can you tell me the number of I/O devices that your VM sees and is collecting data for? Eginhard - Original Message - From: Bill Munson william.mun...@bbh.com That does not look like it is large enough. here is my definition MONDCSS CPDCSS N/A08000 0 SC R It can work for a while but if the segment is not large enough it will soon fail.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
When you immediately take offline the unneeded devices, I presume you mean in some EXEC after the IPL completes? Might the control blocks not even be built of you get a bit more picky in SYSTEM CONFIG with: DEVICES OFFLINE_AT_IPL rdev-rdev ... etc. I do not know for fact, but having CP take them offline at IPL may help reduce MONDCSS storage (and perhaps CP control block structures) with little effort. Mike Walter Hewitt Associates The opinions expressed herein are mine alone, not my employer's. RPN01 nix.rob...@mayo.edu Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 03/30/2010 01:14 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small I plan to talk to the hardware people in a moment to see if anything was added recently? We do have a large amount of DASD that is really owned by z/OS, which we immediately take offline, but they?d still be in the config area. -- Robert P. Nix Mayo Foundation.~. RO-OE-5-55 200 First Street SW/V\ 507-284-0844 Rochester, MN 55905 /( )\ -^^-^^ In theory, theory and practice are the same, but in practice, theory and practice are different. On 3/30/10 11:21 AM, Eginhard Jaeger e.jae...@ch.inter.net wrote: This is usually caused by zVM accessing lots of additional I/O devices that it previously didn't see. While redefining the CONFIG size and the size of the MONDCSS can allow CP to again pack all of the available information into its monitor data, all of the additional monitor records must also be processed. The FCXPMN44E message means probably that PerfKit couldn't process the data fast enough, i.e. it needs a higher share of the CPU. But check first whether the problem is really caused by additional I/O devices. Is your VM seeing lots of devices belonging to other LPARs? Vary off the ones that are not needed, or tell the monitor via MONITOR command not to collect data for the ones you're not interested in. Eginhard My questions now are, What happened to the prior segment that caused it to fail? Could the problem have been avoided, and how? Also, now I?m getting the following two messages, repeatedly, and we?re still not collecting any data: FCXPMN444E IUCV reply failed with reason code 9 HCPMOV6274I The sample data messages and corresponding records have been purged. The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited. All messages sent to and from this e-mail address may be monitored as permitted by applicable law and regulations to ensure compliance with our internal policies and to protect our business. E-mails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, lost or destroyed, or contain viruses. You are deemed to have accepted these risks if you communicate with us by e-mail.
Re: [?? Probable Spam] Re: Perfkit SAMPLE CONFIG size too small
Not sure if my previous note on this was distributed. There is NO problem with having an oversized MONDCSS. A small one however stops you from collecting data. Our default is now 64mb. RPN01 wrote: Our MONDCSS grew, perhaps too large, while fighting this type message a long time ago. Once the problem was resolved, we didn't attempt to back off the changes we'd made, and the large size doesn't seem to hurt anything at the moment. I know that ultimately, making the segment larger was not the answer to the problem at the time, either. Also, Mr. Nunsford says hello.
