VM Dump Quandries
Large systems can be a challenge... Problem #1 -- Dump time. Customer took an abend on a 190GB VM LPAR. VM started dumping. (We assume it was to be a small dump -- just CP areas and the frame table.) Dumping at the rate of 25% every 6 minutes... (And we're dumping...) The customer IPLed at 12 minutes into the dump because they needed to get their production systems back up. Have any of the other large LPAR shops dealt with this? (Other than SET DUMP OFF ;-)Telling a customer to leave his system down for half an hour for a dump isn't popular... Problem #2 -- Dump size. CP seems to allocate about 17GB for dump space out of the spool. Then to try and process the dump I'd need something like a mod-27 as a single large CMS minidisk... (Or a giant SAN LUN.) Then what? If I'm going to send the dump to IBM, I doubt FTPing a 20GB file is a choice... Back to good old tape? #2a -- does IBM accept huge SPXTAPE DUMPs? Thoughts and suggestions (other than make the LPARs smaller) welcome... Lee -- Lee Stewart, Senior SE Sirius Computer Solutions Phone: (303) 996-7122 Email: lee.stew...@siriuscom.com Web: www.siriuscom.com
Re: VM Dump Quandries
On Wednesday, 11/04/2009 at 02:44 EST, Lee Stewart lstewart.dsgr...@attglobal.net wrote: Problem #1 -- Dump time. Customer took an abend on a 190GB VM LPAR. VM started dumping. (We assume it was to be a small dump -- just CP areas and the frame table.) Dumping at the rate of 25% every 6 minutes... (And we're dumping...) The customer IPLed at 12 minutes into the dump because they needed to get their production systems back up. Have any of the other large LPAR shops dealt with this? (Other than SET DUMP OFF ;-)Telling a customer to leave his system down for half an hour for a dump isn't popular... Then don't. If time-to-recovery is less than time-to-obtain-diagnostics, then you can't really depend on a single LPAR. I would bring up my other z/VM LPAR (cold, warm, or hot) and let it start everyone while the other system dumps. I would probably add some automation in AUTOLOG1 using XLINK-formatted shared dasd that would atttempt to determine if the other LPAR was up and, if so, don't start any guests. In the event of an abend, then you have to manually intervene and use XLINK RESET on the to remove links held by the failed system. Problem #2 -- Dump size. CP seems to allocate about 17GB for dump space out of the spool. Then to try and process the dump I'd need something like a mod-27 as a single large CMS minidisk... (Or a giant SAN LUN.) Then what? If I'm going to send the dump to IBM, I doubt FTPing a 20GB file is a choice... Back to good old tape? #2a -- does IBM accept huge SPXTAPE DUMPs? We accept whatever you can get to us. z/VM 6.1 has a new DUMPLD2 utility that can break a dump into pieces as it loads to one or more minidisks. Then you can ftp each piece to us individually. Alan Altmark z/VM Development IBM Endicott
Re: VM Dump Quandries
We have a 110G VM LPAR. Dumps, however rarely they may be taken, do take time and space. We have 2 mod-09s specified as DUMP in SYSTEM CONFIG. Right now, 53% of one of the dump packs is allocated; the other, 0%. DUMPLOAD requires a full mod-09, as well. VMARC PACK of the dump, once loaded, gives considerably better compression than does COPYFILE. The last time that I sent a VMARC packed dump to IBM, it actually made it without error. IIRC, it has been awhile since I last sent a dump to them, it took about an hour. I have had problems FTPing big dumps to other vendors. VSSI has the ability to process partial dumps, so I have not had to break a dump into pieces for them - they usually find the problem in the first part that they receive. The operators do get antsy about the time it takes for the system to complete a dump and restart, but they are getting used to big systems taking a long time to dump. I would say that here, the priorities are, if possible, (1) speed up the time to take the dump and (2) reduce the size of the dumps. The 2 are not mutually exclusive. Regards, Richard Schuh -Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Lee Stewart Sent: Wednesday, November 04, 2009 11:44 AM To: IBMVM@LISTSERV.UARK.EDU Subject: VM Dump Quandries Large systems can be a challenge... Problem #1 -- Dump time. Customer took an abend on a 190GB VM LPAR. VM started dumping. (We assume it was to be a small dump -- just CP areas and the frame table.) Dumping at the rate of 25% every 6 minutes... (And we're dumping...) The customer IPLed at 12 minutes into the dump because they needed to get their production systems back up. Have any of the other large LPAR shops dealt with this? (Other than SET DUMP OFF ;-)Telling a customer to leave his system down for half an hour for a dump isn't popular... Problem #2 -- Dump size. CP seems to allocate about 17GB for dump space out of the spool. Then to try and process the dump I'd need something like a mod-27 as a single large CMS minidisk... (Or a giant SAN LUN.) Then what? If I'm going to send the dump to IBM, I doubt FTPing a 20GB file is a choice... Back to good old tape? #2a -- does IBM accept huge SPXTAPE DUMPs? Thoughts and suggestions (other than make the LPARs smaller) welcome... Lee -- Lee Stewart, Senior SE Sirius Computer Solutions Phone: (303) 996-7122 Email: lee.stew...@siriuscom.com Web: www.siriuscom.com
Re: VM Dump Quandries
Well, if you need to be more highly available and can't wait for a dump on a VM system, you need more than 1 VM system ;) I wonder if IBM could speed up the dump writing time? 24 minutes seems like a long time to write 17GB. It's probably writing 1 4k block at a time :) Marcy This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Lee Stewart Sent: Wednesday, November 04, 2009 11:44 AM To: IBMVM@LISTSERV.UARK.EDU Subject: [IBMVM] VM Dump Quandries Large systems can be a challenge... Problem #1 -- Dump time. Customer took an abend on a 190GB VM LPAR. VM started dumping. (We assume it was to be a small dump -- just CP areas and the frame table.) Dumping at the rate of 25% every 6 minutes... (And we're dumping...) The customer IPLed at 12 minutes into the dump because they needed to get their production systems back up. Have any of the other large LPAR shops dealt with this? (Other than SET DUMP OFF ;-)Telling a customer to leave his system down for half an hour for a dump isn't popular... Problem #2 -- Dump size. CP seems to allocate about 17GB for dump space out of the spool. Then to try and process the dump I'd need something like a mod-27 as a single large CMS minidisk... (Or a giant SAN LUN.) Then what? If I'm going to send the dump to IBM, I doubt FTPing a 20GB file is a choice... Back to good old tape? #2a -- does IBM accept huge SPXTAPE DUMPs? Thoughts and suggestions (other than make the LPARs smaller) welcome... Lee -- Lee Stewart, Senior SE Sirius Computer Solutions Phone: (303) 996-7122 Email: lee.stew...@siriuscom.com Web: www.siriuscom.com
Re: VM Dump Quandries
The man has LPARs, and their associated memory, coming out the wazoo! What other VM LPAR? All the memory that we have available is allocated to the one that is dumping. Regards, Richard Schuh I would bring up my other z/VM LPAR (cold, warm, or hot) and let it start everyone while the other system dumps. I would probably add some automation in AUTOLOG1 using XLINK-formatted shared dasd that would atttempt to determine if the other LPAR was up and, if so, don't start any guests. In the event of an abend, then you have to manually intervene and use XLINK RESET on the to remove links held by the failed system. Alan Altmark z/VM Development IBM Endicott
Re: VM Dump Quandries
I was going to say... who has another 190G lpar sitting around doing nothing! That's over a $1M! Marcy This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. -Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Schuh, Richard Sent: Wednesday, November 04, 2009 12:34 PM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: [IBMVM] VM Dump Quandries The man has LPARs, and their associated memory, coming out the wazoo! What other VM LPAR? All the memory that we have available is allocated to the one that is dumping. Regards, Richard Schuh I would bring up my other z/VM LPAR (cold, warm, or hot) and let it start everyone while the other system dumps. I would probably add some automation in AUTOLOG1 using XLINK-formatted shared dasd that would atttempt to determine if the other LPAR was up and, if so, don't start any guests. In the event of an abend, then you have to manually intervene and use XLINK RESET on the to remove links held by the failed system. Alan Altmark z/VM Development IBM Endicott
Re: VM Dump Quandries
We accept whatever you can get to us. Fan-fold hardcopy? Get a chainsaw, Chuckie... we're gonna need to cut down and process a forest or two (or tree - pun intended)! ;-) Mike Walter Hewitt Associates The opinions expressed herein are mine alone, not my employer's. Alan Altmark alan_altm...@us.ibm.com Sent by: The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU 11/04/2009 02:14 PM Please respond to The IBM z/VM Operating System IBMVM@LISTSERV.UARK.EDU To IBMVM@LISTSERV.UARK.EDU cc Subject Re: VM Dump Quandries On Wednesday, 11/04/2009 at 02:44 EST, Lee Stewart lstewart.dsgr...@attglobal.net wrote: Problem #1 -- Dump time. Customer took an abend on a 190GB VM LPAR. VM started dumping. (We assume it was to be a small dump -- just CP areas and the frame table.) Dumping at the rate of 25% every 6 minutes... (And we're dumping...) The customer IPLed at 12 minutes into the dump because they needed to get their production systems back up. Have any of the other large LPAR shops dealt with this? (Other than SET DUMP OFF ;-)Telling a customer to leave his system down for half an hour for a dump isn't popular... Then don't. If time-to-recovery is less than time-to-obtain-diagnostics, then you can't really depend on a single LPAR. I would bring up my other z/VM LPAR (cold, warm, or hot) and let it start everyone while the other system dumps. I would probably add some automation in AUTOLOG1 using XLINK-formatted shared dasd that would atttempt to determine if the other LPAR was up and, if so, don't start any guests. In the event of an abend, then you have to manually intervene and use XLINK RESET on the to remove links held by the failed system. Problem #2 -- Dump size. CP seems to allocate about 17GB for dump space out of the spool. Then to try and process the dump I'd need something like a mod-27 as a single large CMS minidisk... (Or a giant SAN LUN.) Then what? If I'm going to send the dump to IBM, I doubt FTPing a 20GB file is a choice... Back to good old tape? #2a -- does IBM accept huge SPXTAPE DUMPs? We accept whatever you can get to us. z/VM 6.1 has a new DUMPLD2 utility that can break a dump into pieces as it loads to one or more minidisks. Then you can ftp each piece to us individually. Alan Altmark z/VM Development IBM Endicott The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited. All messages sent to and from this e-mail address may be monitored as permitted by applicable law and regulations to ensure compliance with our internal policies and to protect our business. E-mails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, lost or destroyed, or contain viruses. You are deemed to have accepted these risks if you communicate with us by e-mail.
Re: VM Dump Quandries
DUMPLD2 will help, when we get to 6.1... All the Linux guests are SAN-only (no 3390), so no XLINK. And since it's all NPIV enabled, it's non-trivial (both on VM and the storage unit) to bring up a guest in a different LPAR.. Especially to get multi-path back.. There are multiple LPARs already, mostly full.. And no spare 190GB.. ;-) And they are moving to more of an HA environment. But the Titanic, uh uh uh I mean the Queen Mary doesn't turn quickly... Lee Alan Altmark wrote: Then don't. If time-to-recovery is less than time-to-obtain-diagnostics, then you can't really depend on a single LPAR. I would bring up my other z/VM LPAR (cold, warm, or hot) and let it start everyone while the other system dumps. I would probably add some automation in AUTOLOG1 using XLINK-formatted shared dasd that would atttempt to determine if the other LPAR was up and, if so, don't start any guests. In the event of an abend, then you have to manually intervene and use XLINK RESET on the to remove links held by the failed system. Problem #2 -- Dump size. CP seems to allocate about 17GB for dump space out of the spool. Then to try and process the dump I'd need something like a mod-27 as a single large CMS minidisk... (Or a giant SAN LUN.) Then what? If I'm going to send the dump to IBM, I doubt FTPing a 20GB file is a choice... Back to good old tape? #2a -- does IBM accept huge SPXTAPE DUMPs? We accept whatever you can get to us. z/VM 6.1 has a new DUMPLD2 utility that can break a dump into pieces as it loads to one or more minidisks. Then you can ftp each piece to us individually. Alan Altmark z/VM Development IBM Endicott -- Lee Stewart, Senior SE Sirius Computer Solutions Phone: (303) 996-7122 Email: lee.stew...@siriuscom.com Web: www.siriuscom.com
