That is one possibility. Or they may not have known it was them, keying it in by accident or screwing up an E job,quiesce command, or we haven't asked the right person and so they didn't know they had done anything wrong.
-----Original Message----- From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of Bill Johnson Sent: Wednesday, March 24, 2021 9:28 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [External] Re: z14 HMC log information So the person who issued the quiesce isn’t honest enough to come forward? Sent from Yahoo Mail for iPhone On Wednesday, March 24, 2021, 10:03 AM, Carmen Vitullo <cvitu...@hughes.net> wrote: and there it is, most likely there's no time to research and really our jobs are to get the system back up, and when you have no clue what caused the wait state, IPL would have been my fist option, then diagnose the outage. Carmen Vitullo -----Original Message----- From: Rex <rpomm...@sfgmembers.com> To: IBM-MAIN <IBM-MAIN@LISTSERV.UA.EDU> Date: Wednesday, 24 March 2021 8:58 AM CDT Subject: Re: z14 HMC log information I'm going to agree with *most* of it. I don't like the part about killing the RACF admin. I'm not the one who initially set up the OPERCMDS security but I missed the fact the QUIESCE command wasn't set as "don't let anybody use". Hari-kari is not on my bucket list. :-) On to Radoslaw's comment about logging - it is logged, after the fact. QUIESCE does exactly that - it stops the LPAR in its tracks. Do not pass Go, do not collect $200. No z/OS logging at the time it happens. IBM hardware support found and reported the wait state back to us from some hardware logs that were forwarded to them from our CE. The z/OS logging takes place after the PSW restart from the HMC occurs and yes, it shows the console or user that executed the command. However in our case since the LPAR stopped in the middle of the day and we had managers breathing down our necks to get the system back up we didn't have time to properly diagnose until after the fact - which included an IPL which in turn did not allow the logging of the quiesce command to take place. Rex -----Original Message----- From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of Carmen Vitullo Sent: Wednesday, March 24, 2021 8:07 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [External] Re: z14 HMC log information agree 100%, when I tested the command on my sandbox system I see my ID in the syslog as the culprit :) if done from a console, then the console name is shown. Carmen Vitullo -----Original Message----- From: Radoslaw <r.skoru...@hotmail.com> To: IBM-MAIN <IBM-MAIN@LISTSERV.UA.EDU> Date: Wednesday, 24 March 2021 8:01 AM CDT Subject: Re: z14 HMC log information IMHO there should be a trace in a syslog. Maybe that part of syslog is somehow lost. And it would be good idea to have SMF record for that command. I don't know about console commands, but SMF80 could be cut if you take care about AUDIT(ALL(READ)) in advance. I mean RACF profile. Then find the person and kill him. Don't forget to kill RACF admin who forgot to protect the command. Kill'em all. ;-) Seriously: it is a symptom of bad security model. -- Radoslaw Skorupka (looking for new job) Lodz, Poland W dniu 24.03.2021 o 13:54, Pommier, Rex pisze: > Hi Carmen, > > We use the quiesce parameter of the reset command as well. I'm thinking > that's what happened here as well, but we have no idea "whodunit". We had 1 > operator on duty that day and he was off doing something else when it > happened so the command made it into the system via an SDSF-like console > session. > > Rex > > -----Original Message----- > From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On > Behalf Of Carmen Vitullo > Sent: Wednesday, March 24, 2021 7:43 AM > To: IBM-MAIN@LISTSERV.UA.EDU > Subject: [External] Re: z14 HMC log information > > There is a queisce command to put a job to sleep, and I think that's what my > operator was thinking he was doing, for a runaway job we do use automation to > queisce a job and send us an alert. > from SDSF there's a quiesce line command RQ it translates to RESET > jobname,QUIESCE > > > Carmen Vitullo > > > > -----Original Message----- > > From: Radoslaw <r.skoru...@hotmail.com> > To: IBM-MAIN <IBM-MAIN@LISTSERV.UA.EDU> > Date: Wednesday, 24 March 2021 5:35 AM CDT > Subject: Re: z14 HMC log information > > IMHO not really. > > By freezing the system you disrupt all the online processing and you loose > capability to find out what's wrong. > I had to do with CPU hogging tasks and spool 100% filled by some ugly job. In > both cases the very first thing to fix it was to logon to the system. > > -- > Radoslaw Skorupka > (looking for new job) > Lodz, Poland > > > > W dniu 24.03.2021 o 03:30, kekronbekron pisze: >> It's useful when you want to say "STOP IT!" to run-away CPU or spool jobs. >> Will buy you some time to investigate. >> >> - KB >> >> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >> On Tuesday, March 23, 2021 7:07 PM, Pommier, Rex <rpomm...@sfgmembers.com> >> wrote: >> >>> Hey Carmen, >>> >>> At least you had an operator tell you what they did. :-) Once we determined >>> what had happened (after a painful middle-of-the-day IPL) the only response >>> I got back from anybody was "I didn't do anything like that!" But now that >>> I know what that command does and how it works I doubt I'll ever forget >>> this one. >>> >>> Rex >>> >>> -----Original Message----- >>> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On >>> Behalf Of Carmen Vitullo >>> Sent: Tuesday, March 23, 2021 7:47 AM >>> To: IBM-MAIN@LISTSERV.UA.EDU >>> Subject: Re: [External] Re: z14 HMC log information >>> >>> WOW! this takes me back 20 years when an operator wanted to quiesce >>> a job and issues that command from the console he called my told me >>> what he did, I researched what that command does, seemed to me it >>> was like on the old system consoles O2 and O3 IIRC one was a PSW >>> stop and one a restart, I found the restart just for grins and it >>> worked, been a long time since I've hear tell someone had that same >>> experience great find >>> >>> Carmen Vitullo >>> >>> -----Original Message----- >>> >>> From: Rex rpomm...@sfgmembers.com >>> To: IBM-MAIN IBM-MAIN@LISTSERV.UA.EDU >>> Date: Monday, 22 March 2021 5:18 PM CDT >>> Subject: Re: [External] Re: z14 HMC log information >>> >>> OK, this is embarrassing. We found the problem - but don't know the >>> culprit. We discovered the quiesce command wasn't locked down and >>> somebody somewhere keyed it in. The reason we don't know who/where >>> is because our SDSF-like product had the capability to also do a >>> quiesce and since we didn't know that's what caused it, we IPLed the >>> LPAR. I did some research after the fact to determine just how >>> quiesce works (been working on these things for 30 years and never >>> had a reason to even play with quiesce). Now I know - and I know how >>> to make the machine go again, picking up where it stopped. PSW >>> Restart from the HMC doesn't do what I consider a "restart" but a >>> "take the brakes off and let it run". :-) >>> >>> IBM hardware support found the wait state and informed us of it. >>> >>> Thanks everybody for the bandwidth and suggestions. >>> >>> Rex >>> >>> -----Original Message----- >>> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On >>> Behalf Of kekronbekron >>> Sent: Wednesday, March 17, 2021 8:50 AM >>> To: IBM-MAIN@LISTSERV.UA.EDU >>> Subject: [External] Re: z14 HMC log information >>> >>> Please do share when you find out! >>> >>> I wonder if an LPAR with full BCPii authority over the box can silently >>> query/log information from the HMC for monitoring, i.e., actions occuring >>> not via BCPii itself, but just accessing HMC logs. >>> >>> - KB >>> >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>> On Wednesday, March 17, 2021 6:17 PM, Carmen Vitullo cvitu...@hughes.net >>> wrote: >>> >>> >>>> so the only other thing I can think of is if you are in a sysplex and your >>>> SFM policy took the system out of the plex, XCF may have not seen a >>>> heartbeat in a long enough time it partitioned the system from the plex, >>>> I've had this happen before, you should see some IXC messages if this >>>> happened. >>>> when this happened to me there was no indication anyone did >>>> anything the system was just removed from the plex >>>> >>>> Carmen Vitullo >>>> -----Original Message----- >>>> From: Rex rpomm...@sfgmembers.com >>>> To: IBM-MAIN IBM-MAIN@LISTSERV.UA.EDU >>>> Date: Tuesday, 16 March 2021 4:33 PM CDT >>>> Subject: Re: z14 HMC log information Nothing out of the ordinary in >>>> the HMC logs. >>>> -----Original Message----- >>>> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On >>>> Behalf Of Carmen Vitullo >>>> Sent: Tuesday, March 16, 2021 1:10 PM >>>> To: IBM-MAIN@LISTSERV.UA.EDU >>>> Subject: [External] Re: z14 HMC log information so it looks like >>>> one place to check is from the HMC, /console actions/View control >>>> tasks performed, this may get you what you need another place is >>>> from console actions, View console events >>>> >>>> Carmen Vitullo >>>> -----Original Message----- >>>> From: Rex rpomm...@sfgmembers.com >>>> To: IBM-MAIN IBM-MAIN@LISTSERV.UA.EDU >>>> Date: Tuesday, 16 March 2021 12:51 PM CDT >>>> Subject: Re: z14 HMC log information I found the log - nothing in >>>> it. Heading off to IBM. >>>> Rex >>>> -----Original Message----- >>>> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On >>>> Behalf Of Pommier, Rex >>>> Sent: Tuesday, March 16, 2021 12:24 PM >>>> To: IBM-MAIN@LISTSERV.UA.EDU >>>> Subject: [External] z14 HMC log information Hi all, Probably a >>>> simple question but I'm not having any luck looking for it so I'm asking >>>> here. >>>> Is there a centralized place on a z14 HMC to show activity occurring, LPAR >>>> activation/deactivation, pretty much anything that happens on an HMC or on >>>> a z. We just took a hit where one of our LPARs and we're not seeing >>>> anything. No hardware messages, no dumps, no nothing. The syslog on the >>>> LPAR shows everything running as normal then it just stopped, 15 minutes >>>> later are the IPL starting messages. It almost looks like somebody hit the >>>> "deactivate" button on the HMC so I'm looking for any log info from a >>>> hardware point of view. >>>> Thanks, >>>> Rex >>>> > ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN The information contained in this message is confidential, protected from disclosure and may be legally privileged. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any disclosure, distribution, copying, or any action taken or action omitted in reliance on it, is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to this message and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN The information contained in this message is confidential, protected from disclosure and may be legally privileged. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any disclosure, distribution, copying, or any action taken or action omitted in reliance on it, is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to this message and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN