Re: SMF in logstream and IFASMFDL
On Sun, 23 Mar 2008 10:10:06 -0400, Peter Relson <[EMAIL PROTECTED]> wrote: >To be fair and complete, it has clearly been stated that IBM understands >that the lack of a delete/clear function is considered to be a problem and >we hope to provide a solution. Should be easy enough... just look at IEAMDBLG for Operlog. Regards, Art Gutowski Ford Motor Company [EMAIL PROTECTED] -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
On Sun, 23 Mar 2008 14:42:51 -0400, Knutson, Sam <[EMAIL PROTECTED]> wrote: > >We deal with large volumes of SMF data and sometimes get floods of SMF >data due to looping transactions, DB2 traces, or other diagnostic >captures or application errors. This is the part that concerns me most (regardless of the other dumping issues). I think I posted this before... In the past year or so there have been 3 instances of SMF collection / dumping problems. 1 was a CICS transaction looping, one was a MQ trigger for CICS transaction "looping", and the 3rd was in the last month or so - a DB2 accounting trace (the DB2 team had no clue what it was going to do to SMF so they didn't warn us - guess IBM should have warned them). > >I think SMF to LOGR solves more problems than it creates and I intend to >try it on our test Sysplex as soon as we have z/OS 1.9 up and running >stable there. > I agree that it has that potential. But it doesn't create any problems if you don't use it. :-) Seriously... in a smaller environment like a test sysplex I think it will be great. I plan on doing the same shortly and just managing it with RETPD like I do operlog and LOGREC. But I don't need to worry about archiving it and using it for production processes like billing. It's there in my sandbox to be only "as needed". >My biggest concern about SMF to LOGR is how I can easily detect sudden >changes in profile in data recording or flooding. Exactly. In all of the cases I mentioned above the SMF offloads had trouble keeping up with MANx data sets filling up and operations noticed all the contention messages and alerted us. If this data was all going to our SMS LOGR pool, the first call would go to our DASD storage group when the pool ran out of space (or when HSM ML1 ran out of space depending on how we might archive these logstreams). By they time they responded or figured out what was going on, it would probably be too late since this would of course affect all LOGR applications (this is not like a batch space abend that could wait to be addressed). That would not be a good thing - especially if RRS was affected.Many years ago we ran into that same situation when LOGREC went wild, but it was a much smaller environment and we hadn't planned as well for an abnormal situation. Here, I would probably have a hard time justifying "n" times the amount of needed DASD just sitting doing nothing most of the time in order to handle some abnormal situation should it come up. Today "n" doesn't matter much since the LOGR pool isn't that big (it's about 30% full as I look right now). I just keep trying to picture the 3 times we've had problems in the last year or so and what that would have looked like with SMF going to logger. In one of those cases I know we created about 200 tape GDGs when all was said and done. Looking at a few SMF virtual tapes right now, they are about 1G (uncompressed) each. I'm rambling now, but I think you can see why I'm not ready to jump on this. > I have requirements >against CA-SYSVIEW but I think this is an area IBM should address in a >similar way to what they have done with the WTO Flood processing and >what is being discussed in the statement of direction for autonomic >detection of common storage use out of profile. I would love to see IBM >provide an exit that could alert on out of profile SMF recording. At >the minimum IBM could provide a new record or exit on an interval with a >COUNT/RATE for active address spaces, system COUNT/RATE, SMF >type-subtype COUNT/RATE. This would allow an installation to more >easily become aware of flooding/looping conditions causing large volumes >of wasteful data to dump into SMF that will be a problem both in DASD >use by LOGR and in downstream processing. > Sounds like a great idea. You could try and use FULLTHRESHOLD(n) in the CFRM policy perhaps, but we already use FULLTHRESHOLD(0) for similar logstreams (logrec, operlog). See past rants... er.. I mean posts from Barabara Nitz on this subject. At any rete... this is all very new and I think we'll (us users and IBM) have to learn as we go along. Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
Hi, IBM certainly understands the requirement from customers to have IFASMFDL include a delete or clear function. I think it is a serious mistake to dismiss the entire facility waiting for this functionality. We recently ordered z/OS 1.9 and I have plans to use SMF to LOGR rather than MAN data sets on 1.9 without any further enhancements to IFASMFDL. I will let you know how this works out here and at SHARE. I think I can leverage what IBM has already provided to eliminate several operational problems and daily redundant data movement. Maybe you can too just take the time to consider how what is provided might work for you. We deal with large volumes of SMF data and sometimes get floods of SMF data due to looping transactions, DB2 traces, or other diagnostic captures or application errors. In order to insure that we don't lose data of record every LPAR is provisioned with 8 large (49950 tracks) MAN data sets. Normally we only toggle between 2 dumping and filling but 8 are available before we begin buffering data. Automation alerts and then alerts louder if these start filling and finally if they fill too many stop recording "tourist information". Performance data while of interest to the performance analyst is not required for auditing or sub-capacity billing. In 9 out of 10 instances shutting off everything but core data required to recover catalog, report on data access, process SCRT billing, etc. will prevent us from losing data while staff intervene. All data is not created equal you need to distinguish between data of record and data that is interesting but not business critical. We dump each LPAR MAN data sets to a GDG for that LPAR that are compressed and striped. Daily all of the LPARs are condensed to a days worth of data separated into files of CICS, DB2, WLM, MQ, etc. So data has been written and read twice before it is every used or even if it is never used. For reasons previously explained I retain WLM type 99 records for at least 10 days in case we need them which means even though I don't process them normally they are all written and read at least twice. The daily job that reads all the LPAR data does not start till 00:30 to make sure the last dumps for the data have completed. Those dumps have to post the job scheduler to make sure that we get job demanded in only when we actually get the last data. This is a Rube Goldberg contraption though one that works well. The single biggest benefit of SMF to LOGR as provided is reduction in complexity. Let's look at what I am going to get just by using what IBM has provided so far. Today I have an IEFU29 exit and while IBM has provided a similar exit point I believe that I will no longer need any equivalent exit. IEFU29 will be retired. This is a positive since the business prefers to have as few user modifications and assembler exits as possible. Today I write and read data that is only recorded in case it is required several times wasting processing resources I/O and CPU. In the world of LOGR I can place type 99 data in a separate logstream which will never normally be read again unless we want to do an analysis or gather data for IBM. Today I write and read all data several times before it is read to be processed. I can eliminate one read and write without changing any upstream processes. Since we use compression to conserve DASD this saves both CPU and I/O bandwidth that is being used just to move data from one interim storage location to another without actually getting any benefit. Today I have to wait after midnight to allow for everything to normally be ready. I can now start my daily processing of SMF data at 00:01 saving 29 minutes off the existing schedule. I plan to allow a full day's worth of data to be recorded to LOGR then read once just after midnight and written into a GDG just as today. Only the data that is needed to begin processing MICS, MXG, CPExpert, MVPA, SCRT, reports needs to be extracted before this scan begin. I save lots of MQ type 115/116 data but MICS does not need it so the extraction of that data can be uncoupled from the start of MICS processing. So long as I extract once a day for the full days time the lack of a clear/delete function is not a problem. I may even save DASD since I no longer house the interim LPAR GDG and the mostly empty MAN data sets (1 3390-27 @ LPAR) but instead only the actual data stays in LOGR till expired which we will probably make 2 to 3 days to insure we got the extract done. CA-SYSVIEW includes utility function to trim a logstream which we can use if we need to cleanup after something writes 100 million bad records into SMF from a transaction loop. I think SMF to LOGR solves more problems than it creates and I intend to try it on our test Sysplex as soon as we have z/OS 1.9 up and running stable there. My biggest concern about SMF to LOGR is how I can easily detect sudden changes in profile in data recording or flooding. I have requirements against
Re: SMF in logstream and IFASMFDL
Peter Relson wrote: To be fair and complete, it has clearly been stated that IBM understands that the lack of a delete/clear function is considered to be a problem and we hope to provide a solution. To be even more complete, it should be noted that this requirement had been articulated to Mr. Jones in every disclosure meeting I can recall in which this function was presented. -- Edward E Jaffe Phoenix Software International, Inc 5200 W Century Blvd, Suite 800 Los Angeles, CA 90045 310-338-0400 x318 [EMAIL PROTECTED] http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
On Sat, 22 Mar 2008 23:22:34 -0700, Skip Robinson <[EMAIL PROTECTED]> wrote: >This is a big step in a hopeful direction. It certainly is. But if you read past posts on this subject (including mine), I agree with Radoslow that SMF logger is still not ready for prime time. I'm sure it will get there though - hopefully sooner than later. I haven't seen your SHARE pitch yet, but I'll be sure to get to it next week. Cheers, Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group mailto: [EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
To be fair and complete, it has clearly been stated that IBM understands that the lack of a delete/clear function is considered to be a problem and we hope to provide a solution. Peter Relson z/OS Core Technology Design -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
Radoslaw, Despite aforementioned warts, I stand by my recommendation to check this puppy out of the pound for a trial weekend at home. Spread out newspapers, warn domestic partners of unseemly behaviors, and settle in for some exchange of exuberant pet breath and saliva. The shortcomings of the MANx mantra are legion and legendary. This is a big step in a hopeful direction. "R.S." <[EMAIL PROTECTED] LTIBANK.COM.PL>To Sent by: IBM IBM-MAIN@bama.ua.edu Mainframe cc Discussion List <[EMAIL PROTECTED] Subject .edu> Re: SMF in logstream and IFASMFDL 03/22/2008 09:39 AM Please respond to IBM Mainframe Discussion List <[EMAIL PROTECTED] .edu> Ken, I'm aware how should I delete tunnel-type logstream datasets (by adjusting the logstream definitions, not manual deletion of VSAM files). However I asked about IFASMFDL role in this process. Nevermind, thank you for the interest. According to Skip's answer it's tunnel-type logstream and IFASMFDL is neither aware was already offloaded nor it does delete records from logstream. So, my humble opinion is "no, thanks". YMMV. Regards -- Radoslaw Skorupka Lodz, Poland Kenneth E Tomiak wrote: > The current complaints are you do not clear the logstream. A problem for > shops that do not IPL for months on end. I have not heard of anyone forcibly > reming logstreams, System Logger rules! > > > On Sat, 22 Mar 2008 13:13:18 +0100, R.S. > <[EMAIL PROTECTED] could snip this!> wrote: > >> I'm trying to learn more about new feature: SMF in logstream instead of >> SYS1.MANx. >> Q: Does IFASMFDL utility clear logstream, or it is tunnel-type log >> (deletion of oldest entries is up to logstream administrator) ? -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2008 r. kapita zakadowy BRE Banku SA wynosi 118.642.672 zote i zosta w caoci wpacony. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
Ken, I'm aware how should I delete tunnel-type logstream datasets (by adjusting the logstream definitions, not manual deletion of VSAM files). However I asked about IFASMFDL role in this process. Nevermind, thank you for the interest. According to Skip's answer it's tunnel-type logstream and IFASMFDL is neither aware was already offloaded nor it does delete records from logstream. So, my humble opinion is "no, thanks". YMMV. Regards -- Radoslaw Skorupka Lodz, Poland Kenneth E Tomiak wrote: The current complaints are you do not clear the logstream. A problem for shops that do not IPL for months on end. I have not heard of anyone forcibly reming logstreams, System Logger rules! On Sat, 22 Mar 2008 13:13:18 +0100, R.S. <[EMAIL PROTECTED] could snip this!> wrote: I'm trying to learn more about new feature: SMF in logstream instead of SYS1.MANx. Q: Does IFASMFDL utility clear logstream, or it is tunnel-type log (deletion of oldest entries is up to logstream administrator) ? -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2008 r. kapita zakadowy BRE Banku SA wynosi 118.642.672 zote i zosta w caoci wpacony. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
At the behest of Sam Knutson and the SHARE MVSE Project, I delved into SMF with logger in time to pull together a user pitch in Orlando: Session 2850 , where I tout virtues and expose warts, which IBM informally acknowledge but as yet have no specific solutions to announce. The pitch includes references to an IBM session on the subject presented first in San Diego and repeated in Orlando. Also pointers to some manuals I consulted. One problem I noted with managing Logger for SMF is the large amount of DASD space that could be tied up to contain just one day's worth of data. (System Logger retains data for a minimum of one day.) It was pointed out to me after my session that offload data sets can be migrated. I had read that comment in the doc but never pursued how it would work in practice. SMF data can be offloaded as often as you choose. I wrote a Rexx that dumps data to tape (for example) as often as once an hour. The trouble with the Logger implementation is that dumped data is not 'marked'. The only way to avoid dumping the same data over and over is to adjust the start/end values in IFASMFDL. In any case, the data lives on until System Logger expires it. Nonetheless, I believe that SMF with Logger provides enough benefit to make it worth the exercise. . . JO.Skip Robinson Southern California Edison Company Electric Dragon Team Paddler SHARE MVS Program Co-Manager 626-302-7535 Office 323-715-0595 Mobile [EMAIL PROTECTED] Kenneth E Tomiak To Sent by: IBM IBM-MAIN@bama.ua.edu Mainframe cc Discussion List <[EMAIL PROTECTED] Subject .edu> Re: SMF in logstream and IFASMFDL 03/22/2008 07:00 AM Please respond to IBM Mainframe Discussion List <[EMAIL PROTECTED] .edu> The current complaints are you do not clear the logstream. A problem for shops that do not IPL for months on end. I have not heard of anyone forcibly reming logstreams, System Logger rules! On Sat, 22 Mar 2008 13:13:18 +0100, R.S. <[EMAIL PROTECTED]> wrote: >I'm trying to learn more about new feature: SMF in logstream instead of >SYS1.MANx. >Q: Does IFASMFDL utility clear logstream, or it is tunnel-type log >(deletion of oldest entries is up to logstream administrator) ? > >Happy Easter >-- >Radoslaw Skorupka >Lodz, Poland -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SMF in logstream and IFASMFDL
The current complaints are you do not clear the logstream. A problem for shops that do not IPL for months on end. I have not heard of anyone forcibly reming logstreams, System Logger rules! On Sat, 22 Mar 2008 13:13:18 +0100, R.S. <[EMAIL PROTECTED]> wrote: >I'm trying to learn more about new feature: SMF in logstream instead of >SYS1.MANx. >Q: Does IFASMFDL utility clear logstream, or it is tunnel-type log >(deletion of oldest entries is up to logstream administrator) ? > >Happy Easter >-- >Radoslaw Skorupka >Lodz, Poland -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
