Re: SYSLOG purging . . .
John, I don't know if you are familiar to using OPERLOG. If not, I should suggest you ought to. I personally feel my life got better when we got rid of SYSLOG. In case you are, please take a look at CBT file #513, there you will find a set of tools to take advantage of OPERLOG. In few words, we use a schema of 10 days of log data online, and 45 days in migrated data sets. To exploit the data, you will find the tools to explote data by a lot of search criteria (date, sysname, task name, tso user name, character string, etc.) I hope it will help you. Regards, Jose Maria - LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y móviles desde 1 céntimo por minuto. http://es.voice.yahoo.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
It's not free but you may find a commercial archive product like $AVRS http://www.seasoft.com/ makes storing, searching, and managing retention of SYSLOG easier and would help you fill your requirements. Best Regards, Sam Knutson, GEICO Performance and Availability Management mailto:[EMAIL PROTECTED] (office) 301.986.3574 Think big, act bold, start simple, grow fast... ][ This email/fax message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution of this email/fax is prohibited. If you are not the intended recipient, please destroy all paper and electronic copies of the original message. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Thanks Jose, I will look into OPERLOG. Regards, Jon Bathmaker Senior Systems Programmer, Cornerstone Systems, Inc. 2 Robert Speck Pkwy, 280 Mississauga, ON L4Z-1H8 Desk: 905-854-4156 Cell: 647-400-4156 Office: 905-275-9977 Jose Martinez [EMAIL PROTECTED] To Sent by: IBM IBM-MAIN@BAMA.UA.EDU Mainframe cc Discussion List [EMAIL PROTECTED] Subject .EDU Re: SYSLOG purging . . . 03/14/2006 04:10 AM Please respond to IBM Mainframe Discussion List [EMAIL PROTECTED] .EDU John, I don't know if you are familiar to using OPERLOG. If not, I should suggest you ought to. I personally feel my life got better when we got rid of SYSLOG. In case you are, please take a look at CBT file #513, there you will find a set of tools to take advantage of OPERLOG. In few words, we use a schema of 10 days of log data online, and 45 days in migrated data sets. To exploit the data, you will find the tools to explote data by a lot of search criteria (date, sysname, task name, tso user name, character string, etc.) I hope it will help you. Regards, Jose Maria - LLama Gratis a cualquier PC del Mundo. Llamadas a fijos y móviles desde 1 céntimo por minuto. http://es.voice.yahoo.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
why couldn't you simply do 'w l' . then each day is a disticnt sysout/dsn/joe. then purge individual sysout that are 3 months old Jack Kelly LA Systems @ US Courts x 202-502-2390 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Hi All; Thanks for your help. The piece of the puzzle I was missing is that SDSF treats multiple SYSLOG data sets as a single logically contiguous Syslog. Which is what my user wants. So I just need to spin the syslog every week and after 3 months start writing the oldest one to a GDG. Thanks again. Regards, Jon Bathmaker Senior Systems Programmer, Cornerstone Systems, Inc. 2 Robert Speck Pkwy, 280 Mississauga, ON L4Z-1H8 Desk: 905-854-4156 Cell: 647-400-4156 Office: 905-275-9977 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Jon, One shop I was at we dumped SYSLOG every night at midnight to a DASD GDG. Then every week we used JOBTRAC (job scheduling product) to pickup the weeks work of GDGs and wrote a weekly GDG to DASD. Then we used Jobtrac to pick up the Weeklies and make a monthly tape. You can do something similar using SDSF Batch and a job scheduling product. It will not be pretty (Probably need a REXX or other program in the middle to determine the correct SYSLOG data set) but it can be done. If they just want 3 months worth of Syslog in a contigious group you can used the daily SYSLOG offload to a GDG that is mod'ed onto the end for 3 months. then create a new GDG. Lizette Koehler -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
-Original Message- From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On Behalf Of Jon Bathmaker Sent: Monday, March 13, 2006 3:08 PM To: IBM-MAIN@BAMA.UA.EDU Subject: SYSLOG purging . . . Hi All; I have an unusual request from my user. He wants to keep three months of SYSLOG on the spool in one logically contiguous piece and every week, he wants to dump the oldest week to a GDG, leaving the rest there in a single spool dataset. It's rather like a logger scenario. I have never heard of anyone wanting this, let alone implementing it but, hey, that's what makes life so interesting. One could imagine a program that does this, it would have to read and write the oldest week's records to the GDG and then copy the remaining records to the start of the file and reset the end of file marker, all without JES worrying his pretty little head about it. Fairly tricky I should think. Any thoughts/ideas/droll comments would be much appreciated. Thanks, Jon Bathmaker comment type=droll I'd lay very good odds that it is impossible. Especially if what the customer wants is something like: In March, I want JAN, FEB, and MAR syslog on the spool in a single spool dataset. In April, remove the JAN portion, leaving only FEB, MAR, and APR portion in the single spool dataset. There is simply NO way to purge part of a single SPOOL dataset. Curiously, if you use the OPERLOG using the LOGR, then you can do this. But it is not in the SPOOL, it is in the LOGGER datasets. And you simply remove those entries more than x days old. /comment -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Jon, you could probably go and invent some wheel, a program to work with the external writer program, to do what your user wants, plus some more to handle the retention, tape migration, expiration, etc. However, this wheel has already been invented many times over. Please look on the Internet for SYSOUT Archiver software. There are several software vendors out there willing to accept your good money and sell you a software package than can do what your user wants. I bet, there might also be a freebie on the CBT tape that you could use. In our shop, for example, we have been using CA-VIEW from Computer Associates for over 20 years. It archives the SYSLOG, among many other things (production joblogs, reports, etc), to an on-line database. We keep an on-line copy for a week and keep it on archive tape for 3 months. All the archiving, backup tape creation and expiration, all that stuff is handled by the software package. And all we have to do to get a daily SYSLOG spun off to the archive, is to have a an automatic WRITELOG command executed every night at midnight. Regards, Ulrich Krueger Mainframe Systems Services National Semiconductor Corp. Santa Clara, CA 95051 Tel: (408) 721-8071 Email: [EMAIL PROTECTED] -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Jon, on second thought ... First of all, you can not keep a contiguous hunk of SYSLOG (one big file for 90 days) and then expect to be able to easily delete a day's or week's worth of records. That's just not how the thing's designed. Depending on your system setup, SYSLOG is automatically broken into separate pieces when it exceeds the number of lines specified in SYS1.PARMLIB(IEASYSxx) LOGLMT=xx parameter. Instead of letting it grow to LOGLMT lines, you're much better off executing a WRITELOG x every 24 hours. This spins of a hunk of SYSLOG into an output dataset that you can work with and access with SDSF. If you don't want to invest in a software package (as I stated in my first posting) and you have plenty of diskspace available to add to your existing JES2 SPOOL configuration, to store 90 days' of SYSLOGs in addition to all your other stuff and you can afford to set aside a dedicated SYSOUT class for nothing but SYSLOGs and you have SDSF (and a _huge_ SDSF ISF.HASPINDX dataset!) then you can do the following: - Set up an automatic WRITELOG x command every night at midnight to spin off the daily SYSLOG to the reserved output class - Show the user how to use SDSF to locate SYSLOG information by date and time: In SDSF, go to the log, use the LOCATE hh:mm:ss mm/dd/ command to find the requested info by time and date ... slick stuff! - Finally set up an automatic purge command to purge all output from the reserved SYSOUT class that's older than 90 days: $POJOBQ,READY,Q=x,A90 - Since everything's kept in SPOOL, there'll be no need to handle GDG datasets, tapes, etc. That'll be the cheap solution, but please don't ask me how SDSF will perform when it has to maintain 90 days of SYSLOG in the HASPINDX dataset ... it all depends on size. Which brings up the question: How many lines of SYSLOG do you create in a day? (We're cranking out around 750,000 to 850,000 lines per day and that's why we offload from SPOOL to a SYSOUT Archiver software package.) HTH Regards, Ulrich Krueger Mainframe Systems Services National Semiconductor Corp. Santa Clara, CA 95051 Tel: (408) 721-8071 Email: [EMAIL PROTECTED] -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Jon, I too have to respond on the droll side of life. The SPOOL is meant as a temporary place to put print objects between creation and offload (to either a printer, an external file, another JES node, or the bit bucket). It is an enormous waste of DASD to keep SPOOL there any longer than necessary. This also begs the question: What do they want it for? If the only thing they are going to do is search it with their TSO ID, then educate them on the power of SUPERCE against disk datasets. What can you do with that much data, still on the SPOOL, that you can't do more efficiently with some other storage medium? And how fair is it to everyone else that wants to use the SPOOL? Not very, I think. Personally, I'd find out how much it costs to store that much data, find out if the SPOOL can be expanded that big, determine the cost and feasibility of doing so, and hand the user a bill for that amount. If they pay the bill up front, have a legitimate reason for needing it on the SPOOL, and it proves to be technically feasible then I'd likely set them up. However, I suspect that 3 months of SYSLOG won't fit on a JES SPOOL of maximum size even on mod-27 disk. I know at our site, we'd overrun such a SPOOL in just a couple weeks. As far as how to make the older data drop off, I'd have to refer you to the excellent suggestions that have already been sent to the list by our very sharp and clever posters. Best regards, Gary Diehl -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
In a message dated 3/13/2006 5:28:05 P.M. Central Standard Time, [EMAIL PROTECTED] writes: As far as how to make the older data drop off, I'd have to refer you to the excellent suggestions that have already been sent to the list by our very sharp and clever posters. Yeah don't know how much the end user is willing to pay for their indulgence. We did the ePrint solution couple years ago even for SYSLOG. The production control folks like it so they cay back track if something fails. It also converts to .pdf so if somebody wanted to build an index and glom so pages together that's there too. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Jon, JES2 or JES3? How many syslog records does the site produce on an average week? How many syslog records does the site produce in its busiest week? -- Tom Schmidt Madison, WI -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SYSLOG purging . . .
Three months of SYSLOG sounds rather excessive, unless the user is really willing to foot the bill for the extra JES SPOOL space, and the additional cost of backing up all that data for DR. We only keep about 3 weeks of SYSLOG online to JES and just that much is typically 5 - 10% of our total SPOOL space. Be sure you are segmenting off SYSLOG data (specify lines per segment in CONSOLxx ?) so that you typically get several segments per day. This makes it possible to use standard JES Purge commands (potentially from a scheduled batch job) to delete those segments that are older than x days, while the SDSF LOG function will still see all segments as a single SYSLOG file. Rather than attempt the scheme you describe, I would use a Sunday scheduled job with batch REXX to generate a sequence of commands with appropriate date values for a batch execution of SDSF including a PT 00.00.00 mm/dd/ 00.00.00 mm/dd/ with suitable dates to offload the previous week's SYSLOG data to DD that points to a GDG dataset, and then issue the JES2 purge command to delete SYSLOG segments older than x days, for x of your choice but greater than a week. At this point the most recent x days of data will both be in some offloaded GDG and in the still online JES2 SYSLOG segments. From SDSF all the remaining segments may be viewed as if they were a single time-ordered entity and the SDSF locate command can be used to quickly locate any date-time point within the remaining portion of SYSLOG. By offloading the most recent week each week instead of the oldest week, you don't have to worry about the imprecision in the point at which old SYSLOG data is purged (JES will purge at a segment boundary, which will not be at a consistent time from week to week). This should give the user convenient SDSF-accessible version of SYSLOG for the last x days plus GDG archives of all weekly SYSLOGs up to the GDG limit. No need to play games with copying remaining records, etc. This ought to satisfy the user's requirement with minimal cost. [EMAIL PROTECTED] wrote: Hi All; I have an unusual request from my user. He wants to keep three months of SYSLOG on the spool in one logically contiguous piece and every week, he wants to dump the oldest week to a GDG, leaving the rest there in a single spool dataset. It's rather like a logger scenario. I have never heard of anyone wanting this, let alone implementing it but, hey, that's what makes life so interesting. One could imagine a program that does this, it would have to read and write the oldest week's records to the GDG and then copy the remaining records to the start of the file and reset the end of file marker, all without JES worrying his pretty little head about it. Fairly tricky I should think. Any thoughts/ideas/droll comments would be much appreciated. Thanks, Jon Bathmaker Senior Systems Programmer, Cornerstone Systems, Inc. 2 Robert Speck Pkwy, 280 Mississauga, ON L4Z-1H8 Desk: 905-854-4156 Cell: 647-400-4156 Office: 905-275-9977 -- Joel C. Ewing, Fort Smith, AR[EMAIL PROTECTED] -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
