Hi, Jason. If all you’re looking to do is summarize log data, I can give you some options, at least.
There was/is a tool that Kevin Kelley, since retired, wrote to help clients do log analysis that would do exactly what you want, on platform, in batch; I’m trying to find it again. It was available on testcase.boulder.ibm.com, and may well still be there in some subdirectory; I’ll spend a bit of time searching for it again. Links to it have apparently been lost. I believe it was mentioned in some Redbooks as well. If I can find it, I’ll put it in the z/OS github repository. It was intended to help clients do message suppression and the like. I have a python library, zoslogs (https://pypi.org/project/zoslogs/), that I built to do very basic log parsing. So if you wanted to, you could use it as a basis for something like what you’re trying to do; you at least wouldn’t have to worry about doing the initial parsing yourself. There are some tools that might give you an easier interface to gather the syslog data, such as the z Common Data Provider, or the z/OSMF REST API. I believe zoau does something similar. There are also things like IBM Z Operational Log Analytics and IBM Z Anomaly Analytics with Watson that will do realtime/almost realtime analysis of the logs for you, and try to warn you about potential issues. As others said, however, I don’t think using something like DFSORT will meet your needs, without a lot of work. At the least I’d suggest using a pre-processing tool to handle the initial log parsing before you sort/summarize the data. -- Kevin McKenzie External Phone: 845-435-8282, Tie-line: 8-295-8282 z/OS Test Services - Test Architect, Provisioning z/OS Hardware/Software Interlock From: Jason Cai <ibmm...@foxmail.com> Date: Thursday, July 27, 2023 at 3:07 AM To: IBM-MAIN@LISTSERV.UA.EDU <IBM-MAIN@LISTSERV.UA.EDU>, Kevin Mckenzie <kmcke...@us.ibm.com> Cc: Jason Cai <ibmm...@foxmail.com> Subject: [EXTERNAL] Re: Inquiry about extracting and counting msgid from operlog using sort program Dear Massimo and Kolusu, Thank you for your help. I got the results I wanted. The results are as follows. MSGID NUMREC $HASP000 20 $HASP001 10 .............. ---->DISTINCT COUNT: 530 I also appreciate Kevin McKenzie's reply. The reason why I counted how many times each unique msgid appeared in a day is to compare today's operlog with yesterday's operlog and see which msgids are new and which ones increased more than yesterday, such as by 30%. I got a list of msgids that need special attention. I used this list to extract the specific content of today's OPERLOG to find out the possible risks of the system in time. I have two questions: 1.The first question is whether there is a better way or tool to analyze operlog. 2. The second question is whether sort can delete the numbers marked by line breaks in the multi-line records of MSGID to facilitate deduplication of multiple lines. See the information below: •M.... 00000090 HZS0002E CHECK 423 •E... 423 00000090 IXCH0602E The sysplex couple data set has insufficient The second line 423 indicates a line break of the first line 423.. I want to delete 423 of the first line. REXX processing is very easy but too slow because my operlog for one day is very large. Can sort or other tools handle it? Any suggestions is greatly appreciated Jason Cai ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
