On Fri, 2007-06-15 at 09:01 -0400, Bob Shannon wrote: > Some years ago Barry Merrill wrote a paper on the history of SMF. He > described an SMF-processing system IBM developed but never released > (SMFMAN?).
He presented at SHARE 73 in August of 1989. Here are notes of that session from my trip report of the time: === This was an outstanding presentation by Barry Merrill on the history of SMF. He assembled the information from recently declassified IBM documents, and through interviews with the original architects of the facility. OS/360 was announced on April 7, 1964 with no SMF. Barely three months later the SHARE Systems Objectives and Requirements Committee (SORC) presented IBM with a report that said: IBM S/360 must provide as a standard feature job accounting information. The SORC report included most of the data items that you have seen in SMF data over the past 20 years. The SHARE SORC also asked for a couple of data items that we still don't have after 20 years: * operator intervention time (time awaiting tape/paper mounts, etc.) * core used by program, core used by data The closest thing to SMF provided by IBM at the time was the IEFACTRT exit. IEFACTRT was provided pointers to data items such as the jobname and stepname. OS/360 also provided a system dataset - SYS1.ACCT - to which IEFACTRT could write user records. The SORC report basically lay dormant until June 1966 when Bob Brockish joined IBM (he had previously been active in SHARE). He looked seriously at the SORC report and saw two architectural goals of a future SMF project. Users needed to account for time and resource usage on their expensive mainframes. IBM needed to know how OS/360 was being used in the field! In fact, there was an active project at the time within IBM - called the SYSOUT Project - which was devising programs to look at SYSOUT tapes from customer sites and try to analyze system usage from the output that customers produced. On July 6, 1966, the name SMF - System Management Facility - was picked in a brainstorming session. The philosophy of the project was that IBM would provide a way to collect data, but the customer must actually report on the collected data (IBM couldn't see how to second-guess customers on their use of accounting data). After surveying data requirements, it was determined that SMF would have to modify 160 modules of the operating system. This caused a great deal of worry over system performance, and so IBM planned for installation-selectable "packages" of information. GUIDE finally got involved in the SMF project on May 22, 1967. On August 25, SHARE went to its membership with a survey which asked "How much degradation is OK for collection of SMF data?" The SMF exits and intercepts were originally called the "Dynamic Control Capability". It consisted of a system management dataset, five data collection packages, and "dynamic control entries" (which presumably controlled which packages were to be activated). The data collection packages each were to record a certain level of detail: Package 1: job level data Package 2: step level data Package 3: additional job/step data Package 4: still more data Package 5: sample data Packages 1-4 were for accounting purposes. Package 5 was for performance measurement (and was never implemented until MF/1 came around in MVS). SMF was to provide utility reports based on data recorded in SYS1.MAN - system, job and step profiles. There was to be no performance degradation when no SMF is employed. Packages 1-3 should not degrade the system more than 3%. Package 4 was allowed to degrade the system 4%. Package 5 time/sample was to be no more than 500 milliseconds on a 360/50 CPU. Subset 2 of the SMF definition contained a "dataset management utility". It provided an archive for the SYS1.MAN database. It also reported on dataset OPENs and CLOSEs, and provided additional billing information. On July 27, 1967, volume and PDS reorganization utilities were defined as part of SMF (the system was to write a message to the console when a PDS required reorganization)! IBM at the time had 550 users of OS/360 and they forecasted an eventual total of 1,590 users. With 75,000 CPUs delivered worldwide, this estimate may have been low... One of the SMF architects - Bart Page - was designing a SRM component for SMF, but he was a better technician than he was a salesman. He subsequently left SMF to go work on the MVM - Multiple Virtual Memories - project. In January 1968 the final specification for the management utility was internally published, and included DASD and tape inventory functions. The specification included utility syntax, JCL to execute and output messages (numbered IEH901 through IEH916). The program name: IEHMAN! It was tested on a 360/50, and took 19 minutes to update the archive from a SYS1.MAN with 60 jobs on it. On June 25, 1969 the Resource Management Utility was deleted with no explanation. This was indeed a last-minute decision - there was already an IEHMAN Planning Guide awaiting publication. Apparently signals were crossed in Mechanicsburg, and some of the Planning Guides were actually shipped (and subsequently recalled). Type 16 records were eliminated and incorporated into types 14 and 15. SMF was released. IBM had estimated that SMF would get five APARs over the first six months. Actually, 40 PTFs addressed 80 APARs. SAS began its spectacular climb to success as an SMF analyzer (it originally cost $100). In the mid '70s, SMF began to develop some real performance problems. Each write to the MANX/Y dataset required a TCLOSE, an ENQ and a write verify. The "final" implementation of SMF (MVS/SE2) took 12 people almost two years, four times the original estimate. It finally implemented what SHARE people asked for 20 years earlier: * Performance measurement * Started task measurement * Record selectivity Its write performance was also enhanced when it went to VSAM: a BSAM write takes about 60,000 instructions. A VSAM ICIP write in SRB mode takes about 1,500. Throughout the talk, Barry emphasized the working relationship between IBM and SHARE during the SMF definition. He also repeatedly pointed out the robust qualities of the architecture, that data requirements thought of 20 years ago are still valid today. -- David Andrews A. Duda and Sons, Inc. [EMAIL PROTECTED] ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html