Seymour, I have to interject here and state that I like JCL. There - it is out in the open and I feel much better for having said it :-) I am not going to say that it cannot be improved, of course it can, but I think it is a wonderful tool that has grown over the years, while HAVING to maintain most backward compatibility so it seems limited in how it can change, because of the need to maintain that compatibility.
Regards - Grant DTCC Internal (Green) -----Original Message----- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Seymour J Metz Sent: 09 July 2018 16:10 To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: REXX as JCL replacement ATTENTION! This email originated outside of DTCC; exercise caution. 1. I don't recall anybody on IBM-MAIN claims that JCL is good. 2. Creative solutions are good if they solve the actual problem, not just an imaginary or irrelevant one. 2. Sleep does not address the deadly embrace problem. 3. PDSE does not solve the problem. -- Shmuel (Seymour J.) Metz http://mason.gmu.edu/~smetz3 ________________________________________ From: IBM Mainframe Discussion List <IBM-MAIN@listserv.ua.edu> on behalf of Hobart Spitz <orexx...@gmail.com> Sent: Monday, July 9, 2018 8:46 AM To: IBM-MAIN@listserv.ua.edu Subject: Re: REXX as JCL replacement Ward wrote: >What are the problems (perceived or real) that will be resolved by replacing JCL with REXX? I think that it's important to first understand the characteristics on which many people base their thinking that JCL is good and may never go away. The point is important, and the basis is a valid concern. The conclusion is wrong. All it takes is understanding the problem and coming up with creative solutions. As I describe below, DB2 based applications and those that use only PDSEs are likely to be good starting points. One valuable feature of z/OS batch processing is that all ENQs for the entire JOB are obtained before any step execution actually starts. This is done to prevent a deadly embrace: JOB A has exclusive access to dataset X and needs exclusive access to dataset Y. JOB B has access to Y and needs exclusive on X. Neither JOB can proceed until one is canceled, resulting in lost production. Despite this being a useful service, there is no TSO equivalent, either via a parallel ENQs (as Seymour suggested) or a standard SLEEP mechanism to allow for retries. Other repliers have answered Ward's question well, I was going to cut some of the text out of my draft, but didn't. I think it is important to understand what z/OS batch processing does, and where possible, why, so we don't trivialize the effort or miss easy opportunities. I'll share my best educated guesses with respect to the subject at hand. When a batch JOB becomes eligible for execution these actions, relevant to our subject, happen in this order: 1. SETs are processed, and EXECed PROCs, overrides, and INCLUDEs are merged with the submitted JCL. These are necessary for #2 below. 2. Substitutions are processed. This is needed for #3; if DSNs could change later, via substitution or IF, the ENQs would be incorrect. 3. ENQs are obtained on all datasets. 4. The EXECs for PGMs are processed, in order, one at a time. So any REXX implementation would have either to forgo do initial ENQs, or commit to not changing DSNs during processing, or observe certain conventions. These conventions could include the following, one or the other may be appropriate depending on the applications' needs: - Perform ENQs in a consistent order. (Left as an exercise for the reader. :-) ) - Do all ALLOCations up front, and free all exclusive allocations before a second set of ENQs is requested. - Judiciously test for SYSDSN() returning the value UNAVAILABLE DATASET or ALLOC failure, and taking appropriate action to recover/restart/suspend processing. Initial ENQs are not required. If implemented via a common routine, either site or vendor provided, this could be more viable. A little additional coding to support automated restart might be needed; see also below on my first attempt at using REXX instead of JCL. - Something else. It might be helpful to know if, with all the changes and advancements, deadly-embraces are still the issue it was in the 1960s-1980s. As I explain below, an application that does not update any sequential datasets or PDSs, does not have this as a major concern. E.g. an application where all data updates are done to DB2 tables and PDSEs. Replace sequential datasets and PDSs with PDSEs, even if there is only one member. If you allow a dataset name to be set based on something only available during execution time, there can be no initial ENQs, since the dataset name is not known before the first step starts. Further, z/OS batch processing in general, and initial ENQs, specifically, lock in current customers due to the cost of porting and lack of equivalent features elsewhere. Some smart developer may come to understand this and provide the missing service. Some smart lawyer at a competitor might just decide to take legal action on an anti-competitive basis. Either scenario could represent an exposure to IBM (competitive or legal) and potential new costs to customers. At the same time that existing customers are locked in, new customers are locked out for the same reasons. I'm going to provide some background, because, in some cases there are no barriers to using REXX in batch. That said, if your application does not update sequential or partitioned datasets (in any JOB), then you can use REXX batch today, without any significant risk of deadly-embrace or other problems. This would most likely apply DB2 base applications, where all the DDs are for DB2 libraries reference with DISP=SHR. Give it a try. If you get down to just a fraction your previous conventional JOBs, your management might get the idea that you are onto something and change any fossilized mind-set about JCL. If all your DISP='s are SHR, you may already have the opportunity to streamline your application. This is not theory. I have done it, as I describe in the next paragraph. AFAIK, you can code SHR for PDSEs in the common cases. REWRITE/UPDATE in place is the main exception; you at least get an abend in testing, unlike the scenario if you update-in-place a PDS with DISP=SHR, and you just lose data. (Apologies to anyone who has seen/heard this before. I'll abbreviate.) In the early 1990s, I was working on a DB2 application that had hundreds of JOB streams, most differing by 1 or more of 3 parameter values, phase, client, and bill cycle. JOBs were submitted manually based on a visual check of a DB2 table that tracked the current state of processing for each batch process. Programs checked that same table to be sure that the processing requested had passed the previous phase. All updates were to DB2 tables. I determined that we could eliminate most of the JOBs and automate processing with REXX and a DB2/REXX interface. We ended up with less than 10 job streams, each of which would check for ready work, execute the COBOL program for that work, and then loop back and look for more work. Since the different phase processes could be executed in parallel for different billing cycles and/or clients, one phase might complete its work, making new work ready for the next phase via loopback check, while the previous phase was being processed. Driving work from this phase tracking table meant that we had automatic restart. If a process failed, any needed corrective action would be taken, and the JCL resubmitted, unchanged(!). Sometimes no resubmission was necessary if that phase was already working. No RESTARTs to code, no PARMs to update, no dataset names to override, and not third-party software. This summarizes my thoughts on the JCL problems that REXX would address, per Ward's request: 1. JCL creates an unnecessary, manditory, artificial, arbitrary, and counter-productive distinction between scripting code and application code. This the most important problem with JCL, and the hardest for single platform staff and management to understand and accept. Under REXX, the distinction entirely optional. 2. The communication between scripting (JCL) and application code is limited. One (or some) strings in, and 4 digits (0-4095) out. TSO supports return codes up to 16M, and REXX has no limit other than that imposed by the invocation interface. The PARM= value can't come from a file, DB2 table, or run-time expression. With REXX, these and other options are available. 3. The JCL syntax (which is based on that of assembler AFAIK, a highly inappropriate choice) is obscure, unintuitive, irregular, and has a steep learning curve. That should be no surprise to anyone who has been paying attention to the frequent questions about JCL on IBM-MAIN, and other boards, that almost always get one of two answers: You can't do that, or you have to do something weird and unintuitive. Using REXX, the syntax is consistent, intuitive, powerful, and easy to use. The same expression syntax is used everywhere. 4. JCL is a barrier to attracting new business to z/OS. Bad news if you are a vendor. 5. JCL is a barrier to customers leaving z/OS, even for other IBM platforms. Mixed news if you are a vendor. 6. PROC EXECs, SETs, INCLUDEs cannot be conditional. Initial ENQs prevent this. z/OS must know all dataset names to do the ENQs. 7. You can not loop or retry, as was done in the application I described above. 8. You can't compare strings, do arithmetic, or employ complex logic. 9. You can't call a function or return a string value. 10. String/SET/PROC value substitution can convoluted. Do I need a double period and/or a double ampersand? If you are passing a value down two PROC level, you may have to put in 4 (!) ampersands or quotes to indicate 1. More than two levels? Good luck!! REXX has no such requirement. 11. Many of these limitations are addressed by complex work-arounds, or expensive third-party packages that force you to work in a way dictated by the software and not by your business requirements. There may be other reasons. Basically, JCL is so far from real a programming language, that I can't describe it. z/VM, like almost all other platforms, gets along swimmingly without a separate batch language. You can issue CMS/zLinux/guest commands, or invoke REXX programs to drive any batch process. Replace your JCL with REXX a little at a time. Start with the low hanging fruit and/or those that have automation requirements that are not addressed in existing software. Focus on those that use DB2 table and PDSEs to store data updates, and those JOBs that only have DISP=SHR. Replace updated PDSs and sequential datasets with PDSEs, even if you only have one member. To be really careful, make the changes so that you only have DISP-SHR in your JCL. Once your are satisfied with that, replace the JCL with REXX. You may have to experiment a bit. Post your questions if you are stuck. JOB and any JESx control cards will mostly have to stay, and you will need a TSO batch step. I recommend a PROC like this. *//TSO PROC CMD=PROFILE* *// EXEC PGM=IKJEFT01,PARM='&CMD'* *//SYSPROC DD DISP=SHR,DSN=your site sysproc libs// DD ...//SYSEXEC DD DISP=SHR,DSN=your site sysexec libs// DD ...//SYSTSPRT DD SYSOUT=** *//SYSPRINT DD SYSOUT=* Not required, but commonly used.* *//* Additional commands can be added via SYSTSIN* *//SYSTSIN DD DDNAME=SYSIN In case you forget SYSTSIN DD *.* *// PEND* It can be invoked thusly: *//MYJOBNAM JOB (acct),'M.E.SMITH',MSGCLASS=t,...* *// JCLLIB ORDER=(MY.JCL.LIB) Optional* *// EXEC TSO* *//SYSTSIN DD * Recommended DD to avoid conflicts with SYSIN use.* *st* *%myrexpgmalloc reuse dd(sysin) dummy* *alloc reuse dd(sysut1) shr dsn(in.data)* *alloc reuse dd(sysut2) mod dsn(out.data(results)) cyl space(10 10) dsntype(library)* *call *(iebgener)* *//* In general, all ALLOCs should include REUSE, since you maybe combining multiple steps together or other programs might use the same DD. It doesn't hurt. Other handy things to add as you need them: - A REXX exec to allocate DB2 libraries if you need them. Accept an arg string that identifies the environment if you need to, or interrogate some external source of information. - A REXX exec to invoke batch ISPF. Allocate all your ISP*LIBs and ISPTABL. Take an arg string in the form of CMD(...), PGM(...) PARM(...), or PANEL(...), and append it to ISPSTART. Only non-display panels are valid in batch. Include a loop (DO 10 UNTIL RC <> 995; "ISPSTART" arg(1); END) in case ISPSTART fails with a 995(?) on an ISPTABL enqueue failure. You must VPUT ZISPFRC to get the return code passed up to the caller (IKJEFT01). - A REXX function to convert a relative GDG reference to an absolute one. This will let you ALLOC a GDG member. - An ALLOC wrapper function (in REXX) to check SYSDSN(), and determine if the disposition should be NEW CATALOG or SHR, so you don't have to use the finicky MOD. Then proceed with the "ALLOC REUSE DDNAME(...) ..." , and return the RC of ALLOC. I would use ARG DDNAME DISP DSNAME OPTIONS to receive blank delimited values. Separate concatenated DSNs with abutted commas. A DISP value of "ASIS" could request the condition NEW/SHR disposition. Loop over all the args the same way. If any allocation fails, free them all. This is your chance to join the 21st century. I hope this helps. OREXXMan JCL is the buggy whip of 21st century computing. Stabilize it. Put Pipelines in the z/OS base. Would you rather process data one character at a time (Unix/C style), or one record at a time? IBM has been looking for an HLL for program products; REXX is that language. On Wed, Jul 4, 2018 at 3:53 AM, Ward Able, Grant <gwarda...@dtcc.com> wrote: > I seldom post here, but have been intrigued by this thread. > > What are the problems (perceived or real) that will be resolved by > replacing JCL with REXX? > > > > Regards - Grant. > > In theory, there's no difference between theory and practice. In > practice, there is. > > There is no such thing as the Cloud. It is just somebody else's computer. > > If you don't have time to do it right, when will you have the time to > do it over? - John Wooden > > > DTCC Internal (Green) > > -----Original Message----- > From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] > On Behalf Of Paul Gilmartin > Sent: 03 July 2018 21:39 > To: IBM-MAIN@LISTSERV.UA.EDU > Subject: Re: REXX as JCL replacement > > ATTENTION! This email originated outside of DTCC; exercise caution. > > > On Tue, 3 Jul 2018 15:39:40 -0400, Tony Thigpen wrote: > > >PARSE SOURCE on VSE returns: > >VSE COMMAND TESTREXX PROC IPLLIB.VSE2PDF1.TESTREXX.PROC TESTREXX VSE > >VSE ? > > > So it would be practical to multipath for compatibility. > > >VSE does not have pipelines. > > > z/OS barely has Pipelines. An obsolete version. And hard to order. > > For more than you want to know about Rexx (in)compatibility, see Dave > Alcocks > http://secure-web.cisco.com/1vQAahM_WEN0g-O0uf0FC49Sm1iZY1q5tU650TAzJA > PfZA-6GgzWqmQIrZAGlNK4DGVa_0OXF96LD1tkYz5MlkqTvdRNoNdhc5yjZrD9qmi5h2ES > 78_bh08QhJNj9Tl8l6G5N2knmK6tZyBQ92hb-9vbFH0ehhFhpkXR9Tb00r2kuqUBUccdsM > O2mnO2J3Psii1nswtaVb15FR7bbny20JeLJVyl3oO_T8ex9BQsdGjZHjJU4FVZTWv_IAaT > X5ghLP_FLAX1tyrEbLJ0BmKLSi7_bJ5UESdd6omyotG4JSVGJpY7p--wbWa5JxyzSYgWpD > cc9BjeSUVmx2POrXjBn-w6KLORmh5651iFW8d6mfHyJZ9pvQ4gbB25w1DoQ06brrgi3Vu2 > fG8CU-7Pd-6uVQIZ7q-7f3Q0y1FjOJDEd6abDRR_2QU3p-_TbFGVw4a96/http%3A%2F%2 > Fplanetmvs.com%2Frexxanywhere%2Findex.html > > OK. Thanks. Comparing z/VSE and Z/OS ADDRESS environments, I see: > > z/VSE z/OS > > Ref. SC33-6642-10 SA32-0972-30 > > LINK Like JCL EXEC or Assembler Call I think this is a relic > of CMS calling > with only one argument. conventions. I've never > used it. > > LINKPGM Like Assembler CALL with multiple Like Assembler CALL; > length halfwords not > arguments; length halfwords generated generated automatically. > automatically. > > LINKMVS n/a Like Assembler CALL with > multiple > arguments; length > halfwords generated > automatically. > (This seems to be > z/VSE's LINKPGM.) > > -- gil > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, send > email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN DTCC > DISCLAIMER: This email and any files transmitted with it are > confidential and intended solely for the use of the individual or > entity to whom they are addressed. If you have received this email in > error, please notify us immediately and delete the email and any > attachments from your system. The recipient should check this email > and any attachments for the presence of viruses. The company accepts > no liability for any damage caused by any virus transmitted by this email. > > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, send > email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
