SLEEP is an example of the right answer to the wrong question. The problem is that you can't safely allocate a bunch of datasets that may potentially be used by other jobs.
As for PDSE, it in some ways makes the situation worse; concurrent jobs can update the same members, but in a different sequence, leading to bad updates. -- Shmuel (Seymour J.) Metz http://mason.gmu.edu/~smetz3 ________________________________________ From: IBM Mainframe Discussion List <IBM-MAIN@listserv.ua.edu> on behalf of Hobart Spitz <orexx...@gmail.com> Sent: Monday, July 9, 2018 12:11 PM To: IBM-MAIN@listserv.ua.edu Subject: Re: REXX as JCL replacement On Mon, Jul 9, 2018 at 11:09 AM, Seymour J Metz <sme...@gmu.edu> wrote: > 1. I don't recall anybody on IBM-MAIN claims that JCL is good. > There were such statements as "I like JCL", or similar. > > 2. Creative solutions are good if they solve the actual problem, not > just an imaginary or irrelevant one. > I don't think anything I said was either. > > 2. Sleep does not address the deadly embrace problem. > I didn't claim that. It is needed to do your up front ENQs in REXX. You still might have to exit with an error COND CODE if you can't get the ENQ you need. A single ALLOC does not give the TSU a message or time to free up the dataset. > > 3. PDSE does not solve the problem. > You'll have to explain that. From my reading of the doc., you can have multiple readers and writers, in the same step or different JOBs opening different members at the same time, provided the same member is not open for OUTPUT by two different DCBs or JOBs/TSUs. All this with SHaRed access, i.e. no DISP=OLD or equivalent. Note that I said, perhaps not clearly enough, that the entire application, all foreground and background accesses, must be SHaRed. What am I missing? Thanks. > > > -- > Shmuel (Seymour J.) Metz > http://mason.gmu.edu/~smetz3 > > ________________________________________ > From: IBM Mainframe Discussion List <IBM-MAIN@listserv.ua.edu> on behalf > of Hobart Spitz <orexx...@gmail.com> > Sent: Monday, July 9, 2018 8:46 AM > To: IBM-MAIN@listserv.ua.edu > Subject: Re: REXX as JCL replacement > > Ward wrote: > >What are the problems (perceived or real) that will be resolved by > replacing JCL with REXX? > > I think that it's important to first understand the characteristics on > which many people base their thinking that JCL is good and may never go > away. The point is important, and the basis is a valid concern. The > conclusion is wrong. All it takes is understanding the problem and coming > up with creative solutions. As I describe below, DB2 based applications > and those that use only PDSEs are likely to be good starting points. > > One valuable feature of z/OS batch processing is that all ENQs for the > entire JOB are obtained before any step execution actually starts. This is > done to prevent a deadly embrace: JOB A has exclusive access to dataset X > and needs exclusive access to dataset Y. JOB B has access to Y and needs > exclusive on X. Neither JOB can proceed until one is canceled, resulting > in lost production. Despite this being a useful service, there is no TSO > equivalent, either via a parallel ENQs (as Seymour suggested) or a standard > SLEEP mechanism to allow for retries. > > Other repliers have answered Ward's question well, I was going to cut some > of the text out of my draft, but didn't. I think it is important to > understand what z/OS batch processing does, and where possible, why, so we > don't trivialize the effort or miss easy opportunities. I'll share my best > educated guesses with respect to the subject at hand. > > When a batch JOB becomes eligible for execution these actions, relevant to > our subject, happen in this order: > > 1. SETs are processed, and EXECed PROCs, overrides, and INCLUDEs are > merged with the submitted JCL. These are necessary for #2 below. > 2. Substitutions are processed. This is needed for #3; if DSNs could > change later, via substitution or IF, the ENQs would be incorrect. > 3. ENQs are obtained on all datasets. > 4. The EXECs for PGMs are processed, in order, one at a time. > > So any REXX implementation would have either to forgo do initial ENQs, or > commit to not changing DSNs during processing, or observe certain > conventions. These conventions could include the following, one or the > other may be appropriate depending on the applications' needs: > > - Perform ENQs in a consistent order. (Left as an exercise for the > reader. :-) ) > - Do all ALLOCations up front, and free all exclusive allocations before > a second set of ENQs is requested. > - Judiciously test for SYSDSN() returning the value UNAVAILABLE DATASET > or ALLOC failure, and taking appropriate action to > recover/restart/suspend > processing. Initial ENQs are not required. If implemented via a common > routine, either site or vendor provided, this could be more viable. A > little additional coding to support automated restart might be needed; > see > also below on my first attempt at using REXX instead of JCL. > - Something else. > > It might be helpful to know if, with all the changes and advancements, > deadly-embraces are still the issue it was in the 1960s-1980s. As I > explain below, an application that does not update any sequential datasets > or PDSs, does not have this as a major concern. E.g. an application where > all data updates are done to DB2 tables and PDSEs. Replace sequential > datasets and PDSs with PDSEs, even if there is only one member. > > If you allow a dataset name to be set based on something only available > during execution time, there can be no initial ENQs, since the dataset name > is not known before the first step starts. > > Further, z/OS batch processing in general, and initial ENQs, specifically, > lock in current customers due to the cost of porting and lack of equivalent > features elsewhere. Some smart developer may come to understand this and > provide the missing service. Some smart lawyer at a competitor might just > decide to take legal action on an anti-competitive basis. Either scenario > could represent an exposure to IBM (competitive or legal) and potential new > costs to customers. At the same time that existing customers are locked > in, new customers are locked out for the same reasons. > > I'm going to provide some background, because, in some cases there are no > barriers to using REXX in batch. > > That said, if your application does not update sequential or partitioned > datasets (in any JOB), then you can use REXX batch today, without any > significant risk of deadly-embrace or other problems. This would most > likely apply DB2 base applications, where all the DDs are for DB2 libraries > reference with DISP=SHR. > > Give it a try. If you get down to just a fraction your previous > conventional JOBs, your management might get the idea that you are > onto something and change any fossilized mind-set about JCL. > > If all your DISP='s are SHR, you may already have the opportunity to > streamline your application. This is not theory. I have done it, as I > describe in the next paragraph. AFAIK, you can code SHR for PDSEs in the > common cases. REWRITE/UPDATE in place is the main exception; you at least > get an abend in testing, unlike the scenario if you update-in-place a PDS > with DISP=SHR, and you just lose data. > > (Apologies to anyone who has seen/heard this before. I'll abbreviate.) In > the early 1990s, I was working on a DB2 application that had hundreds of > JOB streams, most differing by 1 or more of 3 parameter values, phase, > client, and bill cycle. JOBs were submitted manually based on a visual > check of a DB2 table that tracked the current state of processing for each > batch process. Programs checked that same table to be sure that the > processing requested had passed the previous phase. All updates were to > DB2 tables. I determined that we could eliminate most of the JOBs and > automate processing with REXX and a DB2/REXX interface. We ended up with > less than 10 job streams, each of which would check for ready work, execute > the COBOL program for that work, and then loop back and look for more > work. Since the different phase processes could be executed in parallel > for different billing cycles and/or clients, one phase might complete > its work, making new work ready for the next phase via loopback check, > while the previous phase was being processed. Driving work from this phase > tracking table meant that we had automatic restart. If a process failed, > any needed corrective action would be taken, and the JCL resubmitted, > unchanged(!). Sometimes no resubmission was necessary if that phase was > already working. No RESTARTs to code, no PARMs to update, no dataset names > to override, and not third-party software. > > This summarizes my thoughts on the JCL problems that REXX would address, > per Ward's request: > > 1. JCL creates an unnecessary, manditory, artificial, arbitrary, and > counter-productive distinction between scripting code and application > code. This the most important problem with JCL, and the hardest for > single > platform staff and management to understand and accept. Under REXX, the > distinction entirely optional. > 2. The communication between scripting (JCL) and application code is > limited. One (or some) strings in, and 4 digits (0-4095) out. TSO > supports return codes up to 16M, and REXX has no limit other than that > imposed by the invocation interface. The PARM= value can't come from a > file, DB2 table, or run-time expression. With REXX, these and other > options are available. > 3. The JCL syntax (which is based on that of assembler AFAIK, a highly > inappropriate choice) is obscure, unintuitive, irregular, and has a > steep > learning curve. That should be no surprise to anyone who has been > paying > attention to the frequent questions about JCL on IBM-MAIN, and other > boards, that almost always get one of two answers: You can't do that, > or > you have to do something weird and unintuitive. Using REXX, the syntax > is > consistent, intuitive, powerful, and easy to use. The same expression > syntax is used everywhere. > 4. JCL is a barrier to attracting new business to z/OS. Bad news if you > are a vendor. > 5. JCL is a barrier to customers leaving z/OS, even for other IBM > platforms. Mixed news if you are a vendor. > 6. PROC EXECs, SETs, INCLUDEs cannot be conditional. Initial ENQs > prevent this. z/OS must know all dataset names to do the ENQs. > 7. You can not loop or retry, as was done in the application I described > above. > 8. You can't compare strings, do arithmetic, or employ complex logic. > 9. You can't call a function or return a string value. > 10. String/SET/PROC value substitution can convoluted. Do I need a > double period and/or a double ampersand? If you are passing a value > down > two PROC level, you may have to put in 4 (!) ampersands or quotes to > indicate 1. More than two levels? Good luck!! REXX has no such > requirement. > 11. Many of these limitations are addressed by complex work-arounds, or > expensive third-party packages that force you to work in a way dictated > by > the software and not by your business requirements. > > There may be other reasons. > > Basically, JCL is so far from real a programming language, that I can't > describe it. > > z/VM, like almost all other platforms, gets along swimmingly without a > separate batch language. You can issue CMS/zLinux/guest commands, or > invoke REXX programs to drive any batch process. > > Replace your JCL with REXX a little at a time. Start with the low hanging > fruit and/or those that have automation requirements that are not addressed > in existing software. Focus on those that use DB2 table and PDSEs to store > data updates, and those JOBs that only have DISP=SHR. Replace updated PDSs > and sequential datasets with PDSEs, even if you only have one member. To > be really careful, make the changes so that you only have DISP-SHR in your > JCL. Once your are satisfied with that, replace the JCL with REXX. You > may have to experiment a bit. Post your questions if you are stuck. > > JOB and any JESx control cards will mostly have to stay, and you will need > a TSO batch step. I recommend a PROC like this. > > *//TSO PROC CMD=PROFILE* > *// EXEC PGM=IKJEFT01,PARM='&CMD'* > *//SYSPROC DD DISP=SHR,DSN=your site sysproc libs// DD > ...//SYSEXEC DD DISP=SHR,DSN=your site sysexec libs// DD > ...//SYSTSPRT DD SYSOUT=** > *//SYSPRINT DD SYSOUT=* Not required, but commonly used.* > *//* Additional commands can be added via SYSTSIN* > *//SYSTSIN DD DDNAME=SYSIN In case you forget SYSTSIN DD *.* > *// PEND* > > It can be invoked thusly: > > *//MYJOBNAM JOB (acct),'M.E.SMITH',MSGCLASS=t,...* > *// JCLLIB ORDER=(MY.JCL.LIB) Optional* > *// EXEC TSO* > *//SYSTSIN DD * Recommended DD to avoid conflicts with SYSIN use.* > *st* > > *%myrexpgmalloc reuse dd(sysin) dummy* > *alloc reuse dd(sysut1) shr dsn(in.data)* > *alloc reuse dd(sysut2) mod dsn(out.data(results)) cyl space(10 10) > dsntype(library)* > *call *(iebgener)* > *//* > > In general, all ALLOCs should include REUSE, since you maybe combining > multiple steps together or other programs might use the same DD. It > doesn't hurt. > > Other handy things to add as you need them: > > - A REXX exec to allocate DB2 libraries if you need them. Accept an arg > string that identifies the environment if you need to, or interrogate > some > external source of information. > - A REXX exec to invoke batch ISPF. Allocate all your ISP*LIBs and > ISPTABL. Take an arg string in the form of CMD(...), PGM(...) > PARM(...), > or PANEL(...), and append it to ISPSTART. Only non-display panels are > valid in batch. Include a loop (DO 10 UNTIL RC <> 995; "ISPSTART" > arg(1); END) in case ISPSTART fails with a 995(?) on an ISPTABL enqueue > failure. You must VPUT ZISPFRC to get the return code passed up to the > caller (IKJEFT01). > - A REXX function to convert a relative GDG reference to an absolute > one. This will let you ALLOC a GDG member. > - An ALLOC wrapper function (in REXX) to check SYSDSN(), and determine > if the disposition should be NEW CATALOG or SHR, so you don't have to > use > the finicky MOD. Then proceed with the "ALLOC REUSE DDNAME(...) ..." , > and > return the RC of ALLOC. I would use ARG DDNAME DISP DSNAME OPTIONS to > receive blank delimited values. Separate concatenated DSNs with abutted > commas. A DISP value of "ASIS" could request the condition NEW/SHR > disposition. Loop over all the args the same way. If any allocation > fails, free them all. > > This is your chance to join the 21st century. > > I hope this helps. > > OREXXMan > JCL is the buggy whip of 21st century computing. Stabilize it. > Put Pipelines in the z/OS base. Would you rather process data one > character at a time (Unix/C style), or one record at a time? > IBM has been looking for an HLL for program products; REXX is that > language. > > On Wed, Jul 4, 2018 at 3:53 AM, Ward Able, Grant <gwarda...@dtcc.com> > wrote: > > > I seldom post here, but have been intrigued by this thread. > > > > What are the problems (perceived or real) that will be resolved by > > replacing JCL with REXX? > > > > > > > > Regards – Grant. > > > > In theory, there's no difference between theory and practice. In > practice, > > there is. > > > > There is no such thing as the Cloud. It is just somebody else’s computer. > > > > If you don't have time to do it right, when will you have the time to do > > it over? - John Wooden > > > > > > DTCC Internal (Green) > > > > -----Original Message----- > > From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On > > Behalf Of Paul Gilmartin > > Sent: 03 July 2018 21:39 > > To: IBM-MAIN@LISTSERV.UA.EDU > > Subject: Re: REXX as JCL replacement > > > > ATTENTION! This email originated outside of DTCC; exercise caution. > > > > > > On Tue, 3 Jul 2018 15:39:40 -0400, Tony Thigpen wrote: > > > > >PARSE SOURCE on VSE returns: > > >VSE COMMAND TESTREXX PROC IPLLIB.VSE2PDF1.TESTREXX.PROC TESTREXX VSE > > >VSE ? > > > > > So it would be practical to multipath for compatibility. > > > > >VSE does not have pipelines. > > > > > z/OS barely has Pipelines. An obsolete version. And hard to order. > > > > For more than you want to know about Rexx (in)compatibility, see Dave > > Alcocks http://secure-web.cisco.com/1vQAahM_WEN0g- > O0uf0FC49Sm1iZY1q5tU650TAzJAPfZA-6GgzWqmQIrZAGlNK4DGVa_ > 0OXF96LD1tkYz5MlkqTvdRNoNdhc5yjZrD9qmi5h2ES78_ > bh08QhJNj9Tl8l6G5N2knmK6tZyBQ92hb-9vbFH0ehhFhpkXR9Tb00r2kuqUBUcc > dsMO2mnO2J3Psii1nswtaVb15FR7bbny20JeLJVyl3oO_T8ex9BQsdGjZHjJU4FVZTWv_ > IAaTX5ghLP_FLAX1tyrEbLJ0BmKLSi7_bJ5UESdd6omyotG4JSVGJpY7p-- > wbWa5JxyzSYgWpDcc9BjeSUVmx2POrXjBn-w6KLORmh5651iFW8d6mfHyJZ9pvQ4g > bB25w1DoQ06brrgi3Vu2fG8CU-7Pd-6uVQIZ7q-7f3Q0y1FjOJDEd6abDRR_ > 2QU3p-_TbFGVw4a96/http%3A%2F%2Fplanetmvs.com%2Frexxanywhere%2Findex.html > > > > OK. Thanks. Comparing z/VSE and Z/OS ADDRESS environments, I see: > > > > z/VSE z/OS > > > > Ref. SC33-6642-10 SA32-0972-30 > > > > LINK Like JCL EXEC or Assembler Call I think this is a relic > > of CMS calling > > with only one argument. conventions. I've > never > > used it. > > > > LINKPGM Like Assembler CALL with multiple Like Assembler CALL; > > length halfwords not > > arguments; length halfwords generated generated > automatically. > > automatically. > > > > LINKMVS n/a Like Assembler CALL > with > > multiple > > arguments; length > > halfwords generated > > automatically. (This > > seems to be > > z/VSE's LINKPGM.) > > > > -- gil > > > > ---------------------------------------------------------------------- > > For IBM-MAIN subscribe / signoff / archive access instructions, send > email > > to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > DTCC DISCLAIMER: This email and any files transmitted with it are > > confidential and intended solely for the use of the individual or entity > to > > whom they are addressed. If you have received this email in error, please > > notify us immediately and delete the email and any attachments from your > > system. The recipient should check this email and any attachments for the > > presence of viruses. The company accepts no liability for any damage > > caused by any virus transmitted by this email. > > > > > > ---------------------------------------------------------------------- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
