Re: GDG question
Robert, No other job can reference a relative generation while the create job is running. The create job holds an exclusive ENQ on the GDG base as well as GDG(0) preventing any other job from running. This does not prevent another address space from referencing the generations by absolute name. Dan -Original Message- From: Robert A. Rosenberg Sent: Sunday, October 27, 2013 4:49 PM Subject: Re: GDG question At 10:57 -0700 on 10/26/2013, Jon Perryman wrote about Re: GDG question: Job's that want to reference the current GDG will use (0). Creating the new version (+1) does not delay use of (0). If you require others wait for the new version (+1), then you will need DISP=OLD for the (0) GDG while creating the new version. GDG's (0), (+1), (+2), ... (-1), (-2),.. are each unique datasets. The only commonality is they are referenced thru the GDG name. Treat them as different datasets. One warning about relative generation numbers. Lets say the current generation number is 1234. A job that reads GDG(0) and creates GDG(+1) will create generation 1235. While the job is still running, GDG(0) will stay as 1234 even to jobs that start while the create job is running. As soon as it ends GDG(0) becomes 1235 while GDG(-1) will now be 1234. Any job that was running during the create job will keep its GDG(0) reference as 1234 even in steps that start after the creating job ends and GDG(0) becomes 1235 - IOW: Who is GDG(0) is fixed when the job starts and is not affected by the creation of additional generations. This can cause problems if you code GDG(0) as DISP=OLD in a create job since all jobs that launch after it starts and reference GDG(0) will wait for that job to end BUT since that job creates a GDG(+1), the waiting jobs have already decided what generation is GDG(0) and thus will actually be working with GDG(-1) not the newly created generation when they run. I do not know of a way to insure you are getting GDG(+!) when you reference GDG(0) by waiting for it to be finished being created - You always get GDG(0) at the time you get launched. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
Robert, Actually (tested on z/OS 1.11), it didn't matter whether I had DISP=OLD or DISP=SHR on my GDG(0), any other jobs submitted while the allocate job was running waited until the allocate of (+1) was complete and cataloged, then each grabbed the new generation as GDG(0). Rex -Original Message- From: Robert A. Rosenberg Sent: Sunday, October 27, 2013 4:49 PM Subject: Re: GDG question At 10:57 -0700 on 10/26/2013, Jon Perryman wrote about Re: GDG question: Job's that want to reference the current GDG will use (0). Creating the new version (+1) does not delay use of (0). If you require others wait for the new version (+1), then you will need DISP=OLD for the (0) GDG while creating the new version. GDG's (0), (+1), (+2), ... (-1), (-2),.. are each unique datasets. The only commonality is they are referenced thru the GDG name. Treat them as different datasets. One warning about relative generation numbers. Lets say the current generation number is 1234. A job that reads GDG(0) and creates GDG(+1) will create generation 1235. While the job is still running, GDG(0) will stay as 1234 even to jobs that start while the create job is running. As soon as it ends GDG(0) becomes 1235 while GDG(-1) will now be 1234. Any job that was running during the create job will keep its GDG(0) reference as 1234 even in steps that start after the creating job ends and GDG(0) becomes 1235 - IOW: Who is GDG(0) is fixed when the job starts and is not affected by the creation of additional generations. This can cause problems if you code GDG(0) as DISP=OLD in a create job since all jobs that launch after it starts and reference GDG(0) will wait for that job to end BUT since that job creates a GDG(+1), the waiting jobs have already decided what generation is GDG(0) and thus will actually be working with GDG(-1) not the newly created generation when they run. I do not know of a way to insure you are getting GDG(+!) when you reference GDG(0) by waiting for it to be finished being created - You always get GDG(0) at the time you get launched. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN The information contained in this message is confidential, protected from disclosure and may be legally privileged. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any disclosure, distribution, copying, or any action taken or action omitted in reliance on it, is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to this message and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
When UNIX is your forte, then recommending a UNIX solution may seem to be the way to go. The OP for this problem knew that GDG's solved his problem but he just didn't understand how and why. Following his lead is the way to go. MVS GDG's provide a natural solution that is preemptive and atomic.It's preemptive in that user's access the (0) generation which is always the current cataloged generation / always available and atomic in that the user's will only see the new version if (or when) it becomes cataloged. If the dataset is deleted before being cataloged, no one ever see's it. The advantage of GDG's over UNIX RENAME is that we don't risk someone inadvertently specifying an incorrect file name that could potentially delete an important file. GDG's are predominantly used in JCL and they work very well, Use of DISP=(NEW,CATLG,DELETE) deletes when there is an abend otherwise it catalogs the new version instantaneously without interrupting current users. There's no need to rename the dataset. IBM could easily implement IDCAM's ALTER ALIAS to serve this purpose but GDG's have served us so well for so long that we don't really need to be a UNIX variant. Jon Perryman.. - Original Message - From: Paul Gilmartin paulgboul...@aim.com To: IBM-MAIN@LISTSERV.UA.EDU Cc: Sent: Friday, October 25, 2013 7:05 AM Subject: Re: GDG question On Thu, 24 Oct 2013 21:25:50 -0700, Phil Smith wrote: But the data is also shared across tasks, so we don't want a window where it's half-written and some task tries to read it. UNIX files provide the natural solution to such a requirement in that the UNIX rename() is preemptive and atomic: o Preemptive in that rename() replaces any previously existing file (directory entry, actually) having the new name. o Atomic in that the operation is serialized by the kernel so that no other process doing a search for the file can encounter a window during which the file is not found. There ought to be a Requirement to provide a similar facility in IDCAMS. PDSE does very similarly for PDSE members. Can you use PDSE members? I regularly make use of this using FTP, actually, to install updated versions of certain files: -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
On Sat, 26 Oct 2013 10:57:09 -0700, Jon Perryman wrote: TSO ALLOC does not support the use of (0) and (+1) in the dataset name. Instead use BPXWDYN which does support it.� Does GDG play well with FTP? NFS? These are two facilities I use extensively to share/move data among mainframe and non-mainframe systems. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
TSO OCOPY has INDD and OUTDD so it will work fine and will use the TSO FREE command to catalog/delete.. FTP PUT/GET supports the specification of a GDG number but does not have the ability to control CATALOG/DELETE. However, RENAME 'xxx' 'MY.GDG(+1)' will do exactly what you want. As for other commands, you can run your own tests to find out how you can use them. Jon Perryman. - Original Message - From: Paul Gilmartin paulgboul...@aim.com To: IBM-MAIN@LISTSERV.UA.EDU Cc: Sent: Saturday, October 26, 2013 2:13 PM Subject: Re: GDG question On Sat, 26 Oct 2013 10:57:09 -0700, Jon Perryman wrote: TSO ALLOC does not support the use of (0) and (+1) in the dataset name. Instead use BPXWDYN which does support it.� Does GDG play well with FTP? NFS? These are two facilities I use extensively to share/move data among mainframe and non-mainframe systems. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
This Friday post of an old MXG change documents a unique problem with GDG wrapping, and the unwillingness of the IBM catalog technician (a/k/a trench holder) to provide the documentation of their catalog records back in 2005. Maybe he was afraid that MicroSoft would offer a competing catalog product??? Barry Change 23.219 The ICF Catalog 05 record variable GATGEN should have VMAC6156 been input as PIB.2., instead of one byte, and variable VMACCTLG GATWRAP='Y' is now set if the first bit of GATGEN is on, Aug 29, 2005 to mark that this GDG member has wrapped; i.e., its generation number has exceeded . If GATWRAP is on, GATGEN=GATGEN-32768 to contain the correct Gen number. This discovery was precipitated by IGD07001I ABENDs in production, which occurred when a GDG that had GATWRAP on for some members, and GATWRAP off for others, when that GDG generation number goes from 999 to 1000. Normally, when all members of a GDG have wrapped, the next catalog action turns off the wrap bit in all of the catalog records. For a normal GDG, that should happen when the +256th gen is created after wrap, as only 255 members can exist in the catalog. But this site had had a very old application that originally created +1 thru +5 each day, and then deleted +1 thru +4, leaving only +5 in the catalog. However, back when Clinton was President, an application change was made that created only +1 thru +3 and deleted only +1 and +2, so there were 2 old GVoo's left in the catalog. When the GDG wrapped, and when the create of +1 would have created G1000V00, those old GVoo's still had their wrap bit off, and the production job failed: IGD07001I; GDG ROLL IN ERROR - RETURN CODE 140 REASON 122 Return Code 140: Inconsistent or conflicting arguments were provided. Reason Code 122: Catalog G1000Vxx will cause the GDG to exceed the limit of 10,999. Programmer Response: Clean up the GDG in error then catalog G1000Vxx. The site found Information APAR II07276, which explained how wrap works, but it says to 'see the Z page for internal details of the wrap bit interface'. So the site asked IBM Support: We need to identify other GDGs that have the same exposure to causing ABENDs. Can I look at the 'Z' page? Does the 'wrap bit' appear in SMF 61, 65, 66 ICF Catalog records? IBM Level 2 Catalog Support refused to answer the quite valid question, with this answer: Unfortunately, the 'Z' page in our info APARs refer to information meant for IBM eyes only, for helping us get to problem determination quicker. We are not at liberty to discuss any control block internals that are not provided in our published manuals. As for information in SMF records relating to this, this info would be included in the updated record portion of the SMF record, however we cannot discuss offsets for those either. I am sorry I cannot be more helpful. Please let me know if there are any other questions that I can answer for you. Even escalation to my IBM Poughkeepsie z/OS support did not convince IBM Level 2 Catalog Support to identify which bit is the GATWRAP bit. The ICF Support and Developers hid behind OCO, Object Code Only, as the reason they could not provide catalog record documentation (but DSECTS are NOT CODE!). So I suggested the site could create a GDG and force it to wrap, and by examining SMF records, we concluded that first bit of GATGEN was the GATWRAP bit. Then, I remembered I still had lots of archaic printed manuals, and located the very old MVS/XA Catalog Diagnosis Reference for DFP Release 1.2, which confirmed that the GATWRAP bit was indeed the first bit of the GATGEN field in the 05 Catalog Record! -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Rob Schramm Sent: Friday, October 25, 2013 12:36 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: GDG question Phil, AFAIK, the whole GDG assignment only happens once.. during initial creation. MODing will change
Re: GDG question
On Thu, 24 Oct 2013 21:25:50 -0700, Phil Smith wrote: But the data is also shared across tasks, so we don't want a window where it's half-written and some task tries to read it. UNIX files provide the natural solution to such a requirement in that the UNIX rename() is preemptive and atomic: o Preemptive in that rename() replaces any previously existing file (directory entry, actually) having the new name. o Atomic in that the operation is serialized by the kernel so that no other process doing a search for the file can encounter a window during which the file is not found. There ought to be a Requirement to provide a similar facility in IDCAMS. PDSE does very similarly for PDSE members. Can you use PDSE members? I regularly make use of this using FTP, actually, to install updated versions of certain files: PUT updated-file tempname RENAME tempname static-name A couple times an observant user has spotted the tempname file while the operation was progress and asked me to explain its existence. But he never had to deal with an incomplete file. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
Gil, I wonder if said observant user would notice a file name which started with a period, followed by the characters newversion-of', another period, then the original file name? E.g. PUT updated-file .newversion-of.static-name RENAME .newversion-of.static-name static-name On Fri, Oct 25, 2013 at 9:05 AM, Paul Gilmartin paulgboul...@aim.comwrote: On Thu, 24 Oct 2013 21:25:50 -0700, Phil Smith wrote: But the data is also shared across tasks, so we don't want a window where it's half-written and some task tries to read it. UNIX files provide the natural solution to such a requirement in that the UNIX rename() is preemptive and atomic: o Preemptive in that rename() replaces any previously existing file (directory entry, actually) having the new name. o Atomic in that the operation is serialized by the kernel so that no other process doing a search for the file can encounter a window during which the file is not found. There ought to be a Requirement to provide a similar facility in IDCAMS. PDSE does very similarly for PDSE members. Can you use PDSE members? I regularly make use of this using FTP, actually, to install updated versions of certain files: PUT updated-file tempname RENAME tempname static-name A couple times an observant user has spotted the tempname file while the operation was progress and asked me to explain its existence. But he never had to deal with an incomplete file. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- This is clearly another case of too many mad scientists, and not enough hunchbacks. Maranatha! John McKown -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
At 21:25 -0700 on 10/24/2013, Phil Smith wrote about GDG question: If a jobstep runs with a DD for a GDG that's got DISP=MOD, and the jobstep reads but never tries to write the file, does a new, zero-length GDG get created? I'm of course hoping the answer is No. The only time a new GDG dataset is created is when you use a DSN of GDG(+x) [GDG(-x) are older generations]. In your case you are asking for GDG(0) so you can update it without creating a new generation. Note however that DISP=MOD is the same as DISP=OLD so far as that jobstep having exclusive access to the dataset (so no other job step can read it). The difference between DISP=OLD and DISP=MOD for an existent dataset is writes to the former erase the data (ie: Only the data you write will be there) while the later leaves the old data and adds the new to the end of the dataset. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: GDG question
Phil, AFAIK, the whole GDG assignment only happens once.. during initial creation. MODing will change the size of the data set.. not the GVoo number(s). With SMS a new GDG entry gets rolled in at step end. Most everything you need to know is in Chapter 29. Processing Generation Data Groups of the DFSMS Using Data Sets book. The link is http://pic.dhe.ibm.com/infocenter/zos/v1r13/index.jsp?topic=%2Fcom.ibm.zos.r13.idad400%2Fch14.htm The only thing I didn't see in there is a detailed explanation of the GDG roll-over when counting past 9 generations. There are a couple of old APARS that go into great detail on how the process works and what to expect. Rob Schramm Rob Schramm Senior Systems Consultant Imperium Group On Fri, Oct 25, 2013 at 12:25 AM, Phil Smith p...@voltage.com wrote: Apologies for the naïve question, but I spent some time Googling and reading and didn't find a clear answer to this. We have a data set that gets fetched from the network and changes occasionally. I'd like to untie this update from normal operation-that is, on the off-chance that the network is down at the precise instant that we check for the data set, I'd like things to continue operating. So I'm thinking that if we cache a copy of the data set on disk (the contents aren't sensitive), we could try to fetch; if we can't fetch, we shrug and continue; if we can, we compare the data read with the existing data (there's a timestamp) and only rewrite if it's changed. But the data is also shared across tasks, so we don't want a window where it's half-written and some task tries to read it. A GDG seems like it would work great for this: the number of generations could be defined as low as two, so the update would write the other copy, and the next time one of the tasks tries to read it, it would get the newer one. So here's my question (aside from the obvious one of Am I missing something above that makes this dumb): If a jobstep runs with a DD for a GDG that's got DISP=MOD, and the jobstep reads but never tries to write the file, does a new, zero-length GDG get created? I'm of course hoping the answer is No. Thanks, -- ...phsiii -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
