Copying portions of a huge data set

[Jesse 1 Robinson]

We have a file that contains one month's worth of syslog/operlog data. 
Unfortunately a user's job output has infiltrated this file at random points by 
inappropriate use of MSGCLASS. I want to copy the good data (log stuff) to 
another file and leave the errant user stuff behind. It seems simple, but I 
can't seem to tweak a utility like REPRO (with SKIP and COUNT) to do what I 
want. I've browsed the file and identified by line number where each good data 
starts/ends and where the bad data starts/ends, like this:

01 - log
932964 - job
933148 - log
0001539016 - job
...
0022175585 - job
0022176053 - EOD log

The output file should contain just the 'log' data. Suggestions?

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office <= NEW
robin...@sce.com


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Paul Gilmartin]

On Tue, 25 Jun 2019 20:07:12 +, Jesse 1 Robinson wrote:

>We have a file that contains one month's worth of syslog/operlog data. 
>Unfortunately a user's job output has infiltrated this file at random points 
>by inappropriate use of MSGCLASS. I want to copy the good data (log stuff) to 
>another file and leave the errant user stuff behind. It seems simple, but I 
>can't seem to tweak a utility like REPRO (with SKIP and COUNT) to do what I 
>want. I've browsed the file and identified by line number where each good data 
>starts/ends and where the bad data starts/ends, like this:
>
>01 - log
>932964 - job
>933148 - log
>0001539016 - job
>...
>0022175585 - job
>0022176053 - EOD log
>
>The output file should contain just the 'log' data. Suggestions?
> 
Rexx?

Swiss Army Knife?

I'm confident you'll get a DFSORT suggestion.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Jerry Whitteridge]

Can you only identify the bad data by the line numbers or is there a
keyword in the log entries that you can include/exclude by ?

Jerry Whitteridge
Delivery Manager / Mainframe Architect
GTS - Safeway Account
602 527 4871 Mobile
jerry.whitteri...@ibm.com

IBM Services

IBM Mainframe Discussion List  wrote on
06/25/2019 01:07:12 PM:

> From: Jesse 1 Robinson 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 06/25/2019 01:08 PM
> Subject: [EXTERNAL] Copying portions of a huge data set
> Sent by: IBM Mainframe Discussion List 
>
> We have a file that contains one month's worth of syslog/operlog
> data. Unfortunately a user's job output has infiltrated this file at
> random points by inappropriate use of MSGCLASS. I want to copy the
> good data (log stuff) to another file and leave the errant user
> stuff behind. It seems simple, but I can't seem to tweak a utility
> like REPRO (with SKIP and COUNT) to do what I want. I've browsed the
> file and identified by line number where each good data starts/ends
> and where the bad data starts/ends, like this:
>
> 01 - log
> 932964 - job
> 933148 - log
> 0001539016 - job
> ...
> 0022175585 - job
> 0022176053 - EOD log
>
> The output file should contain just the 'log' data. Suggestions?
>
> .
> .
> J.O.Skip Robinson
> Southern California Edison Company
> Electric Dragon Team Paddler
> SHARE MVS Program Co-Manager
> 323-715-0595 Mobile
> 626-543-6132 Office <= NEW
> robin...@sce.com<mailto:robin...@sce.com>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Sri h Kolusu]

> but I can't seem to tweak a utility  > like REPRO (with SKIP and COUNT)
to do what I want. I've browsed the  file and identified by line number
where each good data starts/ends
 and where the bad data starts/ends, like this:

Skip,

You can use DFSORT's SUBSET operator to copy the records that you want.

RRN(q,r) or RRN(r,q) : Specifies relative record numbers q through r are to
be kept or removed. q can be less than, equal to, or greater than r. For
example, RRN(5,10) and RRN(10,5) both keep or remove the fifth through
tenth records. q and r must be specified as n or +n where n can be 1 to
99.

Here is the documentation and explanation of the parameters of SUBSET
operator along with examples

https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.icea100/ice2ca_Operand_descriptions14.htm


Use the following DFSORT/ICETOOL JCL which will give you the desired
results

//STEP0100 EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//INP  DD DISP=SHR,DSN=Your syslog file
//OUT  DD DSN=Your output log file,
//DISP=(NEW,CATLG,DELETE),
//UNIT=SYSDA,
//SPACE=(CYL,(X,Y),RLSE)
//TOOLIN   DD *
  SUBSET FROM(INP) TO(OUT) KEEP INPUT-
 RRN(01,932963)  -
 RRN(933148,0001539015)  -
 ...
 RRN(0022176053,*)

//*


Further if you have any questions please let me know

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Sri h Kolusu]

> Swiss Army Knife?  I'm confident you'll get a DFSORT suggestion.

DFSORT lived up to your expectations :)

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Farley, Peter x23353]

Remember to use DISP=(MOD,...) for the output DD and also remember that EACH 
REPRO starts over again at record 1, so first few and last REPRO's would look 
like:

REPRO INFILE(A) OUTFILE(B) COUNT(932963)
REPRO INFILE(A) OUTFILE(B) SKIP(933147) COUNT(605868) /* COUNT IS 1539016 - 
933148 */
ETC.
REPRO INDD(A) OUTDD(B) SKIP(22176052) /* NO COUNT, COPY TO EOF */
 
HTH

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jesse 1 Robinson
Sent: Tuesday, June 25, 2019 4:07 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Copying portions of a huge data set

EXTERNAL EMAIL

We have a file that contains one month's worth of syslog/operlog data. 
Unfortunately a user's job output has infiltrated this file at random points by 
inappropriate use of MSGCLASS. I want to copy the good data (log stuff) to 
another file and leave the errant user stuff behind. It seems simple, but I 
can't seem to tweak a utility like REPRO (with SKIP and COUNT) to do what I 
want. I've browsed the file and identified by line number where each good data 
starts/ends and where the bad data starts/ends, like this:

01 - log
932964 - job
933148 - log
0001539016 - job
...
0022175585 - job
0022176053 - EOD log

The output file should contain just the 'log' data. Suggestions?
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Jesse 1 Robinson]

With 22,807,898 lines in the file, it took a lot of 'inspection' to understand 
why our log print program was getting S0C7. The intrusive user junk always 
starts with 'J E S 2  J O B  L O G'. OTOH every true syslog record seems to 
have an alpha character in position 1 that can be found with "f p'@' 1 word". 
Hence the relevant line numbers can be found easily with alternating ISPF 
browse commands. But very hard to turn into a simple algorithm. 

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler 
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office ⇐=== NEW
robin...@sce.com

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jerry Whitteridge
Sent: Tuesday, June 25, 2019 1:56 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Copying portions of a huge data set

Can you only identify the bad data by the line numbers or is there a keyword in 
the log entries that you can include/exclude by ?

Jerry Whitteridge
Delivery Manager / Mainframe Architect
GTS - Safeway Account
602 527 4871 Mobile
jerry.whitteri...@ibm.com

IBM Services

IBM Mainframe Discussion List  wrote on
06/25/2019 01:07:12 PM:

> From: Jesse 1 Robinson 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 06/25/2019 01:08 PM
> Subject: [EXTERNAL] Copying portions of a huge data set Sent by: IBM 
> Mainframe Discussion List 
>
> We have a file that contains one month's worth of syslog/operlog data. 
> Unfortunately a user's job output has infiltrated this file at random 
> points by inappropriate use of MSGCLASS. I want to copy the good data 
> (log stuff) to another file and leave the errant user stuff behind. It 
> seems simple, but I can't seem to tweak a utility like REPRO (with 
> SKIP and COUNT) to do what I want. I've browsed the file and 
> identified by line number where each good data starts/ends and where 
> the bad data starts/ends, like this:
>
> 01 - log
> 932964 - job
> 933148 - log
> 0001539016 - job
> ...
> 0022175585 - job
> 0022176053 - EOD log
>
> The output file should contain just the 'log' data. Suggestions?
>
> .
> .
> J.O.Skip Robinson
> Southern California Edison Company
> Electric Dragon Team Paddler
> SHARE MVS Program Co-Manager
> 323-715-0595 Mobile
> 626-543-6132 Office <= NEW
> robin...@sce.com<mailto:robin...@sce.com>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send 
> email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Jesse 1 Robinson]

As Gil predicted, ICETOOL won the day with assist from Kolusu. Or else the 
other way around. Final control statements: 

SUBSET FROM(INP) TO(OUT) KEEP INPUT-   
   RRN(01,505691)  -   
   RRN(506330,932966)  -   
   RRN(933148,0001539015)  -   
   RRN(0001539401,0004663958)  -   
   RRN(0004664923,0005126853)  -   
   RRN(0005128145,0006102954)  -   
   RRN(0006104030,0006737975)  -   
   RRN(0006739780,0007367171)  -   
   RRN(0007368279,0010077194)  -   
   RRN(0010077486,0011619220)  -   
   RRN(0011619659,0013619681)  -   
   RRN(0013619911,0014619917)  -   
   RRN(0014620377,0022175584)  -   
   RRN(0022176053,*)   

Just like magic. Thanks!!! 

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler 
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office ⇐=== NEW
robin...@sce.com

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of Sri 
h Kolusu
Sent: Tuesday, June 25, 2019 2:34 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Copying portions of a huge data set

> Swiss Army Knife?  I'm confident you'll get a DFSORT suggestion.

DFSORT lived up to your expectations :)

Thanks,
Kolusu
DFSORT Development
IBM Corporation


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Sri h Kolusu]

> As Gil predicted, ICETOOL won the day with assist from Kolusu. Or  else
the other way around. Final control statements:

Skip,

Glad to hear that proposed ICETOOL solution worked for you. :)

>>With 22,807,898 lines in the file, it took a lot of 'inspection' to
understand why our log print program was getting S0C7. The intrusive user
junk always starts with 'J E S 2  J O B  L O G'. OTOH every true syslog
record seems to have an alpha character in position 1 that can be found
with "f p'@' 1 word". Hence the relevant line numbers can be found easily
with alternating ISPF browse commands. But very hard to turn into a simple
algorithm.

Just so you know DFSORT can identify the junk line and eliminate it
programmatically instead of you manually finding the line numbers to be
eliminated.  If you are interested I can show you way to do it. You
mentioned that you can identify the start of junk data. Can you also
identify the end of junk data?  Please send me an offline message with DCB
attributes and I can show you a job to get the desired results

Thanks,
Kolusu
DFSORT Development
IBM Corporation



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Ward Able, Grant]

This may be simplistic, but using REXX & EXECIO, as long as you can identify 
the errant data easily enough, you should be able to get this done fairly 
easily. Maybe not as quick as REPRO, but without much hassle. 




Regards – Grant.

In theory, there's no difference between theory and practice. In practice, 
there is.

There is no such thing as the Cloud. It is just somebody else’s computer.

If you don't have time to do it right, when will you have the time to do it 
over? - John Wooden




DTCC Internal (Green)

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jesse 1 Robinson
Sent: 25 June 2019 23:31
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Copying portions of a huge data set

ATTENTION! This email originated outside of DTCC; exercise caution.

With 22,807,898 lines in the file, it took a lot of 'inspection' to understand 
why our log print program was getting S0C7. The intrusive user junk always 
starts with 'J E S 2  J O B  L O G'. OTOH every true syslog record seems to 
have an alpha character in position 1 that can be found with "f p'@' 1 word". 
Hence the relevant line numbers can be found easily with alternating ISPF 
browse commands. But very hard to turn into a simple algorithm.

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office ⇐=== NEW
robin...@sce.com

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jerry Whitteridge
Sent: Tuesday, June 25, 2019 1:56 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Copying portions of a huge data set

Can you only identify the bad data by the line numbers or is there a keyword in 
the log entries that you can include/exclude by ?

Jerry Whitteridge
Delivery Manager / Mainframe Architect
GTS - Safeway Account
602 527 4871 Mobile
jerry.whitteri...@ibm.com

IBM Services

IBM Mainframe Discussion List  wrote on
06/25/2019 01:07:12 PM:

> From: Jesse 1 Robinson 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 06/25/2019 01:08 PM
> Subject: [EXTERNAL] Copying portions of a huge data set Sent by: IBM 
> Mainframe Discussion List 
>
> We have a file that contains one month's worth of syslog/operlog data.
> Unfortunately a user's job output has infiltrated this file at random 
> points by inappropriate use of MSGCLASS. I want to copy the good data 
> (log stuff) to another file and leave the errant user stuff behind. It 
> seems simple, but I can't seem to tweak a utility like REPRO (with 
> SKIP and COUNT) to do what I want. I've browsed the file and 
> identified by line number where each good data starts/ends and where 
> the bad data starts/ends, like this:
>
> 01 - log
> 932964 - job
> 933148 - log
> 0001539016 - job
> ...
> 0022175585 - job
> 0022176053 - EOD log
>
> The output file should contain just the 'log' data. Suggestions?
>
> .
> .
> J.O.Skip Robinson
> Southern California Edison Company
> Electric Dragon Team Paddler
> SHARE MVS Program Co-Manager
> 323-715-0595 Mobile
> 626-543-6132 Office <= NEW
> robin...@sce.com<mailto:robin...@sce.com>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send 
> email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN
DTCC DISCLAIMER: This email and any files transmitted with it are confidential 
and intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error, please notify us 
immediately and delete the email and any attachments from your system. The 
recipient should check this email and any attachments for the presence of 
viruses.  The company accepts no liability for any damage caused by any virus 
transmitted by this email.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Copying portions of a huge data set

[Jesse 1 Robinson]

As I said earlier, I can 'browse forward' in the corrupted file to identify the 
beginning of each 'good' segment (syslog) and the beginning of each 'bad' 
segment (user job). Pointing to some random spot in the file may not tell me 
much about where I am at the moment. 

Ironically the bad segments (user output) all seem to end this way:

ICE052I 0 END OF DFSORT  

I see that as just a coincidence of what the user is doing. I wouldn't try to 
count on that for a general case. It turns out we can prevent this problem in 
the future by turning off access to a RACF resource in JES2 Exit 6. That should 
have been done many years ago. 

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler 
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office ⇐=== NEW
robin...@sce.com

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Ward Able, Grant
Sent: Wednesday, June 26, 2019 3:49 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Copying portions of a huge data set

This may be simplistic, but using REXX & EXECIO, as long as you can identify 
the errant data easily enough, you should be able to get this done fairly 
easily. Maybe not as quick as REPRO, but without much hassle. 




Regards – Grant.

In theory, there's no difference between theory and practice. In practice, 
there is.

There is no such thing as the Cloud. It is just somebody else’s computer.

If you don't have time to do it right, when will you have the time to do it 
over? - John Wooden




DTCC Internal (Green)

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jesse 1 Robinson
Sent: 25 June 2019 23:31
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Copying portions of a huge data set

ATTENTION! This email originated outside of DTCC; exercise caution.

With 22,807,898 lines in the file, it took a lot of 'inspection' to understand 
why our log print program was getting S0C7. The intrusive user junk always 
starts with 'J E S 2  J O B  L O G'. OTOH every true syslog record seems to 
have an alpha character in position 1 that can be found with "f p'@' 1 word". 
Hence the relevant line numbers can be found easily with alternating ISPF 
browse commands. But very hard to turn into a simple algorithm.

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office ⇐=== NEW
robin...@sce.com

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jerry Whitteridge
Sent: Tuesday, June 25, 2019 1:56 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Copying portions of a huge data set

Can you only identify the bad data by the line numbers or is there a keyword in 
the log entries that you can include/exclude by ?

Jerry Whitteridge
Delivery Manager / Mainframe Architect
GTS - Safeway Account
602 527 4871 Mobile
jerry.whitteri...@ibm.com

IBM Services

IBM Mainframe Discussion List  wrote on
06/25/2019 01:07:12 PM:

> From: Jesse 1 Robinson 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 06/25/2019 01:08 PM
> Subject: [EXTERNAL] Copying portions of a huge data set Sent by: IBM 
> Mainframe Discussion List 
>
> We have a file that contains one month's worth of syslog/operlog data.
> Unfortunately a user's job output has infiltrated this file at random 
> points by inappropriate use of MSGCLASS. I want to copy the good data 
> (log stuff) to another file and leave the errant user stuff behind. It 
> seems simple, but I can't seem to tweak a utility like REPRO (with 
> SKIP and COUNT) to do what I want. I've browsed the file and 
> identified by line number where each good data starts/ends and where 
> the bad data starts/ends, like this:
>
> 01 - log
> 932964 - job
> 933148 - log
> 0001539016 - job
> ...
> 0022175585 - job
> 0022176053 - EOD log
>
> The output file should contain just the 'log' data. Suggestions?
>
> .
> .
> J.O.Skip Robinson
> Southern California Edison Company
> Electric Dragon Team Paddler
> SHARE MVS Program Co-Manager
> 323-715-0595 Mobile
> 626-543-6132 Office <= NEW
> robin...@sce.com<mailto:robin...@sce.com>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send 
> email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions, send

Re: Copying portions of a huge data set

[Ron Hawkins]

I would have thought SAS is your friend.


RON HAWKINS
Director, Ipsicsopt Pty Ltd (ACN: 627 705 971)
m+61 400029610| t: +1 4085625415 | f: +1 4087912585

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jesse 1 Robinson
Sent: Thursday, 27 June 2019 07:44
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] Copying portions of a huge data set

As I said earlier, I can 'browse forward' in the corrupted file to identify the 
beginning of each 'good' segment (syslog) and the beginning of each 'bad' 
segment (user job). Pointing to some random spot in the file may not tell me 
much about where I am at the moment. 

Ironically the bad segments (user output) all seem to end this way:

ICE052I 0 END OF DFSORT  

I see that as just a coincidence of what the user is doing. I wouldn't try to 
count on that for a general case. It turns out we can prevent this problem in 
the future by turning off access to a RACF resource in JES2 Exit 6. That should 
have been done many years ago. 

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office ⇐=== NEW
robin...@sce.com

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Ward Able, Grant
Sent: Wednesday, June 26, 2019 3:49 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Copying portions of a huge data set

This may be simplistic, but using REXX & EXECIO, as long as you can identify 
the errant data easily enough, you should be able to get this done fairly 
easily. Maybe not as quick as REPRO, but without much hassle. 




Regards – Grant.

In theory, there's no difference between theory and practice. In practice, 
there is.

There is no such thing as the Cloud. It is just somebody else’s computer.

If you don't have time to do it right, when will you have the time to do it 
over? - John Wooden




DTCC Internal (Green)

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jesse 1 Robinson
Sent: 25 June 2019 23:31
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Copying portions of a huge data set

ATTENTION! This email originated outside of DTCC; exercise caution.

With 22,807,898 lines in the file, it took a lot of 'inspection' to understand 
why our log print program was getting S0C7. The intrusive user junk always 
starts with 'J E S 2  J O B  L O G'. OTOH every true syslog record seems to 
have an alpha character in position 1 that can be found with "f p'@' 1 word". 
Hence the relevant line numbers can be found easily with alternating ISPF 
browse commands. But very hard to turn into a simple algorithm.

.
.
J.O.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler
SHARE MVS Program Co-Manager
323-715-0595 Mobile
626-543-6132 Office ⇐=== NEW
robin...@sce.com

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Jerry Whitteridge
Sent: Tuesday, June 25, 2019 1:56 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: (External):Re: Copying portions of a huge data set

Can you only identify the bad data by the line numbers or is there a keyword in 
the log entries that you can include/exclude by ?

Jerry Whitteridge
Delivery Manager / Mainframe Architect
GTS - Safeway Account
602 527 4871 Mobile
jerry.whitteri...@ibm.com

IBM Services

IBM Mainframe Discussion List  wrote on
06/25/2019 01:07:12 PM:

> From: Jesse 1 Robinson 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date: 06/25/2019 01:08 PM
> Subject: [EXTERNAL] Copying portions of a huge data set Sent by: IBM 
> Mainframe Discussion List 
>
> We have a file that contains one month's worth of syslog/operlog data.
> Unfortunately a user's job output has infiltrated this file at random 
> points by inappropriate use of MSGCLASS. I want to copy the good data 
> (log stuff) to another file and leave the errant user stuff behind. It 
> seems simple, but I can't seem to tweak a utility like REPRO (with 
> SKIP and COUNT) to do what I want. I've browsed the file and 
> identified by line number where each good data starts/ends and where 
> the bad data starts/ends, like this:
>
> 01 - log
> 932964 - job
> 933148 - log
> 0001539016 - job
> ...
> 0022175585 - job
> 0022176053 - EOD log
>
> The output file should contain just the 'log' data. Suggestions?
>
> .
> .
> J.O.Skip Robinson
> Southern California Edison Company
> Electric Dragon Team Paddler
> SHARE MVS Program Co-Manager
> 323-715-0595 Mobile
> 626-543-6132 Office <= NEW
> robin...@sce.com<mailto:robin...@sce.com>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send 
> email to lists...@listserv.ua.