Re: Anyone exploiting ZEDC?
I did zBNA study for few environments and every time I see increase (or very small decrease) in CPU seconds usage. I see big difference in IO delta and considerable saving in DASD storage but not much difference in CPU time? Or zBNA includes only batch jobs and no other beneficiary started tasks like HSM, CICS, DB2 etc Thank you, Prashant -Original Message- From: IBM Mainframe Discussion List On Behalf Of Glenn Wilcock Sent: Thursday, April 18, 2024 2:34 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [EXTERNAL] Re: Anyone exploiting ZEDC? DFSMShsm is an excellent use case for zEDC and is our number one best practice for HSM. When enabled, DSS zEDC compresses all blocks of data passed to HSM during Migration and Backup. Because HSM is processing fewer data blocks, both cpu and elapsed time are reduced. When going to ML1, the amount of storage is also significantly reduced. Glenn Wilcock, DFSMS Chief Product Owner -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: VBS file read in windows - end of record issue
Michael Stein,
You are right. Thank very much for pointing to the mistake.
I was trying to read file using "with open as". This way I was expecting to
pull single record at every iteration of loop.
With your suggestion, I opened entire file in memory and read each record based
on record length in RDW. Now I can read each record as expected.
Once again thank you to all member for taking time to read my query and
responding. Its good learning.
Thank you,
Prashant Joshi
-Original Message-
From: IBM Mainframe Discussion List On Behalf Of
Michael Stein
Sent: Tuesday, May 16, 2023 11:56 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: VBS file read in windows - end of record issue
On Tue, May 16, 2023 at 04:14:07AM +, Prashant Joshi wrote:
>> Did you specify binary mode on the python open call? --
> Yes. And I can read the data.
How are you reading the data. Assuming an open like:
myfile = open("filename", "rb")
You need to either read it all into memory:
alldata = myfile.read()
or read specific lengths which is messier as you need to read specific lengths,
first 4 bytes for the RDW and then the length of the record in the RDW-4 (as
you already read the RDW).
The 4 byte RDW includes the length of the record in the first 2 bytes
(bigendian order) and the spanning bits in the last 2 bytes.
Either way you need to walk your way through the binary data, any code looking
for CR or NL or space isn't correct.
A description of VBS records formats:
z/OS 2.4 DFSMS Using Data Sets SC23-6855-40
https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R4sc236855/$file/idad400_v2r4.pdf
pdf page 273 Variable-Length Record Formats (near bottom of page and continues
on to more pages)
> its just that I don't get proper end of reord. Hence every time I read
> record, I get random record length (multiple records/half records
> combined)
There aren't any "end of records" in a VBS file. At the begining of the file
you know you are at the start of a RDW (Well, BDW/RDW but I'm assuming the FTP
removed the BDWs).
You can find the next record by going the length specified in the RDW into the
file -- that is the start of the next RDW. Continue until you've processed all
the records.
More help will likely require you to show some code and/or data so we can see
what is going on...
--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
lists...@listserv.ua.edu with the message: INFO IBM-MAIN
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: VBS file read in windows - end of record issue
Hello Paul, "ignores 0D0A" means it treat is as any other data. And by "line break" I mean end of line/record and so next record will start from new line. Thank you, Prashant Joshi -Original Message- From: IBM Mainframe Discussion List On Behalf Of Paul Gilmartin Sent: Tuesday, May 16, 2023 11:52 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [EXTERNAL] Re: VBS file read in windows - end of record issue On Tue, 16 May 2023 04:45:32 +, Prashant Joshi wrote: >... If it completely ignores 0D0A and treat entire file as single > record, > By "ignores 0D0A" do you mean it deletes them, or treats them as ordinary characters? >I... My problem is, it adds random line break and so when I read each >record in loop, > What's a "line break"? I don't know Python, but it might help others if you show your code. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: VBS file read in windows - end of record issue
Thank you for your response, Charles, I preserve RDW and able to read record length field. I am also able to successfully covert EBCDIC to ASCII and Packed to decimal. So there is no issue with data conversion. As you correctly point out, intel/windows does not understand 0D0A. If it completely ignores 0D0A and treat entire file as single record, I am still ok with that as I can read records, based on record length field in RDW. My problem is, it adds random line break and so when I read each record in loop, it returns me records with record length between 1 to thousands. My records are in range of 150 to 250 length. Thanks, Prashant -Original Message- From: IBM Mainframe Discussion List On Behalf Of Charles Mills Sent: Tuesday, May 16, 2023 5:33 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [EXTERNAL] Re: VBS file read in windows - end of record issue There are so many layers of issues here ... If the data is "binary" (any or nearly any byte value is possible) then no approach that involves CRLF is going to work. Period. Software does not have a way to distinguish between a 0D0A that signifies the end of a record versus a 0D0A that just happens to appear in the data -- say a halfword of 3338 decimal. If you preserve the RDW keep in mind that the Z is a "big-endian" machine: x'0050' represents decimal 80, but Intel is "little-endian": decimal 80 is x'5000'. You will have to account for that in processing the data. If the records contain decimal packed data there are not many programs in Windows that can process packed data. If there are character fields in the data then they are going to have to be translated from EBCDIC to ASCII in order to be generally usable. That's a whole topic of its own. I think without a real good understanding of the record conventions on both machines that success is unlikely. Here is how I have done this ftp> quote type i 200 Representation type is Image ftp> quote mode s 200 Data transfer mode is Stream ftp> quote site rdw 200 SITE command was accepted ftp> get zos.vfromat.file Then in my C++ code I read 4 bytes, decode the RDW accounting for the "endian" issue, and then read the rest of the record. Charles On Mon, 15 May 2023 18:24:12 -0500, Michael Oujesky wrote: >Depend on what you need to do and there at least two levels to be addressed: >* How variable length records appear on Windows >* How to deal with SDW (spanned long record segments If you want >both BDW and RDW/SDW present on Windows at least two approaches: >* TERSE the file, binary transfer to windows, WWUNTERSE from Watson >& Walker) on windows (-V option removes BDW) >* FTP binary transfer from z/OS to Windows using //DD: and JCL to >force RDCFM=U Regular FTP binary transfer with "SITE RDW" will give you >the file with just RDW/SDW (BDW is stripped off). Without the "SITE >RDW", you just get the data with no indication of where the record >ends). > >Then there is the issue of how to re-combine the segments into the full >record length. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: VBS file read in windows - end of record issue
Thank you, Michael, I used WWUNTERSE for my terse file. I am able to get RDW & BDW and also able to read data after proper conversion. Problem is when I read file for each record (using loop), I get different record lengths for each record, in range of single digit to thousands. Seems it adds "new line" randomly in file. I can use record length field in RDW to read each records but this random "new line" breaks my loop. I also tried to STRIP (remove additional space from end of record) these records to remove "new line" character and converted entire file as one single record. Now I get only one record and can use record length field in RDW to split file. But then this STRIP option removes some record information as well along with "new line" character. Hence it affects my real record length. Thanks, Prashant -Original Message- From: IBM Mainframe Discussion List On Behalf Of Michael Oujesky Sent: Tuesday, May 16, 2023 4:54 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [EXTERNAL] Re: VBS file read in windows - end of record issue Depend on what you need to do and there at least two levels to be addressed: * How variable length records appear on Windows * How to deal with SDW (spanned long record segments If you want both BDW and RDW/SDW present on Windows at least two approaches: * TERSE the file, binary transfer to windows, WWUNTERSE from Watson & Walker) on windows (-V option removes BDW) * FTP binary transfer from z/OS to Windows using //DD: and JCL to force RDCFM=U Regular FTP binary transfer with "SITE RDW" will give you the file with just RDW/SDW (BDW is stripped off). Without the "SITE RDW", you just get the data with no indication of where the record ends). Then there is the issue of how to re-combine the segments into the full record length. Does that help any? Michael At 01:54 PM 5/15/2023, Prashant Joshi wrote: >I am trying to read VB & VBS binary file in windows using python. I >tried FTPing file directly, then using Terse and then XMIT but every >time when I read the file in windows, I get random new line (CR/LF) >inserted in record. I am not able to read complete record. > >For some files, when I transfer using IND$FILE (option 6) using Binary >and CRLF option checked, I gets record properly. But it does not work >for every file. And due to different file sizes, IND$FILE may not be >option for me. > > > >Need your help to find, what am I missing? > > >Thanks, >Prashant > >-- >For IBM-MAIN subscribe / signoff / archive access instructions, send >email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: VBS file read in windows - end of record issue
Thank for your response, Paul. I agree, CRLF irrelevant to binary files. Surprisingly, only after using CRLF option, I get those binary records in proper record length. I tried to force it to U format but did not work for m. I will try again with different methods. I can preserve RDW and BDW. How to preserve SDW? I did not see mention of SDW in these options. May be SDW is the key to maintain record length. Thank you, Prashant Joshi Mainframe Architect +91 9743 440 503 (Mobile) -Original Message- From: IBM Mainframe Discussion List On Behalf Of Paul Gilmartin Sent: Tuesday, May 16, 2023 1:38 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [EXTERNAL] Re: VBS file read in windows - end of record issue On Mon, 15 May 2023 18:54:40 +, Prashant Joshi wrote: >... >For some files, when I transfer using IND$FILE (option 6) using Binary and >CRLF option checked, I gets record properly. But it does not work for every >file. And due to different file sizes, IND$FILE may not be option for me. > CRLF should be irrelevant for binary files. >Need your help to find, what am I missing? > Allocate your data set overriding to RECFM=U. Tne BDWs and SDWs should appear ad data. Transfer the file with a method that preserves those BDWs and SDWs. (I don't recommend IND$FILE.) Decode the SDWs and translate to UTF-8 with your Python. I hate EBCDIC! -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: VBS file read in windows - end of record issue
Thank you for your response, Michael, 1. FTP -> what options did you use? -- I use Binary and QUOTE SITE RDW 2. Terse & XMIT and then ? how did it get to windows? -- I used WWUNTERSE (>From Watson & Walker) for TERSE and XMIT manager (from CBT tape) for XMIT 3. IND$FILE which you decided isn't really a candidate due to file sizes. -- That's correct Which python: python 2 or python 3? -- Python 3 Did you specify binary mode on the python open call? -- Yes. And I can read the data. Its just that I don't get proper end of record. Hence every time I read record, I get random record length (multiple records/half records combined) Thanks, Prashant -Original Message- From: IBM Mainframe Discussion List On Behalf Of Michael Stein Sent: Tuesday, May 16, 2023 1:37 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: [EXTERNAL] Re: VBS file read in windows - end of record issue On Mon, May 15, 2023 at 06:54:40PM +, Prashant Joshi wrote: > I am trying to read VB & VBS binary file in windows using python. I > tried FTPing file directly, then using Terse and then XMIT but every > time when I read the file in windows, I get random new line (CR/LF) > inserted in record. I am not able to read complete record. Let's see if I understand the problem, here's a summary I see: You tried 3 transfer mechanisms for the z/OS to your windows machine (I'm assuming it's from z/OS since this is the mainframe list). 1. FTP -> what options did you use? 2. Terse & XMIT and then ? how did it get to windows? What do you have on windows which can process this combination of format? 3. IND$FILE which you decided isn't really a candidate due to file sizes. On the windows machine you are trying to read the resulting file with python. Which python: python 2 or python 3? Did you specify binary mode on the python open call? The raw VB & VBS files sound relatively easy to read with python by reading in binary mode and using the BDW/RDW information to separate the records. I'm not sure how you would deal with undoing the XMIT and Terse on windows once you got an intact file there. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
VBS file read in windows - end of record issue
I am trying to read VB & VBS binary file in windows using python. I tried FTPing file directly, then using Terse and then XMIT but every time when I read the file in windows, I get random new line (CR/LF) inserted in record. I am not able to read complete record. For some files, when I transfer using IND$FILE (option 6) using Binary and CRLF option checked, I gets record properly. But it does not work for every file. And due to different file sizes, IND$FILE may not be option for me. Need your help to find, what am I missing? Thanks, Prashant -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
